aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • DevOps
  • Engineering

Ensuring Scale And Compliance Of Your Terraform Deployment With Cloud Build

  • aster.cloud
  • December 11, 2021
  • 3 minute read

Terraform is an open source Infrastructure as Code tool that is popular with platform developers building reusable cloud automation. The Terraform Provider for Google Cloud Platform continues to add support for the latest Google Cloud features, such as Anthos on GKE, and our teams continue to expand Terraform integrations including Cloud Foundation Toolkit and Terraform Validator.

How do teams use Terraform on Google Cloud? While the simplest approach is to run terraform init, plan and apply directly from your terminal,  it cannot be recommended for automating your production deployments. First, there is a decision on how to store your Terraform state in a way that is secure, compliant and enables team collaboration. Secondly there’s a question of scale and reliability. Over the course of even the simplest cloud deployment, Terraform can end up making thousands of Create/Read/Update/Delete API calls to the endpoints used by the Terraform providers, some of which will inevitably hit quota issues or need to be retried for other reasons. For platform administrators, who are looking to ensure the best deployment practices for their curated Terraform solutions,  while benefiting from the simplicity of Google Cloud Console, there’s Terraform Private Catalog integration that we enabled earlier this year.


Partner with aster.cloud
for your next big idea.
Let us know here.



From our partners:

CITI.IO :: Business. Institutions. Society. Global Political Economy.
CYBERPOGO.COM :: For the Arts, Sciences, and Technology.
DADAHACKS.COM :: Parenting For The Rest Of Us.
ZEDISTA.COM :: Entertainment. Sports. Culture. Escape.
TAKUMAKU.COM :: For The Hearth And Home.
ASTER.CLOUD :: From The Cloud And Beyond.
LIWAIWAI.COM :: Intelligence, Inside and Outside.
GLOBALCLOUDPLATFORMS.COM :: For The World's Computing Needs.
FIREGULAMAN.COM :: For The Fire In The Belly Of The Coder.
ASTERCASTER.COM :: Supra Astra. Beyond The Stars.
BARTDAY.COM :: Prosperity For Everyone.

Outside of Private Catalog, Cloud Build and Cloud Storage have been the recommended approach to use Terraform on Google Cloud. Using a remote backend prevents race conditions and simplifies sharing reusable modules between different configurations. With Cloud Build you can configure a GitOps CI/CD pipeline to automatically plan and apply your Terraform configuration when changes are pushed into the repo.

Read More  How To Migrate From Apache HBase To Cloud Bigtable With Live Migrations

These are widely popularized benefits explored in Managing infrastructure as code with Terraform, Cloud Build, and GitOps. In addition, there are lesser known advantages of Cloud Build, particularly for enterprise customers: Cloud Build’s concurrency capabilities and VPC-SC support, Cloud Storage versioning, security and compliance. Let’s explore these benefits in more detail.

Cloud Build’s ability to scale makes it capable to process multiple Terraform deployments across the regions globally and simultaneously. By default, Cloud Build supports 30 concurrent builds, with additional builds queued and processed after the running builds complete. In some cases it may not be enough. Customers who initiate parallel deployments to multiple zones, or, those who provision infrastructure on behalf of multiple tenants, often require running more concurrent deployments to complete all of them within the allotted deployment window. Cloud Build private pool feature allows up to 100 concurrent builds which may be further adjusted upon request. This is an example of creating a private pool and then using it when submitting a build:

gcloud builds worker-pools create $WORKER_POOL_ID \
    --project=$PROJECT_ID \
    --region=$WORKER_POOL_REGION

gcloud builds submit  \
  --worker-pool=projects/${PROJECT_ID}/locations/${WORKER_POOL_REGION}/workerPools/${WORKER_POOL_ID} . \
        --timeout=1200s \
        --config=cloudbuild.yaml \
        --region=$WORKER_POOL_REGION \
        --substitutions=_BUCKET=$BUCKET_NAME,_SOURCE_DIR=./environments/${REGION}/${ZONE}

 

A full step by step example of creating a private pool and submitting 80+ Terraform deployments with Cloud Build simultaneously is available here.

Using Cloud Build removes the need to build a custom high-scale Terraform provisioning service and provides observability and diagnostics for each of the build instances launched and their results.

Using Cloud Build with private pools enables recommended security features, such as VPC Service Controls that allows setting secure perimeter to protect against data exfiltration, with additional restrictions to further restrict it to using the specified private pools. This makes it unnecessary to configure a dedicated bastion host inside the perimeter, which improves the overall security posture.

Read More  Using A Subflow In Dialogflow CX

Beyond just using Cloud Storage for remote storage, additional reasons to use Cloud Storage include versioning, security and compliance. Enabling versioning protects against state file corruption and allows you to view earlier versions. Versioning can be enabled with gsutil command:

$ gsutil versioning set on gs://my-tf-bucket
Enabling versioning for gs://my-tf-bucket...

 

In addition to versioning, you can use Customer-Supplied Encryption Keys to encrypt the Terraform state file. After you generated the key you can specify it as encryption_key parameter of your backend object:

terraform {
  backend "gcs" {
    bucket  = "my-tf-bucket"
    prefix  = "myprefix"

    encryption_key = "xOJ22WdqRNsVssRxxtnKSGoPka6auCyfWiob1KQfs1k="
  }
}

 

Once encrypted you can still view the contents of your state by adding encryption_key option to boto configuration file.

Finally, Cloud Storage is one of the Google Cloud services covered by FedRAMP High, which is important for enterprises  that are seeking their own FedRAMP on top of Google Cloud (for more details see Compliance resource center).

To summarize, using Cloud Build and Cloud Storage for your Terraform deployments enable high scalability, security and compliance with simpler configuration and via familiar gcloud and Google Cloud console interface. Please check out this sample for step by step guidance.

 

 

By: Alex Bulankou (Engineering Manager)
Source: Google Cloud Blog


For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!

Our humans need coffee too! Your support is highly appreciated, thank you!

aster.cloud

Related Topics
  • Cloud Build
  • devops
  • Encryption
  • Google Cloud
  • SRE
  • Terraform
You May Also Like
View Post
  • Engineering

Just make it scale: An Aurora DSQL story

  • May 29, 2025
View Post
  • Engineering
  • Technology

Guide: Our top four AI Hypercomputer use cases, reference architectures and tutorials

  • March 9, 2025
View Post
  • Computing
  • Engineering

Why a decades old architecture decision is impeding the power of AI computing

  • February 19, 2025
View Post
  • Engineering
  • Software Engineering

This Month in Julia World

  • January 17, 2025
View Post
  • Engineering
  • Software Engineering

Google Summer of Code 2025 is here!

  • January 17, 2025
View Post
  • Data
  • Engineering

Hiding in Plain Site: Attackers Sneaking Malware into Images on Websites

  • January 16, 2025
View Post
  • Computing
  • Design
  • Engineering
  • Technology

Here’s why it’s important to build long-term cryptographic resilience

  • December 24, 2024
IBM and Ferrari Premium Partner
View Post
  • Data
  • Engineering

IBM Selected as Official Fan Engagement and Data Analytics Partner for Scuderia Ferrari HP

  • November 7, 2024

Stay Connected!
LATEST
  • 1
    Just make it scale: An Aurora DSQL story
    • May 29, 2025
  • 2
    Reliance on US tech providers is making IT leaders skittish
    • May 28, 2025
  • Examine the 4 types of edge computing, with examples
    • May 28, 2025
  • AI and private cloud: 2 lessons from Dell Tech World 2025
    • May 28, 2025
  • 5
    TD Synnex named as UK distributor for Cohesity
    • May 28, 2025
  • Weigh these 6 enterprise advantages of storage as a service
    • May 28, 2025
  • 7
    Broadcom’s ‘harsh’ VMware contracts are costing customers up to 1,500% more
    • May 28, 2025
  • 8
    Pulsant targets partner diversity with new IaaS solution
    • May 23, 2025
  • 9
    Growing AI workloads are causing hybrid cloud headaches
    • May 23, 2025
  • Gemma 3n 10
    Announcing Gemma 3n preview: powerful, efficient, mobile-first AI
    • May 22, 2025
about
Hello World!

We are aster.cloud. We’re created by programmers for programmers.

Our site aims to provide guides, programming tips, reviews, and interesting materials for tech people and those who want to learn in general.

We would like to hear from you.

If you have any feedback, enquiries, or sponsorship request, kindly reach out to us at:

[email protected]
Most Popular
  • 1
    Cloud adoption isn’t all it’s cut out to be as enterprises report growing dissatisfaction
    • May 15, 2025
  • 2
    Hybrid cloud is complicated – Red Hat’s new AI assistant wants to solve that
    • May 20, 2025
  • 3
    Google is getting serious on cloud sovereignty
    • May 22, 2025
  • oracle-ibm 4
    Google Cloud and Philips Collaborate to Drive Consumer Marketing Innovation and Transform Digital Asset Management with AI
    • May 20, 2025
  • notta-ai-header 5
    Notta vs Fireflies: Which AI Transcription Tool Deserves Your Attention in 2025?
    • May 16, 2025
  • /
  • Technology
  • Tools
  • About
  • Contact Us

Input your search keywords and press Enter.