aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • DevOps
  • Engineering

Ensuring Scale And Compliance Of Your Terraform Deployment With Cloud Build

  • aster.cloud
  • December 11, 2021
  • 3 minute read

Terraform is an open source Infrastructure as Code tool that is popular with platform developers building reusable cloud automation. The Terraform Provider for Google Cloud Platform continues to add support for the latest Google Cloud features, such as Anthos on GKE, and our teams continue to expand Terraform integrations including Cloud Foundation Toolkit and Terraform Validator.

How do teams use Terraform on Google Cloud? While the simplest approach is to run terraform init, plan and apply directly from your terminal,  it cannot be recommended for automating your production deployments. First, there is a decision on how to store your Terraform state in a way that is secure, compliant and enables team collaboration. Secondly there’s a question of scale and reliability. Over the course of even the simplest cloud deployment, Terraform can end up making thousands of Create/Read/Update/Delete API calls to the endpoints used by the Terraform providers, some of which will inevitably hit quota issues or need to be retried for other reasons. For platform administrators, who are looking to ensure the best deployment practices for their curated Terraform solutions,  while benefiting from the simplicity of Google Cloud Console, there’s Terraform Private Catalog integration that we enabled earlier this year.


Partner with aster.cloud
for your next big idea.
Let us know here.



From our partners:

CITI.IO :: Business. Institutions. Society. Global Political Economy.
CYBERPOGO.COM :: For the Arts, Sciences, and Technology.
DADAHACKS.COM :: Parenting For The Rest Of Us.
ZEDISTA.COM :: Entertainment. Sports. Culture. Escape.
TAKUMAKU.COM :: For The Hearth And Home.
ASTER.CLOUD :: From The Cloud And Beyond.
LIWAIWAI.COM :: Intelligence, Inside and Outside.
GLOBALCLOUDPLATFORMS.COM :: For The World's Computing Needs.
FIREGULAMAN.COM :: For The Fire In The Belly Of The Coder.
ASTERCASTER.COM :: Supra Astra. Beyond The Stars.
BARTDAY.COM :: Prosperity For Everyone.

Outside of Private Catalog, Cloud Build and Cloud Storage have been the recommended approach to use Terraform on Google Cloud. Using a remote backend prevents race conditions and simplifies sharing reusable modules between different configurations. With Cloud Build you can configure a GitOps CI/CD pipeline to automatically plan and apply your Terraform configuration when changes are pushed into the repo.

Read More  How To Succeed With Cloud Computing During An Economic Downturn

These are widely popularized benefits explored in Managing infrastructure as code with Terraform, Cloud Build, and GitOps. In addition, there are lesser known advantages of Cloud Build, particularly for enterprise customers: Cloud Build’s concurrency capabilities and VPC-SC support, Cloud Storage versioning, security and compliance. Let’s explore these benefits in more detail.

Cloud Build’s ability to scale makes it capable to process multiple Terraform deployments across the regions globally and simultaneously. By default, Cloud Build supports 30 concurrent builds, with additional builds queued and processed after the running builds complete. In some cases it may not be enough. Customers who initiate parallel deployments to multiple zones, or, those who provision infrastructure on behalf of multiple tenants, often require running more concurrent deployments to complete all of them within the allotted deployment window. Cloud Build private pool feature allows up to 100 concurrent builds which may be further adjusted upon request. This is an example of creating a private pool and then using it when submitting a build:

gcloud builds worker-pools create $WORKER_POOL_ID \
    --project=$PROJECT_ID \
    --region=$WORKER_POOL_REGION

gcloud builds submit  \
  --worker-pool=projects/${PROJECT_ID}/locations/${WORKER_POOL_REGION}/workerPools/${WORKER_POOL_ID} . \
        --timeout=1200s \
        --config=cloudbuild.yaml \
        --region=$WORKER_POOL_REGION \
        --substitutions=_BUCKET=$BUCKET_NAME,_SOURCE_DIR=./environments/${REGION}/${ZONE}

 

A full step by step example of creating a private pool and submitting 80+ Terraform deployments with Cloud Build simultaneously is available here.

Using Cloud Build removes the need to build a custom high-scale Terraform provisioning service and provides observability and diagnostics for each of the build instances launched and their results.

Using Cloud Build with private pools enables recommended security features, such as VPC Service Controls that allows setting secure perimeter to protect against data exfiltration, with additional restrictions to further restrict it to using the specified private pools. This makes it unnecessary to configure a dedicated bastion host inside the perimeter, which improves the overall security posture.

Read More  How Not To Be A Mediocre Developer!

Beyond just using Cloud Storage for remote storage, additional reasons to use Cloud Storage include versioning, security and compliance. Enabling versioning protects against state file corruption and allows you to view earlier versions. Versioning can be enabled with gsutil command:

$ gsutil versioning set on gs://my-tf-bucket
Enabling versioning for gs://my-tf-bucket...

 

In addition to versioning, you can use Customer-Supplied Encryption Keys to encrypt the Terraform state file. After you generated the key you can specify it as encryption_key parameter of your backend object:

terraform {
  backend "gcs" {
    bucket  = "my-tf-bucket"
    prefix  = "myprefix"

    encryption_key = "xOJ22WdqRNsVssRxxtnKSGoPka6auCyfWiob1KQfs1k="
  }
}

 

Once encrypted you can still view the contents of your state by adding encryption_key option to boto configuration file.

Finally, Cloud Storage is one of the Google Cloud services covered by FedRAMP High, which is important for enterprises  that are seeking their own FedRAMP on top of Google Cloud (for more details see Compliance resource center).

To summarize, using Cloud Build and Cloud Storage for your Terraform deployments enable high scalability, security and compliance with simpler configuration and via familiar gcloud and Google Cloud console interface. Please check out this sample for step by step guidance.

 

 

By: Alex Bulankou (Engineering Manager)
Source: Google Cloud Blog


For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!

Our humans need coffee too! Your support is highly appreciated, thank you!

aster.cloud

Related Topics
  • Cloud Build
  • devops
  • Encryption
  • Google Cloud
  • SRE
  • Terraform
You May Also Like
View Post
  • Engineering
  • Technology

Guide: Our top four AI Hypercomputer use cases, reference architectures and tutorials

  • March 9, 2025
View Post
  • Computing
  • Engineering

Why a decades old architecture decision is impeding the power of AI computing

  • February 19, 2025
View Post
  • Engineering
  • Software Engineering

This Month in Julia World

  • January 17, 2025
View Post
  • Engineering
  • Software Engineering

Google Summer of Code 2025 is here!

  • January 17, 2025
View Post
  • Data
  • Engineering

Hiding in Plain Site: Attackers Sneaking Malware into Images on Websites

  • January 16, 2025
View Post
  • Computing
  • Design
  • Engineering
  • Technology

Here’s why it’s important to build long-term cryptographic resilience

  • December 24, 2024
IBM and Ferrari Premium Partner
View Post
  • Data
  • Engineering

IBM Selected as Official Fan Engagement and Data Analytics Partner for Scuderia Ferrari HP

  • November 7, 2024
View Post
  • Engineering

Transforming the Developer Experience for Every Engineering Role

  • July 14, 2024

Stay Connected!
LATEST
  • college-of-cardinals-2025 1
    The Definitive Who’s Who of the 2025 Papal Conclave
    • May 7, 2025
  • conclave-poster-black-smoke 2
    The World Is Revalidating Itself
    • May 6, 2025
  • 3
    Conclave: How A New Pope Is Chosen
    • April 25, 2025
  • Getting things done makes her feel amazing 4
    Nurturing Minds in the Digital Revolution
    • April 25, 2025
  • 5
    AI is automating our jobs – but values need to change if we are to be liberated by it
    • April 17, 2025
  • 6
    Canonical Releases Ubuntu 25.04 Plucky Puffin
    • April 17, 2025
  • 7
    United States Army Enterprise Cloud Management Agency Expands its Oracle Defense Cloud Services
    • April 15, 2025
  • 8
    Tokyo Electron and IBM Renew Collaboration for Advanced Semiconductor Technology
    • April 2, 2025
  • 9
    IBM Accelerates Momentum in the as a Service Space with Growing Portfolio of Tools Simplifying Infrastructure Management
    • March 27, 2025
  • 10
    Tariffs, Trump, and Other Things That Start With T – They’re Not The Problem, It’s How We Use Them
    • March 25, 2025
about
Hello World!

We are aster.cloud. We’re created by programmers for programmers.

Our site aims to provide guides, programming tips, reviews, and interesting materials for tech people and those who want to learn in general.

We would like to hear from you.

If you have any feedback, enquiries, or sponsorship request, kindly reach out to us at:

[email protected]
Most Popular
  • 1
    IBM contributes key open-source projects to Linux Foundation to advance AI community participation
    • March 22, 2025
  • 2
    Co-op mode: New partners driving the future of gaming with AI
    • March 22, 2025
  • 3
    Mitsubishi Motors Canada Launches AI-Powered “Intelligent Companion” to Transform the 2025 Outlander Buying Experience
    • March 10, 2025
  • PiPiPi 4
    The Unexpected Pi-Fect Deals This March 14
    • March 13, 2025
  • Nintendo Switch Deals on Amazon 5
    10 Physical Nintendo Switch Game Deals on MAR10 Day!
    • March 9, 2025
  • /
  • Technology
  • Tools
  • About
  • Contact Us

Input your search keywords and press Enter.