Managing Cloud Build with Terraform

Learn how to set up and manage Google Cloud Build using Terraform

In this guide, we’ll explore how to manage Google Cloud Build using Terraform.

Video Tutorial

Learn more about managing Google Cloud Build with Terraform in this comprehensive video tutorial:

Prerequisites

  • Google Cloud SDK installed and configured
  • Terraform installed (version 1.0.0 or later)
  • A GCP project with billing enabled

Project Structure

.
├── main.tf                   # Main Terraform configuration file
├── variables.tf              # Variable definitions
├── outputs.tf               # Output definitions
├── terraform.tfvars         # Variable values
└── modules/
    └── cloudbuild/
        ├── main.tf          # Cloud Build specific configurations
        ├── variables.tf      # Module variables
        ├── triggers.tf      # Build trigger configurations
        └── outputs.tf       # Module outputs

Provider Configuration

terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 4.0"
    }
  }
}

provider "google" {
  project = var.project_id
  region  = var.region
}

Variables

variable "project_id" {
  description = "The ID of the GCP project"
  type        = string
}

variable "region" {
  description = "The region to deploy resources to"
  type        = string
  default     = "us-central1"
}

variable "github_owner" {
  description = "GitHub repository owner"
  type        = string
}

variable "github_repo" {
  description = "GitHub repository name"
  type        = string
}

Cloud Build Trigger Configuration

resource "google_cloudbuild_trigger" "github_trigger" {
  name        = "github-trigger"
  description = "Build and deploy on GitHub push"

  github {
    owner = var.github_owner
    name  = var.github_repo
    push {
      branch = "^main$"
    }
  }

  build {
    step {
      name = "gcr.io/cloud-builders/docker"
      args = [
        "build",
        "-t",
        "gcr.io/$PROJECT_ID/${var.github_repo}:$COMMIT_SHA",
        "."
      ]
    }

    step {
      name = "gcr.io/cloud-builders/docker"
      args = [
        "push",
        "gcr.io/$PROJECT_ID/${var.github_repo}:$COMMIT_SHA"
      ]
    }

    step {
      name = "gcr.io/cloud-builders/gke-deploy"
      args = [
        "run",
        "--filename=k8s/",
        "--location=${var.region}",
        "--cluster=my-cluster"
      ]
    }

    artifacts {
      images = ["gcr.io/$PROJECT_ID/${var.github_repo}:$COMMIT_SHA"]
    }
  }
}

Cloud Build Worker Pool Configuration

resource "google_cloudbuild_worker_pool" "pool" {
  name     = "custom-pool"
  location = var.region
  
  worker_config {
    disk_size_gb   = 100
    machine_type   = "e2-standard-4"
    no_external_ip = false
  }

  network_config {
    peered_network = google_compute_network.vpc_network.id
  }
}

Private Pool Configuration

resource "google_cloudbuild_worker_pool" "private_pool" {
  name     = "private-pool"
  location = var.region
  
  worker_config {
    disk_size_gb   = 100
    machine_type   = "e2-standard-4"
    no_external_ip = true
  }

  # Private pools reach your VPC via peered_network (VPC peering) directly -
  # there's no separate VPC Access Connector involved, that's a different
  # mechanism used for connecting serverless products like Cloud Run/Cloud
  # Functions to a VPC, not Cloud Build worker pools.
  network_config {
    peered_network = google_compute_network.vpc_network.id
  }
}

IAM Configuration

resource "google_service_account" "cloudbuild_sa" {
  account_id   = "cloudbuild-service-account"
  display_name = "Cloud Build Service Account"
}

resource "google_project_iam_member" "cloudbuild_sa_roles" {
  for_each = toset([
    "roles/cloudbuild.builds.builder",
    "roles/container.developer",
    "roles/storage.admin"
  ])
  
  project = var.project_id
  role    = each.key
  member  = "serviceAccount:${google_service_account.cloudbuild_sa.email}"
}

Notification Configuration

There’s no dedicated Terraform resource for Cloud Build notifications. Cloud Build automatically publishes every build’s status to a Pub/Sub topic Google creates for you named cloud-builds in the project - you don’t declare that topic yourself. To act on build events, subscribe to it directly, or deploy one of Google’s notifier container images (Slack, HTTP, SMTP) to Cloud Run, configured via a YAML file in GCS:

resource "google_pubsub_subscription" "build_notification_subscription" {
  name  = "cloud-build-notification-subscription"
  topic = "projects/${var.project_id}/topics/cloud-builds"

  ack_deadline_seconds = 20

  push_config {
    push_endpoint = "https://example.com/notifications"
  }
}

Build Configuration File

resource "local_file" "cloudbuild_config" {
  filename = "cloudbuild.yaml"
  content  = <<EOF
steps:
  # Build the container image
  - name: 'gcr.io/cloud-builders/docker'
    args: ['build', '-t', 'gcr.io/$PROJECT_ID/app:$COMMIT_SHA', '.']
  
  # Push the container image
  - name: 'gcr.io/cloud-builders/docker'
    args: ['push', 'gcr.io/$PROJECT_ID/app:$COMMIT_SHA']
  
  # Deploy to GKE
  - name: 'gcr.io/cloud-builders/gke-deploy'
    args:
    - run
    - --filename=k8s/
    - --location=${var.region}
    - --cluster=my-cluster

# Store images in Google Container Registry
images:
  - 'gcr.io/$PROJECT_ID/app:$COMMIT_SHA'

# Configure the timeout
timeout: '1800s'

# Configure the machine type
options:
  machineType: 'N1_HIGHCPU_8'
EOF
}

Outputs

output "github_trigger_id" {
  value       = google_cloudbuild_trigger.github_trigger.id
  description = "The ID of the GitHub trigger"
}

output "worker_pool_id" {
  value       = google_cloudbuild_worker_pool.pool.id
  description = "The ID of the worker pool"
}

output "build_notification_subscription_id" {
  value       = google_pubsub_subscription.build_notification_subscription.id
  description = "The ID of the build notification subscription"
}

Best Practices

  1. Build Configuration:

    • Use caching
    • Optimize steps
    • Parallel execution
    • Regular cleanup

  2. Security:

    • Use service accounts
    • Limit permissions
    • Regular audits
    • Monitor access

  3. Performance:

    • Optimize builds
    • Use worker pools
    • Monitor times
    • Regular review

  4. Cost Optimization:

    • Monitor usage
    • Clean up artifacts
    • Optimize builds
    • Regular review

Common Operations

Creating Resources

terraform init
terraform plan
terraform apply

Build Operations

# Submit a build
gcloud builds submit --config=cloudbuild.yaml .

# List builds
gcloud builds list

# Cancel build
gcloud builds cancel BUILD_ID

Best Practices and Tips

  1. Build Management:

    • Optimize steps
    • Use caching
    • Regular cleanup
    • Monitor times

  2. Security:

    • Limit access
    • Review permissions
    • Regular audits
    • Monitor usage

  3. Operations:

    • Monitor builds
    • Track metrics
    • Set up alerts
    • Regular maintenance

Conclusion

You’ve learned how to set up and manage Google Cloud Build using Terraform. This setup provides:

  • Automated builds
  • Custom worker pools
  • Notifications
  • Best practices implementation