Continuous Integration and Deployment (CI/CD) for Terraform on AWS

Implementing CI/CD for Infrastructure as Code (IaC) is crucial for maintaining consistent and reliable infrastructure deployments. This guide demonstrates how to set up CI/CD pipelines for Terraform using various tools and best practices.

Video Tutorial

Learn more about implementing CI/CD with Terraform in AWS in this comprehensive video tutorial:

Prerequisites

AWS CLI configured with appropriate permissions
Terraform installed (version 1.0.0 or later)
GitHub account
Basic understanding of CI/CD concepts

Project Structure

terraform-cicd/
├── .github/
│   └── workflows/
│       ├── terraform-plan.yml
│       └── terraform-apply.yml
├── modules/
│   └── vpc/
│       ├── main.tf
│       ├── variables.tf
│       └── outputs.tf
├── environments/
│   ├── dev/
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   └── terraform.tfvars
│   └── prod/
│       ├── main.tf
│       ├── variables.tf
│       └── terraform.tfvars
├── scripts/
│   ├── terraform-init.sh
│   └── terraform-plan.sh
└── README.md

GitHub Actions Workflow

Create .github/workflows/terraform-plan.yml:

name: 'Terraform Plan'

on:
  pull_request:
    branches:
      - main
    paths:
      - '**.tf'
      - '**.tfvars'
      - '.github/workflows/terraform-*.yml'

jobs:
  terraform-plan:
    name: 'Terraform Plan'
    runs-on: ubuntu-latest
    
    strategy:
      matrix:
        environment: [dev, prod]
        
    defaults:
      run:
        working-directory: environments/${{ matrix.environment }}

    env:
      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      AWS_REGION: us-west-2
      TERRAFORM_VERSION: 1.5.0

    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v2
        with:
          terraform_version: ${{ env.TERRAFORM_VERSION }}

      - name: Terraform Format
        run: terraform fmt -check -recursive

      - name: Terraform Init
        run: terraform init

      - name: Terraform Validate
        run: terraform validate

      - name: Terraform Plan
        run: terraform plan -no-color
        continue-on-error: true

      - name: Update Pull Request
        uses: actions/github-script@v6
        if: github.event_name == 'pull_request'
        env:
          PLAN: "terraform\n${{ steps.plan.outputs.stdout }}"
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          script: |
            const output = `#### Terraform Plan 📝
            *Environment: ${{ matrix.environment }}*
            
            <details><summary>Show Plan</summary>
            
            \`\`\`\n
            ${process.env.PLAN}
            \`\`\`
            
            </details>`;
              
            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: output
            })

Create .github/workflows/terraform-apply.yml:

name: 'Terraform Apply'

on:
  push:
    branches:
      - main
    paths:
      - '**.tf'
      - '**.tfvars'
      - '.github/workflows/terraform-*.yml'

jobs:
  terraform-apply:
    name: 'Terraform Apply'
    runs-on: ubuntu-latest
    
    strategy:
      matrix:
        environment: [dev, prod]
        
    defaults:
      run:
        working-directory: environments/${{ matrix.environment }}

    env:
      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      AWS_REGION: us-west-2
      TERRAFORM_VERSION: 1.5.0

    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v2
        with:
          terraform_version: ${{ env.TERRAFORM_VERSION }}

      - name: Terraform Init
        run: terraform init

      - name: Terraform Apply
        if: matrix.environment == 'dev' || github.ref == 'refs/heads/main'
        run: terraform apply -auto-approve

AWS CodePipeline Configuration

Create aws-pipeline.tf:

# S3 bucket for artifacts
resource "aws_s3_bucket" "artifacts" {
  bucket = "${var.project_name}-terraform-artifacts"

  tags = {
    Name        = "${var.project_name}-terraform-artifacts"
    Environment = var.environment
  }
}

# CodeBuild project
resource "aws_codebuild_project" "terraform" {
  name          = "${var.project_name}-terraform-build"
  description   = "Terraform build project"
  build_timeout = "30"
  service_role  = aws_iam_role.codebuild.arn

  artifacts {
    type = "CODEPIPELINE"
  }

  environment {
    compute_type                = "BUILD_GENERAL1_SMALL"
    image                      = "aws/codebuild/amazonlinux2-x86_64-standard:3.0"
    type                       = "LINUX_CONTAINER"
    image_pull_credentials_type = "CODEBUILD"

    environment_variable {
      name  = "TERRAFORM_VERSION"
      value = "1.5.0"
    }
  }

  source {
    type      = "CODEPIPELINE"
    buildspec = "buildspec.yml"
  }

  tags = {
    Name        = "${var.project_name}-terraform-build"
    Environment = var.environment
  }
}

# CodePipeline
resource "aws_codepipeline" "terraform" {
  name     = "${var.project_name}-terraform-pipeline"
  role_arn = aws_iam_role.codepipeline.arn

  artifact_store {
    location = aws_s3_bucket.artifacts.bucket
    type     = "S3"
  }

  stage {
    name = "Source"

    action {
      name             = "Source"
      category         = "Source"
      owner            = "AWS"
      provider         = "CodeStarSourceConnection"
      version          = "1"
      output_artifacts = ["source_output"]

      configuration = {
        ConnectionArn    = aws_codestarconnections_connection.github.arn
        FullRepositoryId = "${var.github_owner}/${var.github_repo}"
        BranchName      = "main"
      }
    }
  }

  stage {
    name = "Plan"

    action {
      name            = "Plan"
      category        = "Build"
      owner           = "AWS"
      provider        = "CodeBuild"
      input_artifacts = ["source_output"]
      version         = "1"

      configuration = {
        ProjectName = aws_codebuild_project.terraform.name
        EnvironmentVariables = jsonencode([
          {
            name  = "TERRAFORM_COMMAND"
            value = "plan"
            type  = "PLAINTEXT"
          }
        ])
      }
    }
  }

  stage {
    name = "Approve"

    action {
      name     = "Approve"
      category = "Approval"
      owner    = "AWS"
      provider = "Manual"
      version  = "1"
    }
  }

  stage {
    name = "Apply"

    action {
      name            = "Apply"
      category        = "Build"
      owner           = "AWS"
      provider        = "CodeBuild"
      input_artifacts = ["source_output"]
      version         = "1"

      configuration = {
        ProjectName = aws_codebuild_project.terraform.name
        EnvironmentVariables = jsonencode([
          {
            name  = "TERRAFORM_COMMAND"
            value = "apply"
            type  = "PLAINTEXT"
          }
        ])
      }
    }
  }
}

BuildSpec Configuration

Create buildspec.yml:

version: 0.2

phases:
  install:
    runtime-versions:
      python: 3.9
    commands:
      - curl -s -qL -o terraform.zip "https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip"
      - unzip terraform.zip
      - mv terraform /usr/local/bin/
      - chmod +x /usr/local/bin/terraform

  pre_build:
    commands:
      - cd environments/${ENVIRONMENT}
      - terraform init

  build:
    commands:
      - |
        if [ "${TERRAFORM_COMMAND}" = "plan" ]; then
          terraform plan -no-color
        elif [ "${TERRAFORM_COMMAND}" = "apply" ]; then
          terraform apply -auto-approve -no-color
        fi

  post_build:
    commands:
      - echo "Terraform ${TERRAFORM_COMMAND} completed on $(date)"

artifacts:
  files:
    - environments/${ENVIRONMENT}/**/*
    - modules/**/*
  name: terraform-artifacts

Testing and Validation

Pre-Commit Hooks

Create .pre-commit-config.yaml:

repos:
  - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: v1.77.1
    hooks:
      - id: terraform_fmt
      - id: terraform_docs
      - id: terraform_tflint
      - id: terraform_validate

  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.4.0
    hooks:
      - id: check-merge-conflict
      - id: end-of-file-fixer
      - id: trailing-whitespace
      - id: check-yaml

Unit Testing

Create test/main_test.go:

package test

import (
    "testing"
    "github.com/gruntwork-io/terratest/modules/terraform"
    "github.com/stretchr/testify/assert"
)

func TestTerraformBasicExample(t *testing.T) {
    terraformOptions := &terraform.Options{
        TerraformDir: "../environments/dev",
        Vars: map[string]interface{}{
            "environment": "test",
        },
    }

    defer terraform.Destroy(t, terraformOptions)
    terraform.InitAndApply(t, terraformOptions)

    output := terraform.Output(t, terraformOptions, "vpc_id")
    assert.NotEmpty(t, output)
}

Best Practices

State Management
- Use remote state (S3 + DynamoDB)
- Enable state locking
- Implement state backups
Security
- Use IAM roles with least privilege
- Encrypt sensitive variables
- Implement proper access controls
Code Organization
- Use consistent structure
- Implement proper versioning
- Document changes

Deployment Strategies

Progressive Deployment
- Deploy to dev first
- Automated testing
- Manual approval for prod
Environment Separation
- Separate state files
- Environment-specific variables
- Isolated IAM roles
Rollback Strategy
- State backups
- Version control
- Automated rollback

Monitoring and Alerting

Pipeline Monitoring
- CloudWatch metrics
- SNS notifications
- Slack integration
Infrastructure Changes
- Resource tracking
- Cost monitoring
- Compliance checks

Conclusion

You’ve learned how to implement CI/CD for Terraform on AWS. This setup provides:

Automated infrastructure deployment
Consistent testing and validation
Secure deployment process
Proper change management

Remember to:

Follow security best practices
Implement proper testing
Document your pipeline
Monitor deployments

Continuous Integration and Deployment (CI/CD) for Terraform on AWS

Continuous Integration and Deployment (CI/CD) for Terraform on AWS

Video Tutorial

Prerequisites

Project Structure

GitHub Actions Workflow

AWS CodePipeline Configuration

BuildSpec Configuration

Testing and Validation

Best Practices

Deployment Strategies

Monitoring and Alerting

Conclusion

We value your privacy

Cookie Preferences

Necessary Cookies

Analytics Cookies

Marketing Cookies