Terraform - Automated Testing, Security & Code Quality

Intro

If you did any kind of software development, you hopefully also wrote some automated tests before. Unit tests are amazing and blazing fast (or at least they should be). Integration or E2E tests are a bit more difficult because you might need to prepare the appropriate environment and components the tests depend on. But still, setting up that environment and components once should last you a long time until you might need another component in the future. If you ran some plans and applies with Terraform in the past, you know that they don't always work as expected. Question is how do we increase the reliability of Terraform tasks in advance with automated tests?

Built-In Terraform Testing And QA

Terraform provides built-in testing features so let's look at those first.

Terraform fmt

We all know that consistent formatting tremendously improves readability. For this purpose we can and should use the terraform fmt command.

terraform fmt

Terraform Validate Command

One of the simplest things we can do is running the Terraform validate command which will validate that our syntax is valid and that our code is internally consistent.

terraform validate

Terraform validate is the least QA you should add to your CI.

Input Validation

Next thing we can do is validate that our inputs meet desired conditions.

variable "aws_private_subnet_ids" {  
  description = "VPC private subnet ids."  
  type        = list(string)  

  validation {    
    condition     = length(var.aws_private_subnet_ids) > 1    
    error_message = "This application requires at least two private subnets."  
  }
}

This will help you validate inputs upon running Terraform plan.

Preconditions

Introduced in Terraform 1.2, preconditions, as the name suggests, can help you validate that desired conditions are met when running Terraform plan.

data "aws_ec2_instance_type" "app" {
  instance_type = var.aws_instance_type
}

resource "aws_instance" "app" {
  count = var.aws_instance_count

  instance_type = var.aws_instance_type
  ami           = var.aws_ami_id

  subnet_id              = var.aws_private_subnet_ids[count.index % length(var.aws_private_subnet_ids)]
  vpc_security_group_ids = [module.app_security_group.security_group_id]

  lifecycle {
    precondition {
      condition     = var.aws_instance_count % length(var.aws_private_subnet_ids) == 0
      error_message = "The number of instances (${var.aws_instance_count}) must be evenly divisible by the number of private subnets (${length(var.aws_private_subnet_ids)})."
    }

    precondition {
      condition     = data.aws_ec2_instance_type.app.ebs_optimized_support != "unsupported"
      error_message = "The EC2 instance type (${var.aws_instance_type}) must support EBS optimization."      
    }
  }
}

Postconditions

Use postconditions to validate that desired conditions are met after we run apply.

data "aws_vpc" "app" {
  id = var.aws_vpc_id

  lifecycle {
    postcondition {
      condition     = self.enable_dns_support == true
      error_message = "The selected VPC must have DNS support enabled."      
    }
  }
}

Pre/Postconditions

Pre and postconditions are great for intercepting errors that would be thrown with generic error messages that might not be so helpful to users. Instead you can use pre/postconditions to intercept those errors and provide more useful, context specific error messages.

Checks And Assertions

Available from Terraform version 1.5, we have checks and assertions at our disposal.

check "health_check" {
  data "http" "terraform_io" {
    url = "https://www.example.com/some/api"
  }

  assert {
    condition = data.http.terraform_io.status_code == 200
    error_message = "${data.http.terraform_io.url} returned an unhealthy status code"
  }
}

Use checks to perform custom checks that will be executed at the end of plan and apply and warn you through error messages of failed checks.

The Terraform Testing Framework

NOTE:
This one is available from Terrafrom version 1.6 which with the new BSL licence instead of the previous MPL 2.0 licence. You should read about it online and see if and how it matters to you.

The Terraform test framework was introduced in version 1.6. It allows you to run tests on real, short lived, resources. Running the tests on real resources is equivalent to integration testing (command = apply) which is the default behavior. You can also run the tests without actual resources as unit tests (command = plan).

# main.tf

provider "aws" {
    region = "eu-central-1"
}

variable "bucket_prefix" {
  type = string
}

resource "aws_s3_bucket" "bucket" {
  bucket = "${var.bucket_prefix}-bucket"
}

output "bucket_name" {
  value = aws_s3_bucket.bucket.bucket
}

# valid_string_concat.tftest.hcl

variables {
  bucket_prefix = "test"
}

run "valid_string_concat" {

  command = plan

  assert {
    condition     = aws_s3_bucket.bucket.bucket == "test-bucket"
    error_message = "S3 bucket name did not match expected"
  }

}

Third-Party Testing and QA

After exploring the built-in testing and QA capabilities of Terraform, let's look at some third-party tools.

Terratest

Terratest is GoLang library that enables us to write tests for our Terraform IaC in the form of Go tests (_test.go). This means you can enjoy a high-level language and all its features with built-in Terraform capabilities like init, plan, apply, reading outputs, etc.

package test

import (
	"testing"

	"github.com/gruntwork-io/terratest/modules/terraform"
	"github.com/stretchr/testify/assert"
)

func TestTerraformHelloWorldExample(t *testing.T) {
	// Construct the terraform options with default retryable errors to handle the most common
	// retryable errors in terraform testing.
	terraformOptions := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
		// Set the path to the Terraform code that will be tested.
		TerraformDir: "../examples/terraform-hello-world-example",
	})

	// Clean up resources with "terraform destroy" at the end of the test.
	defer terraform.Destroy(t, terraformOptions)

	// Run "terraform init" and "terraform apply". Fail the test if there are any errors.
	terraform.InitAndApply(t, terraformOptions)

	// Run `terraform output` to get the values of output variables and check they have the expected values.
	output := terraform.Output(t, terraformOptions, "hello_world")
	assert.Equal(t, "Hello, World!", output)
}

In addition to Terraform, Terratest also enables you to write tests for Dockerfiles, Kubernetes and a few other.

TFLint

TFLint is a Terraform linter that focuses on possible errors, best practices, and style conventions in Terraform code. It enforces coding standards, naming conventions, and checks for deprecated syntax or unused declarations.

docker run --rm -v /my/terraform/code:/data -t ghcr.io/terraform-linters/tflint

Trivy (TFSec)

Trivy is a comprehensive and versatile security scanner. Trivy has scanners that look for security issues, and targets where it can find those issues.

TFSec will continue to remain available for the time being, although our engineering attention will be directed at Trivy going forward.

docker run -v /my/terraform/code:/data aquasec/trivy config /data

Terrascan

Terrascan is a static code analyzer for Infrastructure as Code. Terrascan allows you to seamlessly scan infrastructure as code for misconfigurations, monitor provisioned cloud infrastructure for configuration changes that introduce posture drift, and enables reverting to a secure posture.

docker run -v /my/terraform/code:/data accurics/terrascan scan /data

Chekov

Checkov is a static code analysis tool focused on detecting security misconfigurations and compliance violations in Terraform code. It scans Terraform configurations and evaluates them against a comprehensive set of predefined security and compliance policies.

docker run --rm -v /my/terraform/code:/data bridgecrew/checkov -d /data

Conclusion

You want to increase the code quality, security and compliance of your Terraform IaC? You want to make your IaC deployments more reliable? Great tools are available and, if not already the case, you should be integrating the tools and practices into your Terraform IaC game.