Cloud Infrastructure

Multi-Cloud Infrastructure as Code with Terraform: Lessons Learned

20 April 202514 min readBy Bayseian Engineering

Best practices for managing infrastructure across AWS, GCP, and Azure using Terraform, including state management, modules, and CI/CD integration.

Introduction: Why Multi-Cloud with Terraform?

Managing infrastructure across AWS, Azure, and GCP manually is a recipe for disaster. Terraform enables consistent, repeatable infrastructure deployment across all clouds.

•Single tool, multiple clouds: one HCL syntax and workflow instead of learning CloudFormation, ARM templates, and Deployment Manager separately.
•Infrastructure as Code: changes are version controlled and reviewable in a PR, so infrastructure changes get the same scrutiny as application code.
•State management and drift detection: Terraform's state file is the source of truth, so terraform plan catches when someone made a manual change in the console before it causes an incident.
•Modular and reusable components: write a network or compute module once, parameterize it, and reuse it across environments and clouds.
•Plan before apply (no surprises): terraform plan shows exactly what will change before it happens, turning infrastructure changes from a leap of faith into a reviewed diff.

1.Disaster Recovery: Primary in AWS, failover in Azure
2.Vendor Diversification: Avoid single-vendor lock-in
3.Cost Optimization: Use cheapest region/service for each workload
4.Regulatory Compliance: Data residency requirements
5.Best-of-Breed: Use best service from each cloud

•Level 1: Manual state, no modules (1-2 engineers)
•Level 2: Remote state, basic modules (5-10 engineers)
•Level 3: Workspaces, CI/CD, governance (10-50 engineers)
•Level 4: Platform team, custom modules, policy as code (50+ engineers)

Production infrastructure managed: 2,000+ resources across 3 clouds, 15+ regions.

Write → plan → review → apply → monitor: one workflow, three clouds.

State Management

Terraform state is critical: it tracks your infrastructure and enables collaboration.

Remote State Best Practices:

•S3 for state storage: versioned and encrypted, so a bad apply can be rolled back to the last known-good state file.
•DynamoDB for state locking: prevents two engineers (or two CI runs) from applying concurrently and corrupting state.

•Enable server-side encryption (SSE-S3 or KMS): state files contain resource IDs and often secrets in plaintext; treat the bucket like a secrets store.
•Encrypt sensitive values (passwords, keys) with SOPS: belt-and-suspenders for values that shouldn't be readable even by someone with S3 read access.

•Separate state files per environment (dev/staging/prod): a bad plan in dev should never be able to touch prod resources because they don't share state.
•Use workspaces or separate backends: workspaces are lighter-weight but easier to apply to the wrong one by mistake; separate backends are safer at scale.
•Never share state across unrelated infrastructure: one giant state file means one lock blocks every team and one corruption event takes down everything.

•Enable S3 versioning (rollback capability): every state write is retained, so a corrupted or bad-apply state can be restored to a prior version.
•Periodic state backups to separate location: protects against the bucket itself being deleted or misconfigured.
•Test state recovery process: an untested backup is a guess; actually restore from it before you need to under pressure.

•Restrict state access (IAM policies): anyone who can read state can read every resource ID, IP, and often secret in your infrastructure.
•Separate read/write permissions: most engineers only need to read plans; limit write/apply access to CI and a small operator group.
•Audit state access (CloudTrail): state access logs are often the first place to look when investigating an unexpected change.

Common State Issues:

Problem: State drift (Terraform state != actual infrastructure)
Solution: Run terraform refresh or terraform plan regularly

Problem: State corruption
Solution: Use state locking, enable versioning, keep backups

Problem: Secrets in state
Solution: Use AWS Secrets Manager/Vault, not Terraform variables

Terraform

# Backend configuration for multi-cloud state management

# backend.tf
terraform {
  backend "s3" {
    bucket         = "company-terraform-state"
    key            = "production/multi-cloud/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "terraform-state-lock"
    
    # Enable versioning for rollback
    versioning     = true
    
    # Server-side encryption with KMS
    kms_key_id     = "arn:aws:kms:us-east-1:123456789:key/..."
  }
}

# State locking with DynamoDB
resource "aws_dynamodb_table" "terraform_lock" {
  name           = "terraform-state-lock"
  billing_mode   = "PAY_PER_REQUEST"
  hash_key       = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }

  tags = {
    Name        = "Terraform State Lock"
    Environment = "production"
  }
}

# State bucket with versioning and encryption
resource "aws_s3_bucket" "terraform_state" {
  bucket = "company-terraform-state"

  lifecycle {
    prevent_destroy = true  # Protect state bucket
  }

  tags = {
    Name        = "Terraform State"
    Environment = "production"
  }
}

resource "aws_s3_bucket_versioning" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id

  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm     = "aws:kms"
      kms_master_key_id = aws_kms_key.terraform_state.arn
    }
  }
}

# Block public access
resource "aws_s3_bucket_public_access_block" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

# IAM policy for state access
resource "aws_iam_policy" "terraform_state_access" {
  name        = "TerraformStateAccess"
  description = "Policy for Terraform state operations"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "s3:ListBucket",
          "s3:GetObject",
          "s3:PutObject"
        ]
        Resource = [
          aws_s3_bucket.terraform_state.arn,
          "${aws_s3_bucket.terraform_state.arn}/*"
        ]
      },
      {
        Effect = "Allow"
        Action = [
          "dynamodb:GetItem",
          "dynamodb:PutItem",
          "dynamodb:DeleteItem"
        ]
        Resource = aws_dynamodb_table.terraform_lock.arn
      }
    ]
  })
}

Reusable Modules for Multi-Cloud

Modules are the key to maintainable multi-cloud infrastructure. Build once, deploy everywhere.

Module Structure Best Practices:

•Define a common interface across clouds: the same variables (instance_size, environment) map to whichever cloud's module is called, so callers don't need cloud-specific knowledge.
•Hide cloud-specific details: instance type strings, VM sizing tables, and API quirks stay inside the module, not leaked into every caller.
•Use consistent naming conventions: a resource named the same way across AWS/Azure/GCP modules makes cross-cloud diffs and audits tractable.

•Validate inputs with variable validation blocks: catch a typo'd instance size at plan time, not after a failed apply halfway through a change.
•Provide sensible defaults: most callers shouldn't need to specify every variable; defaults keep module usage terse for the common case.
•Document all variables: undocumented modules become tribal knowledge that only the original author can safely change.

•Return consistent outputs (IDs, endpoints, etc.): downstream modules can consume any cloud's compute module the same way if the output shape matches.
•Include all necessary information for dependent modules: missing an output means the caller has to re-derive or hardcode it, defeating the abstraction.
•Use descriptive output names: instance_id beats id when a root module wires together a dozen modules' outputs.

•Pin module versions in production: an unpinned module source means the next terraform init can silently pull a breaking change.
•Use semantic versioning: lets callers upgrade patch/minor versions confidently while treating majors as a deliberate, reviewed step.
•Test upgrades in lower environments: validate a module version bump in dev/staging before it touches production state.

Module Organization:

modules/
├── compute/
│ ├── aws/ (AWS EC2-specific implementation)
│ ├── azure/ (Azure VM-specific implementation)
│ ├── gcp/ (GCP Compute-specific implementation)
│ └── interface.tf (Common interface)
├── database/
│ ├── aws/ (RDS)
│ ├── azure/ (Azure SQL)
│ └── gcp/ (Cloud SQL)
└── networking/
├── aws/ (VPC)
├── azure/ (VNet)
└── gcp/ (VPC)

Example: Multi-Cloud Compute Module

Terraform

# modules/compute/interface.tf
# Common interface for compute resources across clouds

variable "cloud_provider" {
  description = "Cloud provider (aws, azure, gcp)"
  type        = string

  validation {
    condition     = contains(["aws", "azure", "gcp"], var.cloud_provider)
    error_message = "Provider must be aws, azure, or gcp"
  }
}

variable "instance_size" {
  description = "Instance size (small, medium, large)"
  type        = string
  default     = "medium"

  validation {
    condition     = contains(["small", "medium", "large"], var.instance_size)
    error_message = "Size must be small, medium, or large"
  }
}

variable "environment" {
  description = "Environment (dev, staging, prod)"
  type        = string
}

# modules/compute/aws/main.tf
# AWS-specific implementation

locals {
  instance_types = {
    small  = "t3.medium"
    medium = "t3.large"
    large  = "t3.xlarge"
  }
}

resource "aws_instance" "app" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = local.instance_types[var.instance_size]

  tags = {
    Name        = "${var.environment}-app-server"
    Environment = var.environment
    ManagedBy   = "terraform"
  }

  root_block_device {
    volume_type = "gp3"
    volume_size = 50
    encrypted   = true
  }

  metadata_options {
    http_tokens = "required"  # IMDSv2
  }
}

output "instance_id" {
  value = aws_instance.app.id
}

output "public_ip" {
  value = aws_instance.app.public_ip
}

# modules/compute/azure/main.tf
# Azure-specific implementation

locals {
  vm_sizes = {
    small  = "Standard_B2s"
    medium = "Standard_D2s_v3"
    large  = "Standard_D4s_v3"
  }
}

resource "azurerm_linux_virtual_machine" "app" {
  name                = "${var.environment}-app-vm"
  resource_group_name = var.resource_group_name
  location            = var.location
  size                = local.vm_sizes[var.instance_size]

  admin_username = "adminuser"

  admin_ssh_key {
    username   = "adminuser"
    public_key = var.ssh_public_key
  }

  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Premium_LRS"
    disk_size_gb         = 50
  }

  source_image_reference {
    publisher = "Canonical"
    offer     = "UbuntuServer"
    sku       = "20.04-LTS"
    version   = "latest"
  }

  tags = {
    Environment = var.environment
    ManagedBy   = "terraform"
  }
}

output "instance_id" {
  value = azurerm_linux_virtual_machine.app.id
}

output "public_ip" {
  value = azurerm_linux_virtual_machine.app.public_ip_address
}

# Root module usage - environment/prod/main.tf
module "compute_aws" {
  source = "../../modules/compute/aws"

  cloud_provider = "aws"
  instance_size  = "large"
  environment    = "production"
}

module "compute_azure" {
  source = "../../modules/compute/azure"

  cloud_provider     = "azure"
  instance_size      = "large"
  environment        = "production"
  resource_group_name = azurerm_resource_group.prod.name
  location           = "eastus"
}

CI/CD Pipeline for Terraform

Automated Terraform workflows with GitHub Actions ensure consistent, safe deployments.

CI/CD Best Practices:

•terraform fmt check (code formatting)
•terraform validate (syntax validation)
•Security scan (Checkov, Trivy)
•Cost estimation (Infracost)
•terraform plan (preview changes)
•Comment plan output on PR

•Require PR approval (2+ reviewers)
•Auto-run plan again
•Manual approval for apply
•terraform apply on approval
•Notify on Slack/Teams

•Checkov: policy-as-code validation that catches misconfigurations (public S3 buckets, open security groups) before they're ever applied.
•Trivy: security best-practice scanning. tfsec was merged into Trivy, so migrate CI steps accordingly, since the standalone tfsec action is deprecated.
•OPA/Conftest: enforce org-specific policy rules (tagging standards, allowed instance types) that generic scanners don't know about.
•Prevent merges on critical findings: a scan that only warns gets ignored; block the merge and force a fix or explicit exception.

•Infracost: estimates the dollar cost of a plan before apply, turning "will this be expensive" from a guess into a number on the PR.
•Alert on >20% cost increase: catches runaway resource sizing or an accidentally duplicated module before the bill does.
•Require approval for >$1K/month changes: routes meaningfully expensive changes through a human check, not just an automated scan.

Production Pipeline Example:

YAML

# .github/workflows/terraform.yml
name: 'Terraform CI/CD'

on:
  pull_request:
    paths:
      - 'terraform/**'
      - '.github/workflows/terraform.yml'
  push:
    branches:
      - main
    paths:
      - 'terraform/**'

env:
  TF_VERSION: '1.6.0'
  AWS_REGION: 'us-east-1'

jobs:
  terraform-validate:
    name: 'Validate and Plan'
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: terraform/production

    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v2
        with:
          terraform_version: ${{ env.TF_VERSION }}

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          role-to-assume: ${{ secrets.AWS_TERRAFORM_ROLE_ARN }}
          aws-region: ${{ env.AWS_REGION }}

      - name: Terraform Format Check
        id: fmt
        run: terraform fmt -check -recursive
        continue-on-error: true

      - name: Terraform Init
        id: init
        run: terraform init

      - name: Terraform Validate
        id: validate
        run: terraform validate -no-color

      - name: Run Checkov Security Scan
        id: checkov
        uses: bridgecrewio/checkov-action@master
        with:
          directory: terraform/production
          framework: terraform
          output_format: cli
          soft_fail: false  # Fail on security issues

      - name: Run tfsec
        uses: aquasecurity/tfsec-action@v1.0.0
        with:
          working_directory: terraform/production

      - name: Terraform Plan
        id: plan
        run: terraform plan -no-color -out=tfplan
        continue-on-error: true

      - name: Setup Infracost
        uses: infracost/actions/setup@v2
        with:
          api-key: ${{ secrets.INFRACOST_API_KEY }}

      - name: Generate Cost Estimate
        id: cost
        run: |
          infracost breakdown --path tfplan --format json --out-file /tmp/cost.json
          infracost output --path /tmp/cost.json --format github-comment --out-file /tmp/cost_comment.md

      - name: Comment PR with Plan
        uses: actions/github-script@v6
        if: github.event_name == 'pull_request'
        with:
          script: |
            const fs = require('fs');
            const plan = fs.readFileSync('terraform/production/tfplan.txt', 'utf8');
            const cost = fs.readFileSync('/tmp/cost_comment.md', 'utf8');
            
            const output = `#### Terraform Format and Style 🖌\`${{ steps.fmt.outcome }}\`
            #### Terraform Initialization ⚙️\`${{ steps.init.outcome }}\`
            #### Terraform Validation 🤖\`${{ steps.validate.outcome }}\`
            #### Terraform Plan 📖\`${{ steps.plan.outcome }}\`

            <details><summary>Show Plan</summary>

            \`\`\`terraform
            ${plan}
            \`\`\`

            </details>

            ${cost}

            *Pusher: @${{ github.actor }}, Action: \`${{ github.event_name }}\`, Workflow: \`${{ github.workflow }}\`*`;

            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: output
            });

  terraform-apply:
    name: 'Apply Changes'
    runs-on: ubuntu-latest
    needs: terraform-validate
    if: github.ref == 'refs/heads/main' && github.event_name == 'push'
    environment: production  # Requires manual approval in GitHub
    defaults:
      run:
        working-directory: terraform/production

    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v2
        with:
          terraform_version: ${{ env.TF_VERSION }}

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          role-to-assume: ${{ secrets.AWS_TERRAFORM_ROLE_ARN }}
          aws-region: ${{ env.AWS_REGION }}

      - name: Terraform Init
        run: terraform init

      - name: Terraform Apply
        id: apply
        run: |
          terraform apply -auto-approve -no-color | tee apply.log
          echo "APPLY_OUTPUT<<EOF" >> $GITHUB_ENV
          cat apply.log >> $GITHUB_ENV
          echo "EOF" >> $GITHUB_ENV

      - name: Notify Slack on Success
        if: success()
        uses: slackapi/slack-github-action@v1
        with:
          payload: |
            {
              "text": "- Terraform apply succeeded in production",
              "blocks": [
                {
                  "type": "section",
                  "text": {
                    "type": "mrkdwn",
                    "text": "*Terraform Apply Successful*

Environment: Production
Commit: ${{ github.sha }}
Actor: @${{ github.actor }}"
                  }
                }
              ]
            }
        env:
          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}

      - name: Notify Slack on Failure
        if: failure()
        uses: slackapi/slack-github-action@v1
        with:
          payload: |
            {
              "text": "❌ Terraform apply failed in production",
              "blocks": [
                {
                  "type": "section",
                  "text": {
                    "type": "mrkdwn",
                    "text": "*Terraform Apply Failed*

Environment: Production
Commit: ${{ github.sha }}
Actor: @${{ github.actor }}

Check workflow: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
                  }
                }
              ]
            }
        env:
          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}

Multi-Cloud Networking and Security

Consistent networking and security policies across AWS, Azure, and GCP.

Network Architecture Patterns:

•Central hub VPC/VNet for shared services: DNS, NAT, logging, and security appliances live once in the hub instead of being duplicated per application.
•Spoke VPCs/VNets for applications: each application gets an isolated network that routes through the hub, limiting blast radius if one spoke is compromised.
•Transit Gateway (AWS) / Virtual WAN (Azure) / Network Connectivity Center (GCP): the managed routing layer that connects every spoke to the hub without a full mesh of peering connections.

•Public zone: Internet-facing resources. This is the only layer that should have a public IP or internet-facing load balancer.
•Private zone: Application tier, reachable only from the public zone, never directly from the internet.
•Data zone: Databases and sensitive data. This is the most restricted zone, reachable only from the app tier, ideally with no outbound internet access at all.
•Management zone: Admin access and monitoring. It isolates operator/bastion access so a compromised app tier can't reach admin tooling.

•VPN tunnels for site-to-site: the cheapest option, encrypted over the public internet, fine for moderate throughput and non-latency-critical links.
•Direct Connect / ExpressRoute / Interconnect: dedicated, private connectivity for workloads that need guaranteed bandwidth and lower latency than a VPN over the internet.
•Cloud Router for BGP peering: dynamic route propagation between clouds so new subnets don't require manual route table updates on both sides.

Security Best Practices:

•Separate subnets per tier (web, app, data): a compromised web-tier instance shouldn't have a network path straight to the database subnet.
•Security groups / NSGs / Firewall rules: deny-by-default with explicit allow rules per tier, not one flat rule set shared across the network.
•Zero-trust architecture: authenticate and authorize every request, even between internal services, instead of trusting anything inside the VPC perimeter.

•TLS for data in transit: include internal service-to-service traffic, not just the edge, because internal networks get breached too.
•KMS / Key Vault / Cloud KMS for data at rest: managed key services handle rotation and access auditing that hand-rolled encryption won't.
•Rotate keys automatically: a compromised key that's never rotated stays exploitable indefinitely; automatic rotation caps the exposure window.

•IAM roles (least privilege): grant only the permissions a resource actually needs; broad roles turn a single compromised service into a full-account breach.
•Service accounts (no long-lived credentials): short-lived, auto-rotated credentials mean a leaked key expires before it's useful to an attacker.
•MFA enforcement for human access: the single highest-leverage control against credential-stuffing and phishing for console/admin access.

•Flow logs for all networks: the only way to reconstruct what actually happened during an incident if application logs don't cover it.
•CloudTrail / Activity Log / Cloud Audit Logs: records every API call against your infrastructure, which is what you need when auditing who changed what.
•SIEM integration (Splunk, Datadog, ELK): centralizes logs across three clouds so an investigation doesn't mean logging into three separate consoles.

•15 regions across AWS, Azure, GCP
•50+ VPCs/VNets globally
•99.99% uptime SLA
•Sub-50ms latency between clouds

Hubs peer over IPsec/dedicated links (<20ms, 10Gbps+, BGP failover); one Terraform management plane provisions all three.

## Conclusion Managing multi-cloud infrastructure with Terraform gives you one workflow for provisioning and maintaining resources across AWS, Azure, and GCP. By treating infrastructure as code, you gain version control, reproducibility, and the ability to automate complex deployments that would be error-prone if done manually. The key advantages of Terraform for multi-cloud deployments: - Provider abstraction allows managing resources across multiple clouds using a consistent workflow and syntax - Reusable modules enable standardization of infrastructure patterns and best practices across teams and projects - State management provides a single source of truth for infrastructure configuration and drift detection - CI/CD integration automates testing, validation, and deployment while maintaining security and compliance Implementing multi-cloud Terraform well requires careful attention to state management (remote backends with locking), modular design (DRY principles), and CI/CD pipelines with proper secret management. The upfront investment in solid architecture returns lower operational overhead and faster deployments. At Bayseian, we've architected and managed multi-cloud Terraform deployments for clients spanning thousands of resources across AWS, Azure, and GCP. Our approach emphasizes reusable modules, comprehensive testing (with terratest), and automated CI/CD pipelines that enable teams to ship infrastructure changes safely and rapidly. Whether you're starting fresh with a multi-cloud strategy or migrating existing infrastructure to Terraform, the patterns outlined in this guide provide a production-ready foundation. Start with small, well-tested modules, implement proper state management from day one, and gradually expand your infrastructure as code footprint. Building multi-cloud infrastructure? Contact us at contact@bayseian.com to discuss Terraform architecture for your needs.

TerraformIaCMulti-CloudAWSGCPAzure

Cloud Infrastructure

Kubernetes in Production: Security, Monitoring, and Cost Optimization

Production-hardened Kubernetes deployment strategies covering service mesh, observability, auto-scaling, and infrastructure-as-code best practices.

Working on something like this?

No pitch, just a practical conversation with the team that builds and operates these systems in production.

Start a conversation