..

TIER infra pt. 3 - TF-CIX as an approach to share information between terraform stacks

First appeared at: tier.engineering

Title: TF-CIX as a new approach to share information between terraform stacks

Preface

Don’t share the state – this is the pattern of Terraform Configuration Inter eXchange (TF-CIX) we implemented in order to exchange information between our different Terraform stacks.

Problem solving

When sharing information between stacks, you may face several issues, eg.:

  1. Sensitive Data in State
  2. Full access to all data of a stack
  3. Hard dependencies between stacks

👉 There is one simple rule to follow – Don’t share the state between teams

What to do in this case? Here’s a solution: use an intermediary data storage. The source stack writes SSM parameters in a defined structure while a TF-CIX module reads and exposes the data.

TIER implementation

Let’s take a look at the ring-0/core-infra stack to see how we use this approach.

The stack creates:

Then there is another stack eg. teamFoo/serviceBar. The new stack may need some basic resources

Share it

# $> cat ring-0/core-infra/tf-cix.tf

locals {
  ssm_prefix = "/tf-cix/${var.metadata["service"]}/${var.region}"
}

resource "aws_ssm_parameter" "main_db_networks" {
  description = "The ID of the _db_networks_"
  name        = "${local.ssm_prefix}/main-vpc/db_networks"
  tags        = local.aws_tags
  type        = "String"
  value       = jsonencode(var.network_config.db_networks)
  overwrite   = true
}

resource "aws_ssm_parameter" "aaa" { }
resource "aws_ssm_parameter" "bbb" { }
resource "aws_ssm_parameter" "ccc" { }
resource "aws_ssm_parameter" "ddd" { }


----
SSM Parameter: /tf-cix/core-infra/eu-central-1/main-vpc/db_networks
Value:         ["10.0.56.0/21","10.0.120.0/21","10.0.184.0/21"]

Behind the scenes

# $> cat TF-CIX/core-infra/main.tf

data "aws_ssm_parameter" "vpc_id" {
  name = "/tf-cix/${var.service}/${var.region}/${var.name}/id"
}

data "aws_ssm_parameter" "aaa" { }
data "aws_ssm_parameter" "bbb" { }


$> cat TF-CIX/core-infra/outputs.tf

output "vpc_id" {
  description = "The ID of the _main-vpc_"
  value       = data.aws_ssm_parameter.vpc_id.value
}

output "aaa" { }
output "bbb" { }

Consume it

# $> cat teamFoo/serviceBar/data.tf

module "core-data" {
  source  = "your.terraform.registry.local/core-infra"
  version = "~> 1.0"

  service = "core-infra"
  region  = "eu-central-1"
  name    = "main-vpc"
}

$> cat teamFoo/serviceBar/service.tf

module "db" {
  source  = "your.terraform.registry.local/tier/postgresql/aws"
  version = "~> 1.0"

  name                       = "db"
  subnets                    = module.core-data.vpc_aux_networks
  instance_type              = "db.t3.micro"
  engine_version             = "11.8"
  vpc_id                     = module.core-data.vpc_id
  allowed_security_groups    = [module.core-data.eks-worker-security-group-id]
  vault_enabled              = true
  [...]
}

Conclusion

To wrap up, exchanging information in a standardized and abstracted way allows to layer the infrastructure without the risk of sharing sensitive information between teams.

The actual implementation of the shared resources can be changed without breaking their dependencies when using an API.

The series