Back to all blog posts

How to inject Backstage Metadata into Terramate Stacks

Photo of Michael Reichenbach
Michael Reichenbach Senior Platform Engineer @1KOMMA5°
Reading Time:7 min read

Ensuring consistent and reliable cloud resource labeling is a challenge in large-scale infrastructure. This guide explores how to leverage Terramate’s code generation to dynamically inject Backstage metadata into Terraform stacks—improving governance, automation, and visibility across environments.

Injecting Backstage Metadata Cover

In my previous blog post, I talked about a pain point many of us face: the mess of inconsistent and incomplete resource labeling in the cloud.

I explained how this can make cost tracking, automation, and generally keeping things organized a real headache. I also showed how you can use Backstage to tackle this by automatically injecting metadata into your cloud resources, ensuring everything gets labeled consistently, no matter how it’s created.

Now, I want to dive deeper into the nuts and bolts of how you can actually generate and inject this metadata module into every stack using Terramate. Consider this a practical guide to building a similar solution for yourself, so you can finally get your cloud labels under control.

After implementing this solution, you will be able to use always up-to-date labels in every single one of your stacks. Just like this:

# main.tf
resource "something" "in_the_cloud" {
  labels = module.metadata.labels
}

Why Terramate?

Why would you want to use Terramate to inject the metadata module into your stacks?

The Terramate code generation is a powerful feature that allows you to dynamically generate Terraform code across all of your stacks.

This is especially useful when you have a lot of stacks and want to keep your Terraform code DRY.

When thinking about injecting a metadata module that provides the same labels for all resources in a stack, Terramate’s code generation is the perfect fit.

What is the Metadata Module?

The metadata module is a module that wraps the Backstage Terraform provider and fetches always up-to-date metadata for a given Backstage entity directly from the Backstage API.

It then transforms this entity metadata into a set of standard labels that can be applied to any cloud resource. Here is an example of the generated labels output:

labels = {
    "created-by"  = "terraform"
    "entity"      = "artist-lookup"
    "kind"        = "component"
    "owner"       = "team-a"
    "system"      = "artist-engagement-portal"
    "lifecycle"   = "operational"
    "type"        = "service"
    "environment" = "production"
}

The metadata module also supports a fallback mechanism that allows you to use the last snapshot from the state if the Backstage API is not available. This prevents your pipelines from failing in case the Backstage API is not available.

Read my previous blog post for more details on the fallback solution.

Injecting the Metadata Module

To use the metadata module, the following needs to be generated into your stacks:

  1. A backend for the remote state
  2. The providers and their versions required by the used resources
  3. The backstage provider including authentication against the Backstage API
  4. The metadata module itself, taking an entity name, and an optional kind and namespace as an input
  5. A terraform_remote_state data block that fetches the latest snapshot from the remote state in case the Backstage API is not available

All of the following code can be found in this example repository.

Let’s go 🏗.

Step 1: Backend

To keep your backend configuration DRY, you can use Terramate’s globals feature to define the backend configuration once and reuse it across all stacks.

This guide will focus on Google Cloud, but the same principle can be applied to other cloud providers. You can find an example for AWS in the terramate-io/terramate-examples repository.

Grab the imports/generate_backend.tm.hcl file from the examples and create an imports.tm.hcl file at the root of the project to include the imports/generate_backend.tm.hcl file.

mkdir -p imports/
curl -s https://raw.githubusercontent.com/Silthus/terraform-backstage-metadata-module/main/examples/terramate-code-generation/imports/generate_backend.tm.hcl > imports/generate_backend.tm.hcl
cat << EOF >> imports.tm.hcl
import {
  source = "./imports/generate_backend.tm.hcl"
}
EOF

Let’s take a look at the imports/generate_backend.tm.hcl file:

# imports/generate_backend.tm.hcl
globals "terraform" "backend" "gcs" {
  prefix = "/terramate/stacks/by-id/${terramate.stack.id}"
}

This part is pretty straightforward. It just defines a gcs backend configuration with a unique prefix for each stack. By setting this as a globals variable, you have the option to override the prefix in the config.tm.hcl file for specific stacks.

NOTE

Using the terramate.stack.id decouples the state path from the stack and folder name, thus allowing you to move stack folders around without breaking the state.

# imports/generate_backend.tm.hcl
generate_hcl "_terramate_generated_backend.tf" {
  content {
    terraform {
      backend "gcs" {
        bucket = global.terraform.backend.gcs.bucket
        prefix = global.terraform.backend.gcs.prefix
      }
    }
  }
}

This will generate a _terramate_generated_backend.tf file in each stack using a Google Cloud Storage bucket as the backend.

The last step is to create a config.tm.hcl file at the root of the project to configure the terraform.backend.gcs.bucket variable.

# config.tm.hcl
globals "terraform" "backend" "gcs" {
  bucket = "terramate-example-terraform-state-backend"
}

Create a stack and run terramate generate to test it out.

terramate generate

Step 2: Dynamic Providers

To keep your provider configuration DRY, you can use Terramate globals to dynamically generate various provider configurations based on global variables.

Let’s grab the imports/generate_providers.tm.hcl file from the examples and add it to the imports.tm.hcl file.

curl -s https://raw.githubusercontent.com/Silthus/terraform-backstage-metadata-module/main/examples/terramate-code-generation/imports/generate_providers.tm.hcl > imports/generate_providers.tm.hcl
cat << EOF >> imports.tm.hcl
import {
  source = "./imports/generate_providers.tm.hcl"
}
EOF

The dynamic provider generation is explained in detail in this blog post. The only change made was to exclude the backstage provider from the dynamic generation, because it will be handled separately in the next step.

# imports/generate_providers.tm.hcl
...
providers = { for k, v in tm_try(global.terraform.providers, {}) :
  k => v.config if tm_alltrue([
    tm_length(tm_split(".", k)) == 1,
    tm_try(v.enabled, true),
    tm_can(v.config),
    tm_try(v.source, null) != "datolabs-io/backstage" # ignore backstage provider as it is generated in the generate_metadata.tm.hcl
  ])
}
...

You can read up on the various functions used by the generate_providers.tm.hcl file in the Terramate docs.

Step 3: Backstage Provider

The Backstage provider generation is a bit more tricky as you additionally need to provide it with an API key. The example repository provides a generate_backstage_provider.tm.hcl file that handles this and allows setting the API key via a GCP Secret Manager secret or directly in the config.tm.hcl file. For obvious security reasons, it is recommended to use the secret manager to store the API key.

curl -s https://raw.githubusercontent.com/Silthus/terraform-backstage-metadata-module/main/examples/terramate-code-generation/imports/generate_backstage_provider.tm.hcl > imports/generate_backstage_provider.tm.hcl
cat << EOF >> imports.tm.hcl
import {
  source = "./imports/generate_backstage_provider.tm.hcl"
}
EOF

Let’s take a look at the imports/generate_backstage_provider.tm.hcl file and break it down:

# imports/generate_backstage_provider.tm.hcl
globals "terraform" "providers" "backstage" {
  source  = "datolabs-io/backstage"
  version = "~> 3.1.0"
  enabled = true
}

This part simply defines a default backstage provider configuration with a version and enables it. This can be overridden in the global config.tm.hcl file or for specific stacks.

# imports/generate_backstage_provider.tm.hcl
condition = tm_try(global.terraform.providers.backstage.enabled, false) && tm_try(global.metadata_module.enabled, false)

This condition ensures that the Backstage provider is only generated if the metadata_module is enabled and the backstage provider is enabled.

tm_dynamic "data" {
  labels    = ["google_secret_manager_secret_version_access", "fallback"]
  condition = tm_can(global.terraform.providers.backstage.config.api_key_secret_id)

  content {
    project = tm_try(global.terraform.providers.backstage.config.api_key_secret_project, null)
    secret  = tm_try(global.terraform.providers.backstage.config.api_key_secret_id, null)
    version = tm_try(global.terraform.providers.backstage.config.api_key_secret_version, "latest")
  }
}

The tm_dynamic block will only generate the google_secret_manager_secret_version_access data source if the api_key_secret_id is set.

locals {
  headers = tm_try(global.terraform.providers.backstage.config.headers, null)
}

tm_dynamic "provider" {
  labels = ["backstage"]

  content {
    base_url = tm_try(global.terraform.providers.backstage.config.base_url, null)
    retries  = tm_try(global.terraform.providers.backstage.config.retries, 3)
    headers = local.headers != null ? local.headers : {
      "Authorization" = "Bearer ${tm_ternary(tm_try(global.terraform.providers.backstage.config.api_key_secret_id, null) != null,
        tm_hcl_expression("data.google_secret_manager_secret_version_access.fallback.secret_data"),
        tm_try(global.terraform.providers.backstage.config.api_key, null)
      )}"
    }
  }
}

The next tm_dynamic block will generate the backstage provider configuration using one of the following authentication methods, trying them in order and picking the first one that is set:

  • Authorization header from the headers variable
  • [RECOMMENDED] API key from the GCP Secret Manager secret using the data.google_secret_manager_secret_version_access.fallback.secret_data expression
  • API key from the api_key variable

A warning will be shown if multiple authentication methods are set. The warning and general validation assertions are handled at the bottom of the generate_backstage_provider.tm.hcl file.

Awesome, now you have a backstage provider configuration that is generated for each stack if the metadata_module and backstage provider is enabled and an authentication method configured.

The last thing to do is to add the Backstage provider configuration to our config.tm.hcl file and configure the base URL of the Backstage instance together with a valid authentication method.

# config.tm.hcl
# Backstage provider configuration
globals "terraform" "providers" "backstage" "config" {
  # Configure the base URL of the Backstage instance.
  # base_url = "https://demo.backstage.io"

  # Use one of the following to configure the API key

  # It is recommended to use the secret manager to store the API key.
  # api_key_secret_id = "your-secret-id"
  # api_key_secret_project = "your-project-id"
  # api_key_secret_version = "latest"

  # Use the following to configure the API key directly.
  # Only use this in test environments!
  # api_key = "your-api-key"

  # Use the directly override the Authorization header.
  # headers = {
  #   "Authorization" = "Bearer your-api-key"
  # }

  # Optionally configure the number of retries for the Backstage provider before using the fallback.
  # retries  = 3
}

Step 4: Metadata Module

Now that you have the backend, providers, and the backstage provider configured, you can finally generate the metadata module for each stack.

Grab the imports/generate_metadata.tm.hcl file from the examples and add it to the imports.tm.hcl file.

curl -s https://raw.githubusercontent.com/Silthus/terraform-backstage-metadata-module/main/examples/terramate-code-generation/imports/generate_metadata.tm.hcl > imports/generate_metadata.tm.hcl
cat << EOF >> imports.tm.hcl
import {  source = "./imports/generate_metadata.tm.hcl"}EOF

Let’s take a look at the imports/generate_metadata.tm.hcl file:

# imports/generate_metadata.tm.hcl
globals "metadata_module" {
  enabled               = tm_contains(terramate.stack.tags, "inject_metadata")
  remote_state_fallback = false

  source  = "github.com/Silthus/terraform-backstage-metadata-module.git"
  version = "v1.0.0"

  defaults = {
    entity_name      = terramate.stack.name
    entity_kind      = "Component"
    entity_namespace = "default"
  }
}

Again, this just configures the default globals for the metadata module, which can be overridden in the global config.tm.hcl file or for specific stacks.

The part to point out here is that the enabled variable is set to tm_contains(terramate.stack.tags, "inject_metadata") , which allows you to enable the metadata module for specific stacks by adding the inject_metadata tag to the stack. For example like this:

# stack.tm.hcl
stack {
  name        = "example-component"
  description = "example-component"
  id          = "a0eae098-9cb9-478c-9e7e-07b36448e1e6"
  tags        = ["inject_metadata"]
}

Following that is the generate_hcl block that will generate the metadata module for each stack.

# imports/generate_metadata.tm.hcl
generate_hcl "_terramate_generated_metadata.tf" {
  condition = tm_try(global.metadata_module.enabled, false)

  lets {
    is_fallback_enabled = tm_try(global.metadata_module.remote_state_fallback, false)

    backend = "gcs"
    backend_config = {
      bucket = global.terraform.backend.gcs.bucket
      prefix = global.terraform.backend.gcs.prefix
    }

    entity_name      = tm_try(global.metadata_module.entity_name, tm_try(global.metadata_module.defaults.entity_name, terramate.stack.name))
    entity_kind      = tm_try(global.metadata_module.entity_kind, tm_try(global.metadata_module.defaults.entity_kind, "Component"))
    entity_namespace = tm_try(global.metadata_module.entity_namespace, tm_try(global.metadata_module.defaults.entity_namespace, "default"))
  }

This will generate a _terramate_generated_metadata.tf file in every stack where the metadata_module is enabled or the inject_metadata tag is set.

In addition, there are also some local lets variables defined to make the generation code more readable. Think of them like locals but for Terramate.

The thing to point out here is that the remote state fallback will only be enabled if a correct GCS backend is configured and remote_state_fallback is set to true .

IMPORTANT
If you are using a different backend than GCS, you need to adjust this part to match your backend.

The last thing left is the actual generation of the data "terraform_remote_state" and the module.metadata block. I will skip the generation of the variables above as they are straightforward.

NOTE
The variables are used to allow overriding the entity name, kind, and namespace using *.tfvars files or by setting the Terramate globals.metadata_module variables.They also come in very handy to reference when naming resources, e.g., using var.entity_name.

Let’s start with the dynamic terraform_remote_state block:

# _terramate_generated_metadata.tf
# ... variables ...

tm_dynamic "data" {
  labels    = ["terraform_remote_state", "fallback"]
  condition = let.is_fallback_enabled
  content {
    backend = let.backend
    config  = let.backend_config
  }
}

This uses the lets variables defined above to generate the data "terraform_remote_state" block if the global.metadata_module.remote_state_fallback variable is set to true .

The next step is to generate the module.metadata block:

# _terramate_generated_metadata.tf
module "metadata" {
  source    = "${global.metadata_module.source}?ref=${global.metadata_module.version}"
  name      = var.entity_name
  kind      = var.entity_kind
  namespace = var.entity_namespace
  fallback  = tm_ternary(let.is_fallback_enabled, try(data.terraform_remote_state.fallback.outputs.metadata.entity, null), null)
}

Here the source is constructed from the config and the generated variables are used as inputs. Here is the last part for the config.tm.hcl file:

# config.tm.hcl
# Configure the metadata module
globals "metadata_module" {
  # Set to true to globally enable the injection.
  # Use the `inject_metadata` tag on a stack to enable it for a specific stack.
  # enabled               = true
  remote_state_fallback = true

  source  = "github.com/Silthus/terraform-backstage-metadata-module.git"
  version = "v1.0.0"
}

🥳 You made it!!! 🎉

You can now run terramate generate to generate the metadata module for each stack.

Conclusion

In this blog post, I showed how you can inject the Backstage metadata module into your Terramate Stacks using Terramate’s code generation feature. This will help you to keep your cloud resource labels consistent and always up-to-date with your metadata located in Backstage.

Check out my other blog post on the topic that dives deeper into the fallback mechanism and the how the idea of the Backstage Metadata Module was born.

Dive Deeper at BackstageCon Europe 2025: London, April 1st

I am giving a talk at BackstageCon Europe 2025 in London, April 1st. Stop by if you are there and get a chance to discuss the exciting topic of binding Backstage to Terraform in person.

Thanks for reading!

Michael


Ready to supercharge your IaC?

Explore how Terramate can uplift your IaC projects with a free trial or personalized demo.

Start free trial Book a demo