Manage Terraform Providers Programmatically with Terramate

Introduction

Welcome back! In our last blog, we talked about managing provider and backend configurations with Terramate. In this guide, we’re going to explore this further and demonstrate how to programmatically manage multiple providers, across stacks, with Terramate. If you’re joining us, Terramate Code Generation helps you to keep your code DRY. We do this by abstracting configurations to the root of the stack or repo. Then, whenever you run Terramate generate, we create the code necessary based on these configuration files. All of this can be automated as part of your development and deployment processes.

Key Benefits

We have already learned what DRY is. It’s doing more with less. But, how does this benefit us when it comes to programmatically managing providers? Well, there are several key reasons why this is a great practice. The key benefits include:

• Time Savings: Add / Remove / Change the provider configuration across multiple stacks, and sub-stacks, programmatically. Update 1 set of files, and Terramate will make all the necessary changes for you. No more updating stacks 1 at a time.
• Canary Testing: That’s right. Roll out updates to the dev stacks for testing, before you roll everything out to prod. All of this can be automated into your existing CI/CD and E2E processes.
• Lower Overhead: By keeping things DRY, you drastically reduce the management overhead involved with working with multiple stacks, multiple providers, etc.

Example

First things first. We need an example repository to help you get started. That can be found at:

https://github.com/terramate-io/terramate-examples/tree/main/02-manage-providers-programmatically

If we look in the /imports folder, we can see the generate_providers.tm.hcl This file helps us create the _terramate_generated_providers.tf file in each of the stacks. We call this with the imports.tm.hcl at the root of the repository.

imports.tm.hcl

import {
  source = "./imports/generate_providers.tm.hcl"
}

From there, we just need a config.tm.hcl in each “top level” folder where the providers would be separated.

config.tm.hcl

// Configure default Terraform version and default providers
globals "terraform" {
  version = "1.6.0"
}

# Will be added to all stacks
globals "terraform" "providers" "random" {
  source  = "hashicorp/random"
  version = "~> 3.5"
  enabled = true
}

If you have a provider that you want used across every stack, globally, then you put the config.tm.hcl in the root, and every time you run terramate generate or terramate create it will update the provider info in all existing stacks, or it will add the provider to each newly created stack. This is very powerful when you re-use the same provider in all of your stacks.

But what about when you have different stacks with different providers? For example, what if you are multi-cloud? If you have some things deployed in AWS, and some in GCP, you will have to manage those 2 providers, separately. You don’t want to call the AWS provider globally into the GCP stacks. This is where we place the config.tm.hcl down into the root of those subfolder structures. In our example repo, we have a /gcp and /aws folder. Each of these folders contain all of the stacks for each cloud. If we place the config.tm.hcl in the top level of each cloud providers folder, we can then manage each provider separately, while still programmatically managing all of the stacks in our repository with 1 run of the terramate generate or terramate create command.

How cool is that? If we want to generate providers globally, we edit the config.tm.hcl in the root of the repo. If we want to update the AWS provider info, or any other provider used with just the AWS resources for that matter, we edit the /aws/config.tm.hcl file. And then editing the /gcp/config.tm.hcl will update all of the GCP resources.

Think of how much time this is going to save you, as opposed to going through and updating each and every one of the .tf files that call the providers. Keep it easy, keep it DRY.

Conclusion

As you can see, keeping your code DRY has so many benefits. Mainly, your time and effort. Why duplicate work when you can do it once? If you are already using this feature, then let us know how you like it! You can join our community Discord to ask questions, provide feedback, or just see what other Terramate users are doing. You can also check us out on LinkedIn, and X (Formerly Twitter)!