FinOps Cloud Cost Estimation with Terramate, Terraform and Infracost

Money! Dinero! Soldi! Cash rules everything around us, and in the world of cloud infrastructure, keeping track of them is paramount! Whether you are a Cloud Architect or a DevOps engineer, cost considerations take priority over anything else.

Today, we will explore how we can keep track of costs for each resource in our cloud infrastructure. We are going to use two tools: Terramate, a powerful orchestrator and code generator for Terraform, and Infracost, a FinOps tool for Terraform that estimates cloud costs and helps you understand expenses before making changes.

Let's see how we can combine these tools.

Introducing Terramate and Infracost

A proper introduction to these tools is well deserved.

Terramate is an open-source Infrastructure as Code (Iac) orchestrator that helps you scale your Terraform code. Essentially, it enhances the DRY (Don't Repeat Yourself) principle as you can split your Terraform code, and consequentially its state, into multiple, isolated stacks. Terramate supports change detention, which means you can deploy only what you have changed without having to deploy all of the other resources within the different stacks, hence decreasing deployment time and lowering blast radius.

Infracost is your FinOps ally. It allows you to understand and keep track of how much your infrastructure costs. Knowing how much you are spending enables you to make informed decisions following budget goals.

Enough talking. Let's see how easy it is to integrate Infracost with Terramate.

Terramate Setup

Getting started with Terramate is straightforward, I'll give you two ways:

• Follow the quickstart guide: Begin by following the quickstart guide available https://terramate.io/docs/cli/getting-started/. This guide provides a comprehensive overview of setting up Terramate for your infrastructure projects.
• Clone example repository: Alternatively, you can clone the example repository directly by executing the following command: git clone https://github.com/terramate-io/terramate-infracost.git

Just remember to install Terramate CLI and Terraform to follow along with this tutorial.

If you've cloned my repository, you'll discover three different stacks: first , second , and third . Each stack contains a different resource configuration:

• The 'first' stack comprises an Amazon S3 bucket.
• The 'second' stack hosts an Amazon DynamoDB table.
• The 'third' stack manages an AWS SNS topic.

You can list all stacks in a terramate project with the terramate list command.

To witness Terramate in action, execute the following commands:

terramate run terraform init
terramate run terraform plan

These commands will initiate the init and plan processes for each stack independently, highlighting Terramate's flexibility and efficiency in managing infrastructure deployments.

Now, how can we run Infracost? Given that we have multiple stacks, we need to run Infracost on each one.

Integrating Infracost

First, we need to install Infracost. Here are the two steps you need to do :

• Downloading and installing Infracost
• Get the Infracost API key

When you're ready, run the following command to ensure everything is set up properly:

terramate run infracost --version

Yep, that's right! The Terramate CLI can run any command, meaning we can run Infracost in each stack separately.

Before running Infracost, we need to add another step: cloud consumption estimations. If you have on-demand resources, you'll need to estimate how much these resources are going to be used.

To accomplish this, we create a config file. I prefer using one file in each stack, with each file containing only the resources needed in that stack.

As an example, let's create a new file called infracost-usage-medium.yml in one stack, which specifies the usage for our SNS topic:

version: 0.1
resource_type_default_usage:
  aws_sns_topic:
    monthly_requests: 10000 # Monthly requests to SNS.
    email_subscriptions: 1000 # Number of Email/Email-JSON subscriptions

With everything in place, all that's left is to run one command:

terramate run infracost breakdown --path . --usage-file infracost-usage-medium.yml

Alright, that worked flawlessly. But what if we want to calculate the costs of a single stack? In that case, Terramate has us covered by allowing us to specify the stack we want the command to run on:

terramate run -C stacks/first infracost breakdown --path . --usage-file infracost-usage-medium.yml

And here you have it, folks! Now you have learned how we can leverage Terramate and Infracost to calculate the cost of each stack.

Integrate Infracost Into Your CI/CD Pipeline

Let's tackle another challenge before we conclude this article, shall we?

First, we need to save our INFRACOST_API_KEY as a secret in GitHub Actions. For the Infracost action, we are going to use this action: infracost/actions/setup@v2.

To complete the puzzle, we need to write the GitHub action workflow. Essentially, we need to run Terraform and Infracost commands for each stack. Eventually, we end up with these commands:

- name: Create Terraform plan on changed stacks
      run: |
        terramate run --changed -- terraform init
        terramate run --changed -- terraform validate
        terramate run --changed -- terraform plan -out ./out.tfplan
    - name: Generate Infracost cost estimate baseline
      run: |
        terramate run --changed -- infracost breakdown \
          --path ./out.tfplan \
          --format=json \
          --usage-file infracost-usage-medium.yml \
          --out-file=./infracost-base.json
    
    - name: Post Infracost comment
      run: |
        terramate run --changed -- infracost comment github \
          --path=./infracost-base.json \
          --repo=$GITHUB_REPOSITORY \
          --github-token=${{github.token}} \
          --pull-request=${{github.event.pull_request.number}} \
          --behavior=new

And there you have it, your GitHub Actions Pipeline is up and running.

Conclusion

Leveraging Terramate with Infracost to see how much we are going to be billed for each stack is truly something you're going to love. What really stands out is the ability to see the costs of each stack separately.

We've delved into how Terramate works and even explored how to use Infracost in a GitHub Actions Pipeline to comment on a pull request. Now, your pull requests will really stand out .

Check out our example repository  https://github.com/terramate-io/terramate-infracost

Community Support

I highly encourage you to join Terramate's Discord Community if you have questions or need help with Terramate. They have an active and supportive community ready to assist you!