)
)
)
)
Explore how Terramate can uplift your IaC projects with a free trial or personalized demo.
In 2014, HashiCorp fundamentally revolutionized the way cloud infrastructure is deployed and managed by inventing Terraform. Terraform wasn’t the first IaC tool, but it was the first to introduce the concept of state, which fundamentally changed the game by allowing teams to build cloud infrastructure safely because it remembers everything you've already built.
Today, 11 years later, Terraform has become the de facto industry standard for teams to deploy and manage cloud infrastructure at scale.
It’s a common agreement across engineering organizations of all sizes that cloud infrastructure must be provisioned and managed with Infrastructure-as-Code. But while IaC tools such as Terraform and OpenTofu are widely understood and loved by expert users, aka “platform teams”, developers often dislike them due to their complexity and declarative nature. Let’s explore this in detail.
The IaC Self-Service Of Today
In today’s world, the de facto standard for platform teams is to establish an internal catalog of vetted Terraform modules, often called “Service Catalog” or “Infrastructure Service Catalog”, that provides blueprints for deploying the most commonly used cloud infrastructure services in an organization. As an example, take a database that requires specific security and backup settings to be configured. For platform teams, providing those “golden paths” with baked-in best practices, IaC is the perfect tool, as it allows them to codify requirements, automate the lifecycle & deployment processes, and review changes before they are deployed.
The idea of platform teams is that modules can serve as a unit of governance, allowing them to codify and enforce the best practices mentioned. But in reality, developers take those modules, overwrite settings or even the code entirely according to their needs, generating fragmented Snowflakes-as-Code in the process.
One of more pressuring problems is how modules are written and what configuration options they expose to developers. For example, take this popular open source module for creating Amazon Elastic Kubernetes (EKS) resources. It exposes 102 input variables, leaving the developer with an overwhelming amount of configuration options and choices.
Why “Normal Developers” Dislike IaC
The harsh reality of Service Catalogues is that developers still need to learn and understand Terraform. In a world of overwhelming complexity (think Distributed Systems, Microservices, Kubernetes, Observability, Security, etc.), we require developers to learn HCL, state management, input variables, workspaces, and more. That’s a pretty overwhelming amount of technical knowledge required to deploy the “simple” database that a developer needs to run a service in production.
Can they learn HCL and the cloud specifics? Sure. But is it an efficient use of developer talent? Absolutely not.
Self-Service Ducktape - Internal Developer Platform
To overcome this, the industry invented Internal Developer Platforms (“IDP”). Often built on top of open-source frameworks such as Spotify’s Backstage, an IDP usually has two responsibilities:
- Provide a catalog of all services and infrastructure available
- Provide self-service workflows for developers to simplify complex tasks
While IDPs do a pretty good job of providing a single control plane that bundles all available information, they usually fall short when it comes to self-service workflows. Yes, basic tools such as templates and scaffolding exist, but they only help to automate the configuration of IaC templates. They are usually not that helpful when it comes to abstracting technical details, such as state, and surely not when it comes to maintaining and updating templates.
On Why Day 2 Remains Overlooked
While IDPs and service catalogs are sometimes successfully used by some of the best-performing teams in the industry, they tend to completely ignore important Day II management tasks required to keep your IaC healthy. It’s one thing to scaffold the configuration for a Terraform module for deploying a database initially. But the real work kicks in when configurations need to be upgraded, modules and providers need to be updated. The same counts for updating Terraform itself. This kind of chore work is usually beyond the developer's control and mandate.
Worse, outdated versions or dependencies often lead to Terraform code wastelands that are hard to maintain in the long run and pose security risks. The reckoning often arises when trying to deploy outdated code as part of disaster recovery, when we often find ourselves facing IaC that doesn’t work any longer.
IaC Was Never Meant For Developers
The truth is that IaC was never meant for the average developer. It’s a tool designed to enable experts. Experts in the domain of cloud infrastructure and IaC.
At Terramate, we’re building an IaC developer platform that makes IaC accessible and scalable for everyone, including non-expert developers. As such, we rethought how the division of labor between experts and non-experts should look to enable elegant infrastructure self-service that actually works. Without requiring your developers to learn Terraform in the first place.
More on that in our next post on Terramate Catalyst.