Explore how Terramate can uplift your IaC projects with a free trial or personalized demo.
us, in this article we are further diving into the pros and cons of each approach.
While giving all the props to our colleagues at Digger, we have some opinionated views on the topic. Let’s dive right in by commenting on the points that are taken from the article mentioned before (we did not change the quoted section in any way, as we mainly agree with the said and want to provide context for additional considerations).
As this topic is highly opinionated, we believe that recommending merge-before-apply
is the optimal workflow to use in 2025, as services like GitHub Actions allow for those kinds of flows better than ever before.
As the CTO of Terramate, I’ve seen many different approaches, and the customer feedback was always very positive after implementing the opinionated flows that we also share in the Terramate docs, for you to be ready to start using Terramate and GitHub Actions to deploy your IaC on scale today.
Let’s dive into the not recommended flow first and see what pros and cons were shared in the mentioned article.
Repository as Source of Truth: When Terraform changes are applied before merging, the repository serves as the definitive source of truth. Each pull request (PR) accurately represents the intended changes to the infrastructure, aligning closely with the concept of Infrastructure as Code.
This is partially correct but misses a significant point: The pull requests can not be considered as a single object that can be applied reproducibly. This is mainly due to the fact that continuous iterative applies never apply the pull request as a whole, as there is a rollback missing that would start the PR from zero (as for example AWS CloudFormation is doing it). This results in the inability to lift the code from one environment to the other, independent of the order of merge
and apply
.
Immunity to Failed Applies: This approach is not impacted by failed Terraform applies. Since changes are validated and applied before merging, only successful alterations are integrated into the main branch.
Terraform and OpenTofu support partial applies that are not associative in all cases, so applying A
then B
is not the same as applying the sum of A and B
together. This mainly shows in the case of failing dependency definition in code, when a second apply can fix a first apply without any code changes. As it is possible to redeploy the same code also via a rerun on main
or through an automated drift reconciliation, the order of merge
and apply
is not relevant.
Simplified Collaboration: Collaborating on Terraform changes is straightforward, as there’s no need for an external dashboard to track the status of infrastructure changes. Team members can easily discuss and review changes within the context of PRs.
Instead of an external dashboard, an external and privileged automation orchestrator is required. This orchestrator also needs to be configured with full RBAC in addition to the RBAC already provided in the VCS.
It is true, though, that the developer experience is slightly improved due to the fact that additional reviews normally get approved faster. This can be due to the simplicity of changes or due to the fact that a previous approval is considered “enough”. There is a risk of applying changes without giving it the full attention after a failed deployment due to fatigue in reviewing the same code again - even when there are minimal changes for a required fix, the changed files section in e.g. GitHub will still show all changes of the full PR.
Deviation from GitOps Principles: Applying changes before merging does not fully adhere to GitOps principles, which advocate for the main branch to always reflect the current state of your infrastructure.
In addition, drift detection running on main will lead to constantly finding a drift as long as pull requests are in progress. As this is also true for an already merged stack that has failed deployment, the detected drift is inverted: not merged code (apply-before-merge
) leads to a drift that wants to destroy the code partially deployed while merged code in the merge-before-apply
flow leads to a drift that wants to deploy the missing infrastructure. This leads to being able to auto-reconcile drift on main even though it might fail again.
Complexity in Maintaining Current State: Ensuring PRs are up to date with the main branch requires careful rebasing, which can be complex and error-prone. Additionally, to guarantee the consistency of the apply phase, plan artifacts need to be stored and managed.
While this is true, rebasing is an essential process in both flows.
Dependency on Plan Artifacts: The necessity to store plan artifacts to ensure that the apply step reflects the reviewed changes adds complexity and overhead to the workflow.
Important to note: Requiring storing intermediate plans requires a trusted storage as plans can contain sensitive data and should not just be blindly pushed to an artifact cache that is out of reach of your organization.
After discussing the pros and cons of the previous flow, it seems obvious that the pros are not that strong and that the cons are just hitting the surface or are even shared between both flows.
How about applying IaC after merging? Let’s dive deep in the pros and cons mentioned here:
Elimination of Plan Artifacts Storage: By applying Terraform changes after merging, there is no need to store plan artifacts, simplifying the workflow and reducing storage requirements.
Though it is recommended to have an up-to-date plan reviewed and keep the time between review and merge and apply as low as possible.
Storing intermediate plans is an option and can lead to additional guardrails as Terraform plans can get stale and deny to apply.
Alignment with Software Build Pipelines: This method is more in line with traditional software build pipelines, treating Infrastructure as Code similarly to application code, which many teams find intuitive and straightforward.
We can confirm that this benefits teams by reducing mental overhead and complexity. When creating a new pull request to fix the main
branch, those pull requests are normally small and can be reviewed very fast and can thus lead to a learning effect to prevent such failure in the future for all involved collaborators.
Simplified Process for IaC Management: Treating infrastructure code similarly to application code streamlines the process, making it more familiar and manageable for teams accustomed to software development practices.
This is essential when enabling developer self-service where infrastructure could be aligned with application workflows. Having contradicting workflows will prevent some simplicity in the long run and lead to additional mental overhead in on-boardings as well as in the maintenance lifecycle.
Necessity for External UI: Managing Terraform projects becomes challenging without an external UI that provides a comprehensive view of all projects and their states, making it harder to track and collaborate on changes.
This is true as the number of failures can be high and the main branch might reach a state where a high number of stacks are drifted due to failed deployments on top of the normal reasons for a drift like manual changes on the cloud aka click-ops.
This UI is Terramate Cloud, which is a fully managed SaaS product that doesn’t require you to run any additional infrastructure as opposed to choosing a tool like Atlantis, which comes with considerable overhead to setup and operate.
Main Branch: The main branch can become extremely noisy with frequent merges and applications, potentially obscuring the overview of changes and complicating the management of the infrastructure.
This is why solutions like Terramate Cloud exist that help to keep the mean time to recovery for teams as low as possible by providing real-time notifications and helping remediating the failure as fast as possible to keep main
clean at all times.
Among Terraform and OpenTofu users, there’s a consensus on a mixed approach:
For Critical Environments: The preferred workflow is to merge changes into the main branch and then apply them to ensure that the infrastructure remains stable and reliable.
For Lower Environments: Applying changes before merging is favored, allowing for more flexibility and rapid iteration without risking the stability of critical infrastructure.
While I agree on the summary provided here, I must decline going for the mixed approach simply for the reason of keeping dev/prod parity in workflows. Maintaining two different workflows is overhead that can be avoided, and it is better to have one workflow that perfectly works for all environments. For different environments, it might be better to apply a different number of safeguards to make merging for non-prod environments easier, leading to faster lead time for changes in lower environments.
For us at Terramate, the recommendation is clear in many ways to always merge-before-apply
. While we totally understand that teams want to keep things easy and sometimes struggle with changing the flow, we see that adjusting to merge-before-apply
does not complicate things. Tooling can make creating a new pull request as easy as keep working on an existing pull request.
While the main argument for apply-before-merge
often is the reduced cognitive load for the author, the cognitive load in reality is mainly shifted to the reviewer, who is required to understand intermediate changes in a pull request for each iteration. This cognitive load for reviewers is actually true for each reviewer involved e.g. for three reviewers, the shifted load is 3x as high.
Terramate is meant to enable developers, DevOps and platform engineers by making IaC deployable as easy as possible and by providing an end-to-end experience from generating code, orchestrating stacks, and automating (with merge-before-apply
) workflows to also provide the required visibility and observability for teams on scale through notifications, pull request previews, alerts, and AI (yes, ofc you get this on top today).
If you want to learn more about how Terramate can help your IaC practices, feel free to book a call with one of our experts or schedule a demo today.
Explore how Terramate can uplift your IaC projects with a free trial or personalized demo.