The importance of DORA Metrics in Infrastructure Automation

We recently came across Kief Morris’s great post, “Where is the value with infrastructure automation?”, which describes the main business value of infrastructure as enabling value streams that allow a business to operate and grow faster by enabling developers to respond to customer feedback more rapidly.

But how can we actually measure the effectiveness of infrastructure and infrastructure teams, such as DevOps and Platform teams?

The old adage “what gets measured gets done” is also applicable here. Alas, many organizations lack the necessary visibility when it comes to understanding the business value created by infrastructure teams. This can create a disconnect between the actual work done by infrastructure and platform teams and the (lack of) perception from the leadership level, which in turn may perceive infrastructure as a cost center in the first place.

So what measurable key performance indicators (KPIs) exist to measure and analyze the effectiveness of infrastructure and teams managing it?

Introducing DORA metrics

DORA (”DevOps Research and Assessment”) metrics deliver evidence-based insights about your DevOps performance. They are a set of key performance indicators used to measure the impact of software development and delivery teams. These metrics are: Deployment Frequency, Lead Time for Changes, Change Failure Rate, and Time to Restore Service. (For a more in-depth explanation of the metrics, see Google’s original post.)

DORA metrics are designed to provide visibility to leadership, enabling them to assess the performance of their teams and platform. In order to make well-informed decisions, the required data needs to be collected and analyzed. But implementing and measuring DORA metrics correctly is hard as multiple systems, teams and end-to-end processes need to be considered and consolidated. To mention some of the most common challenges when implementing DORA metrics:

• Implementation complexity - Cross-team and system dependencies make attribution difficult.
• Data quality and consistency issues - teams define metrics differently, tooling is fragmented across systems, and manual collection introduces errors and delays.
• Context and nuance gaps - one-size-fits-all benchmarks ignore team differences, miss business priorities, and don't capture technical debt or quality trade-offs.
• Measurement blind spots - difficulty determining what constitutes failures or full recovery, plus metrics don't reflect code quality, security, or thorough testing sacrifices.

In practice, many teams refrain from implementing DORA metrics in their organization simply because the effort is often overwhelming.

The DevOps / Developer ratio

At Terramate, we are supporting hundreds of DevOps and platform teams to manage infrastructure with Infrastructure as Code at scale. While working with customers, we observed a common pattern:

A good indicator for the performance of infrastructure teams is the ratio between infrastructure engineers (e.g. platform, DevOps and SRE’s) and developers in an organization:

• Elite teams: 1 ops supports 20 or more developers
• Good teams: 1 ops supports 10 or more developers
• Average teams: 1 ops supports 5 or more developers

This ratio is based on the assumption that the ops function actually is successful in delivering value to the dev teams; for dysfunctional organizations, these ratios are not a helpful indicator. The validity of this rule of thumb also depends widely on the technical sophistication of the platform and the scope of responsibilities of the team. So an elite team with a complicated platform and a broad remit may have a ratio more in the 1 to 10 ballpark.

How the lack of visibility is hurting great teams

We often encounter teams who, in our opinion, do an exceptional job, especially compared to industry averages, yet get little to no love both from leadership and internal customers - i.e., the developers. This is a little akin to the risk management function of banks. Scandal-free banks often have solid risk managers, but those are often unsung heroes. It is only in the absence of sophisticated risk management that trading scandals and extraordinary losses emerge.

The net result is that great ops teams with poor visibility of their value metrics often get little recognition, career progression, and other accolades. To the detriment of organizations, the corollary is unfortunately also true: Bad teams often remain undetected despite failing developers and providing comparatively little to no value.

As a result, any infra and platform team ought to carefully consider creating visibility into their DORA metrics and value streams a priority. It is ultimately in their self-interest.

Out-of-the-box DORA metrics

With Terramate Cloud, we are now tackling the metrics implementation effort and providing out-of-the-box DORA metrics support on all pricing tiers. This is possible as Terramate already has access to Pull Requests and Deployments and can thus leverage the existing data.

Our mission is to enable platform teams to deliver exceptional internal infrastructure experiences while also demonstrating and communicating the impact of their outstanding work to leadership and key stakeholders. Do good and speak about it.