Engineering Metrics Are Overrated

Introduction

Engineering metrics are overrated, I don't think so! I think they are vitally important and a valuable tool for gauging the health and progress of software engineering teams.

While some argue that metrics can be misleading or misused, they provide crucial data points that can inform decision-making and identify areas for improvement.

We all know software development is a complex process with numerous moving parts. Measurement is a key way to quantify these parts.

DORA

DevOps Research and Assessment, DORA for short, these metrics focus on outcomes that directly impact software delivery.

However, DORA metrics are lagging indicators, meaning they reveal information after the fact. To gain more real-time insights into development health, leading indicators are essential.

For some quick insight into DORA and its Core Model, please visit DORA is More Than DORA.

Leading indicators 📈

Here are some leading indicators that we commonly use:

• Pull Request (PR) size: Smaller PR's are generally easier to review and merge, leading to fast deployment cycles.
• Pull Request (PR) review time: Faster PR review times indicate smoother collaboration and knowledge sharing within the team, which can lead to faster deployment cycles; similar to above.
• Code coverage: Measures the percentage of code executed by automated tests. High code coverage indicates a lower likelihood of undetected bugs.
• Code churn: Tracks the amount of code that is added, deleted, or modified. Low churn suggests a more stable codebase.
• Code complexity: Complex code can be difficult to understand and maintain, leading to higher development costs and increased risk of errors. By monitoring code complexity, teams can identify areas for refactoring to improve code quality.

...and there are many more leading indicators that you can add to your team toolbox to continually monitor and improve 😎

How do I get these metrics?

Well that's the hard yet can be easy part, you could build your own integrations and all, crunch this data and present them, leverage open source, however, I would guess this is not your core business. Therefore, I'd argue purchasing a SaaS platform off-the-shelf is the easy way. You'll just need to evaluate the market, ensure it fits your needs and your toolchain etc. There is some open source out there (Apache DevLake) in this space, I'd encourage you to take a look and see if it suits your needs, wants desires.

Final thoughts

The key to using software engineering metrics effectively lies in selecting the right metrics for the specific product/project goals and context. It's also crucial to avoid relying solely on metrics to make decisions, remember, these are data points. Metrics should be used in conjunction with other factors such as team feedback and code reviews to get a holistic view of the team's development process; i.e. data-driven decisions for continuous improvement.

It's not just about lagging metrics such as DORA, but leading metrics for you and your team to monitor, understand, adjust how you go about software engineering and delivering on agreed targets & outcomes 🎯

More information

• DORA is More Than DORA: https://dev.to/peteking/dora-is-more-than-dora-22ic
• DORA: https://dora.dev

SaaS Platforms

• LinearB: https://linearb.io/
• Sleuth: https://www.sleuth.io/
• Swarmia: https://www.swarmia.com/
• Jellyfish: https://jellyfish.co/
• Haystack: https://www.usehaystack.io/
• AllStacks: https://www.allstacks.com/
• Apache DevLake (Open Source): https://devlake.apache.org/

The above is not an extensive list