Building a unified metrics story for your business
Using a Metrics Tree to tie your metrics discussions together cohesively
This post is a continuation of my Learning from Metrics Review series for data leaders, where I discuss using signal from Metrics Reviews to drive your organization's data culture and data-driven decision-making forward.
The first post is here, and will be updated with links out to all other posts as they are published. Be sure to subscribe to stay up to date with future installments!
Successful business leaders know that focusing on just your own metrics isn't enough to succeed. You need to understand the broader context of those metrics in the business. When your KPIs are climbing, what does that mean for your peers' KPIs? Are there tradeoffs? Is the growth you're experiencing sustainable, and what cues in other metrics are you seeing to suggest that's the case?
Despite its criticality, building a broad understanding of the organization’s metrics can be hard for a leader to do in practice. It’s a lot to keep in one’s head. What's more, one leader’s mental model doesn't scale: She may lock in a definition, or make assumptions, that others don't share. The result: Lots of time spent clarifying metrics and the relationships between them, instead of moving the business forward.
To ensure business leaders are building a shared understanding of how metrics relate to each other, they need a shared artifact. One that connects each business unit's metrics to the overall KPI of the business by showing the relationships between them. In short, you need a Metrics Tree.
Building a Metrics Tree
At its simplest, a Metrics Tree is just a box-and-line diagram. Each box is a metric, and each line represents one metric contributing to another. For example, a company with a subscription-based B2B app would articulate Annual or Monthly Recurring Revenue (ARR/MRR) as their topline KPI. Feeding into revenue would be revenue from new subscribers, as well as revenue tied to renewals. Each of those, in turn, have metrics driving them, and so on.
The example above shows three levels of a rudimentary Metrics Tree. Note that the first two levels are formulaic: You could create a mathematical equation tying them together. Not so for the second and third levels. Driving renewals is fuzzier! While you might build an ML model to predict renewal or churn likelihood, there is no formula that tells you precisely how much revenue you'll get from renewals next month. Nevertheless, we anticipate customers who receive value from our product are more likely to renew, so we track metrics that may be indicative of that value, like usage, satisfaction, or something even more granular.
All this to say, the goal with a Metrics Tree is not mathematical precision with each relationship (although your Data Science / Analytics team may choose to interrogate and model the more tenuous relationships). Rather, you want the artifact to represent the current set of success metrics being used by the business. For each one, ask: Why do we care about this? The answer should be, "We expect optimizing this will, in turn, optimize, or trade off with, [some other metric]" (otherwise, it may not be worth your time to track it!). By repeating this process, you end up with a visual representation of the business's core metrics, with a short-hand of why, collectively, you believe they matter.
Why bother with a Metrics Tree?
I sometimes get pushback when building conceptual models like a Metrics Tree. By design, creating a Metrics Tree means taking time to document and review "what we already know." It can be a challenge getting business leaders to devote energy to this, when driving the business forward needs their attention.
But the payoff from a central Metrics Tree artifact more than makes up the time spent building it. The benefits of having a Metrics Tree for the organization are numerous:
Build a shared understanding of metrics: A Metrics Tree ensures that the landscape of business metrics is documented, and accessible, for anyone in the company.
Reduce cognitive load of remembering metrics and relationships: In my experience, data professionals and those with a data background tend to downplay how hard it is to build a mental model around metrics, because they are so practiced at it. For business leaders whose sole focus isn’t data, a Metrics Tree can be a useful tool to quickly rehydrate their mental model when they need to context switch to thinking about metrics.
Articulate Emerging Hypotheses: When we start talking about metrics and how they relate to each other, we quickly think of it as work for a Data Science or Analytics team. Can you prove that more of metric A will cause an increase in metric B? How strongly are metrics C and D correlated? While there's always some value in answering questions like this, the business won't stop while the data team goes to answer them. A Metrics Tree allows you to put stakes in the ground, in a light-weight artifact that can be updated over time as the data team learns more.
Support Product Experimentation: A cornerstone of an experimentation program is establishing an Overall Evaluation Criterion, or OEC. This metric needs to tie to the overall business KPI, but has to be low enough in altitude to be moveable by proposed changes. A Metrics Tree facilitates finding, and reaching agreement on, your OEC. (On the flip side: If you've already articulated your OEC, a Metrics Tree is lightweight, visual documentation of the decomposition process you used to arrived there. Documenting it will streamline your next OEC update conversation!)
Assist Data Team in Triaging One-Off Requests: I'm sure I'm not the only data leader with this experience: A stakeholder promises they will be unleashed to do amazing things, if only we could drop everything to define, track, and measure these 15 new metrics. Having a Metrics Tree can help that stakeholder start from where the business is today, and iteratively build their idea out from there.
Additional Reading
The last year has seen quite the boost in publishing on this topic, which is great! It’s cool to read how other data leaders are talking about this artifact. I’ve collected what I've read, if you want more examples to build from. If you’re aware of another resource, please share it in the comments!
Designing Metrics Trees: Ergest Xheblati details an approach to building Metrics Trees that goes beyond simply capturing the metrics that matter to the organization today. This post has some great tips for a data team to use their Metrics Tree to drive further leadership on metrics for the business.
How to describe your business as an equation: A great post from Lenny Rachitsky and Dan Hockenmaier that articulates the top portion of metrics trees across multiple business models. I typically find more to build out past these equations, but you’ll likely find the “equation” aspect quickly breaks down when there’s more fuzziness around how to drive a metric. Still, this is a great reference as you are building the root of your Metrics Tree.
Metrics Loop: Product-Driven Approach to KPIs: I enjoy Elena Dyachkova's content for all things Product Analytics, and this post is no exception. Elena extends the Metrics Tree concept into the Product space, pulling metrics from the overall tree to create a user-driven loop that can guide growth efforts. I especially like her qualitative annotations — it gives space for Product, Engineering, Design, and Data to all collaborate on optimizing each step.
These resources show some breadth in terms of structure and level of detail. But if you're a data leader just starting to think about this topic, don't be overwhelmed! Keep it simple. Start with your business's topline KPI, and partner with your stakeholders to build out the tree, one metric at a time. Before long, you'll notice the increased ease in your metrics conversations, and the need for any additional structure will come naturally.
Post photo by Eilis Garvey on Unsplash Nature | Roots