Guardrail Metrics for Sustainable Growth
Save yourself from gaming metrics
At some point in my career, I was lucky to take part in an interesting motivational scheme rolled out to all company managers. The goal of the company was to double the business, and the top management team wanted all managers to work towards that goal. The rules were simple – double the business, and every participant gets a substantial financial reward. To give us a taste, they even split that goal into three milestones – you get 25% of the reward once the company reaches 25% of the target and sustains it for one month, then another 25%, and then the rest.
In practice, it was based on 3 key metrics – to double the business, we needed to double our revenue, profit, and monthly active users. Having this triangle of metrics was essential to protect the scheme from gaming. We couldn’t double revenue by bringing low-quality users. We couldn’t spend all our profit on marketing and ignore margins. And we couldn’t simply squeeze more money from existing customers – the user base wouldn’t grow. We needed to grow the business sustainably.
That was the first time I saw the concept of guardrail metrics put into practice at such a scale.
In an ideal world, we could just give people a single goal and they would focus on achieving it.
Reality is more nuanced.
According to Goodhart’s Law – “when a measure becomes a target, it ceases to be a good measure.” This means that over-optimization of any individual metric makes things worse as a whole.
This is especially true at the department level. With multiple department silos, people tend to optimize for their local maximum. Product teams focus on retention without paying enough attention to monetization. The marketing team focuses on bringing new users and initial conversions, without taking into account the long-term performance of these users. Top-of-the-funnel teams focus on initial conversions. As a result, the whole system doesn’t work well.
This is where guardrail metrics become important.
Whenever we set a target metric for a department, we need to think of a few common ways this metric can be gamed. Not necessarily with bad intentions – just over-optimized.
Let’s look at a few examples in growth.
Performance marketing
The most common goal of a marketing team is to scale the spend. Usually, we want to invest in growth as much as possible if it’s profitable. To ensure profitability, we want to keep our cost-per-action under control (usually, the cost of acquiring a paying customer). That becomes our first guardrail metric.
Another metric is often overlooked – audience fit. In performance marketing, it’s quite easy to come up with an ad that doesn’t bring the right audience for our product. Especially if people convert before they see the actual product – through landing pages and onboarding quizzes. As a result, people go through our funnel, make an initial purchase or activate a trial, and then discover that the ad promised them something completely different. They cancel their subscription and never come back. I’ve seen cases where the 24-hour cancellation rate differs between creatives by a factor of three.
What do we do? We add it to the list of guardrail metrics for our marketing.
As a result:
Main metric – scale the ad spend
Guardrail metric #1 – cost-per-action under $XX
Guardrail metric #2 – 24-hour cancellation rate lower than Y%
SEO
In SEO, our goal is to bring as much organic traffic as possible. But, traffic that converts. Bringing relevant traffic works on several levels. First, we make sales. Second, it increases the time users spend on our website, which sends the right signal to search engines and improves our rankings.
However, individual pages might not receive enough traffic to measure conversions. That’s where clustering comes into play.
There are several ways we can cluster SEO visits. A couple of examples:
Website sections – the main page, various landing pages, the blog overall, or separate blog categories
Keywords – we determine the most popular pages per keyword and then check conversions from these pages
As a result, our SEO metrics might look like this:
Main metric – number of visits from Google and Bing
Guardrail metric #1 – conversion rate from organic traffic
Guardrail metric #2 – conversion rate for each cluster
Counterintuitively, setting the right guardrail metrics helps teams focus on the main metric.
In any game, we need rules to play. Chaos paralyzes and creates even more chaos. Rules give us a system we can rely on.
The same happened in our case. With 3 metrics balancing each other, we had to find ways to grow all 3 of them sustainably. It also gave the entire management team one common goal and language to work with.
So, did we double the company? We managed to achieve only the first 25% milestone, and then the company was sold and the scheme was abolished.
Set goals, but keep an eye on how they’re implemented.