Setting Good Metrics

Most bad decisions don’t come from misreading a metric. They come from adopting the wrong one in the first place: a number that was easy to grab, sounds impressive, and turns out to drive nothing. Reading Data and How Data Misleads are about handling metrics you already have. This page is earlier: how to decide whether a metric deserves to exist at all.

Third of three. North Star picks the outcome, Goals & Success Metrics turns an input into a team goal, and this page is the craft of writing the metric itself so it survives contact with a team.

The four tests#

A metric is worth adopting only if it passes all four. Fail any one, and it’s noise dressed up as a KPI.

It ties to a decision. If this number moves, something changes: you’d ship, cut, or escalate differently. No dependent decision means decoration, not a metric. Same “so what, now what” test as Reading Data, applied before the dashboard gets built instead of after.
It’s actionable. The team watching it must be able to move it. A number that only shifts with macro conditions, seasonality, or another team’s work isn’t a target; it’s weather. Track weather if you need to, but don’t hold anyone accountable to it.
It’s leading where possible. A number that predicts an outcome lets you act while there’s still time; one that only records an outcome only lets you grade it. Leading indicators aren’t always available, but reach for one before settling for a lagging proxy.
It’s hard to game. Ask before you ship it: if someone optimised this number alone, what would break? If the honest answer is “quite a lot,” it will eventually get gamed, Goodhart’s law arriving early instead of late.

A metric that ties to a decision but can’t be moved is frustrating. A metric that’s actionable but doesn’t tie to a decision is busywork. You need all four together.

The anatomy of a metric you ship#

A metric on its own is incomplete. Every metric you actually put in front of a team should ship as three parts:

Part	What it is	Why it’s required
The metric	The number itself, precisely defined	Ambiguous definitions produce arguments about the number instead of the decision
The target	A specific value, with a timeframe	”Improve X” isn’t a target. “X above 40% within 2 weeks” is
The counterbalance	A second metric that would move if you gamed the first	Without it, the metric above is an invitation to Goodhart’s law

The counterbalance is the part teams skip, because it feels like extra work for a metric that’s already agreed. It isn’t extra; it’s what keeps the first metric honest. Trust the Report has the fuller argument, plus a table of common pairings (speed/quality, volume/completion, growth/retention). Pick the counterbalance alongside the metric, not after the first sign of gaming.

Define it before you build#

The metric is not a reporting afterthought bolted on after launch; it’s part of the spec. Define makes this explicit: nothing moves to Build without a primary metric, a target, and a note on how it’s measured. If the instrumentation doesn’t exist yet, building it is part of the work, not a follow-up ticket.

The forcing function: if the team can’t state the metric and target before Build starts, the problem isn’t fully understood yet. “We’ll see if users engage more” is a hope, not a metric. “Cart save rate above 40% within 2 weeks, measured by sessions where a cart is restored vs. sessions where a user abandons at payment” is a metric. Write the second kind, or go back and sharpen the problem.

Avoid vanity metrics#

A vanity metric is one that fails the first test quietly: it looks like it ties to a decision, but doesn’t. Total signups ever, page views, lines of code shipped: all big, all always going up, all telling you nothing you’d act on. How Data Misleads names these outright: if a number can’t go down and nothing changes when it moves, it’s decoration.

The tell is usually in how the metric is presented: as a cumulative total rather than a rate, without a denominator, without a comparison to a baseline or a competing segment. Before adopting a metric, rewrite it as a rate over a defined period and ask if it still sounds impressive. Often it doesn’t, and that’s the useful signal.

Vanity is the passive failure: the metric simply does not steer. The active failure is a metric a team can hit without helping anyone, which is gaming, and it is worth checking for separately.

Checklist: is this a good metric?#

Before a number becomes a target, run it through this list:

Does something actually change if this metric moves?
Can the team that owns it actually move it?
Is there a leading version of this, or are we settling for lagging?
What would gaming this metric look like, and have we picked a counterbalance for it?
Is the target specific, with a number and a timeframe?
Is the instrumentation to measure it real, or still to be built?
Rewritten as a rate over a period, does it still matter?

If you can’t answer all seven cleanly, the metric isn’t ready to ship: sharpen it before it goes on a dashboard or into a Define doc.

Example#

Illustrative example, threaded through this documentation: an online store.

Vanity / weak metric	Good metric
Total orders completed, all-time	Weekly checkout completion rate (started → finished)
Orders started	Add-on attach rate: % of completed orders with a related item or protection plan attached
App downloads	Page load time for product pages

Written out in full, the way a metric should ship:

Metric: Add-on attach rate: the share of completed orders where the shopper adds a related item or protection plan before checking out, measured weekly.
Target: 18% → 30% within the quarter.
Counterbalance: Return/refund rate. If attach rate climbs because the checkout is pushing add-ons shoppers don’t actually want, returns (and refund volume) is where that shows up. It’s what keeps the first number honest.