references/business-model-metrics.md
# Metrics by Business Model
## Table of Contents
- [How to Use This File](#how-to-use-this-file)
- [E-commerce](#e-commerce)
- [SaaS](#saas)
- [Free Mobile App](#free-mobile-app)
- [Media Site](#media-site)
- [User-Generated Content](#user-generated-content)
- [Two-Sided Marketplace](#two-sided-marketplace)
- [Hybrid Models](#hybrid-models)
## How to Use This File
Name your primary business model, install that model's metric tree, write a definition doc for each metric before building dashboards, and watch the failure modes — most analytics disasters are measurement errors, not data errors. The metrics below are the canonical set; your OMTM at any moment is one of them (chosen by stage), and the rest are supporting or counter-metrics.
## E-commerce
**Core question:** do visitors buy, and do buyers come back?
| Metric | Formula | Why it matters |
|--------|---------|----------------|
| Conversion rate | orders ÷ unique visitors | The fundamental efficiency of the store |
| Average order value (AOV) | revenue ÷ orders | Second lever on revenue besides traffic |
| Annual repurchase rate | % of buyers purchasing again within 12 months | Decides your mode (below) |
| Cart abandonment | 1 − (orders ÷ carts created) | Where intent dies; checkout friction signal |
| Revenue per customer (RPC) | revenue ÷ customers in period | Conversion × AOV × repurchase in one number |
| Customer acquisition cost (CAC) | acquisition spend ÷ new customers | Must sit far below RPC over the customer's life |
**The mode decision.** The annual repurchase rate determines what kind of e-commerce business you are, and the playbooks differ sharply. Under roughly 40% of buyers repurchasing in a year, you're in **acquisition mode**: the game is top-of-funnel efficiency, conversion, and AOV — loyalty programs are wasted spend. Above roughly 60%, you're in **loyalty mode**: the game is repeat behavior, recommendations, reorder flows, and share of wallet. Between the two, run a **hybrid**: grow acquisition while nudging the repurchase rate up. Misdiagnosing the mode is the most expensive e-commerce metric mistake — building loyalty infrastructure for a one-and-done product (mattresses) or burning ad spend where retention should compound (consumables).
**Instrumentation notes:** instrument the full funnel as distinct events (product view → add-to-cart → checkout start → payment → confirmation) so abandonment localizes to a step; tie every order to an acquisition source at the *customer* level, not the session level; deduplicate customers across devices and guest checkouts (email is the usual join key); record refunds and returns as first-class events.
**Measurement failure modes:** celebrating traffic while conversion slides; reporting one blended conversion rate across channels with wildly different intent (brand search converts 10x display); ignoring returns, refunds, and shipping costs so "revenue" overstates reality; comparing holiday weeks to ordinary ones instead of year-over-year; and counting sessions as shoppers, which double-counts the same person comparison-shopping on two devices.
## SaaS
**Core question:** do trials become long-lived, expanding subscriptions?
| Metric | Formula | Why it matters |
|--------|---------|----------------|
| MRR / ARR | sum of active subscription value per month/year | The growth headline, normalized |
| Customer churn | customers lost ÷ customers at period start | The leak that caps growth |
| Revenue churn (gross/net) | MRR lost (gross); MRR lost − expansion (net) | Net churn below 0% means the base grows itself |
| LTV | ARPA × gross margin ÷ churn rate | What a customer is worth, honestly margined |
| CAC | sales + marketing spend ÷ new customers | What a customer costs, fully loaded |
| LTV:CAC | LTV ÷ CAC | >3 is the standard health line |
| CAC payback | CAC ÷ (ARPA × gross margin) | Months to recover acquisition; <12 is the heuristic |
| Trial-to-paid conversion | paying ÷ trials in cohort | Funnel efficiency; watch by channel |
| Time-to-value | median time from signup to first core value | The strongest onboarding lever on conversion |
| Expansion/upsell | MRR added from existing accounts | Cheapest growth; drives net churn negative |
**Instrumentation notes:** revenue numbers must come from the billing system, not the analytics tool — analytics events drop, double-fire, and miss dunning; model the subscription as a state machine (trial → active → past_due → paused → churned, plus upgrade/downgrade transitions) and emit an event per transition; define "active use" by core action so you can see *engagement churn* — accounts that stopped using but haven't stopped paying — which is next quarter's revenue churn, visible today.
**Measurement failure modes:** conflating MRR with cash collected (annual prepays inflate cash now and create a renewal cliff later — track both, label both); reporting gross churn as if it were net, or hiding gross churn behind healthy expansion; computing LTV without gross margin (an LTV of revenue, not profit); survivor-biased LTV from your oldest, happiest cohort; counting paused or downgraded accounts inconsistently across reports; and blending churn across plan tiers when enterprise and self-serve behave like different species.
## Free Mobile App
**Core question:** do downloads become engaged users, and does a small slice pay enough to carry everyone?
| Metric | Formula | Why it matters |
|--------|---------|----------------|
| Launch/activation rate | users reaching first value ÷ installs | Installs are shelf-ware until first value |
| D1 / D7 / D30 retention | % of install cohort active N days later | The shape of the leak |
| DAU/MAU | daily actives ÷ monthly actives | Habit strength; how many days a month users need you |
| % paying | payers ÷ active users | The size of the monetized slice |
| ARPDAU | daily revenue ÷ DAU | Revenue intensity, comparable across days |
| ARPPU | revenue ÷ paying users | Depth of monetization among payers |
| k-factor | invites per user × invite conversion | Organic distribution multiplier |
| Churn | users inactive >N days ÷ actives | Defines the ceiling on MAU growth |
**Instrumentation notes:** define "active" as performing the core action, not opening the app (push-notification-driven opens flatter DAU without value); fix your retention convention — calendar-day vs rolling 24-hour windows produce different numbers, and mixing them invalidates trends; expect platform/store reporting lag and reconcile attribution SDK installs against store-reported installs; track the revenue distribution, not just the mean — monetization usually concentrates in a small cohort of high spenders, so medians and deciles tell the truth that ARPPU hides.
**Measurement failure modes:** treating downloads as the growth metric (it's the top of a funnel that loses ~80%+ by week one); averaging revenue across all users when a tiny whale cohort dominates it; forgetting the platform's ~30% cut so "revenue" overstates net by nearly a third; comparing retention across cohorts acquired from incentivized installs vs organic (they're different populations); and celebrating DAU spikes caused by notification blasts that simultaneously spike opt-outs — the engagement equivalent of a payday loan.
## Media Site
**Core question:** can you grow attention and monetize it without destroying the reason it exists?
| Metric | Formula | Why it matters |
|--------|---------|----------------|
| Audience growth | unique visitors, trended + return-visitor rate | Reach, qualified by loyalty |
| Engaged time | actively-measured time on content | Honest attention; raw time-on-page lies |
| CTR | ad clicks ÷ ad impressions | What advertisers ultimately buy |
| RPM | revenue ÷ 1,000 pageviews (or sessions) | Monetization efficiency of attention |
| Ad inventory | impressions available per period | The supply you sell |
| Content-to-ad balance | ad density per page/session | The tradeoff dial between revenue and trust |
**Instrumentation notes:** measure engaged time with activity signals (scroll, focus, pointer movement), not last-page timestamps — a tab open in the background is not attention; attribute traffic by source and content type so you can see which content earns loyal readers vs drive-by social spikes; track the *return* rate of new visitors per content category — it tells you what to commission next.
**Measurement failure modes:** pageview inflation via pagination and slideshows — it manufactures ad inventory while destroying engaged time and reader trust, a vanity trade that monetizes the brand to death; counting bounce-heavy viral traffic as "audience" when it never returns; chasing CTR with clickbait that raises short-term clicks and long-term unsubscribes; and reporting RPM gains that came entirely from ad density increases (the counter-metric — return-visitor rate — catches this).
## User-Generated Content
**Core question:** do visitors climb the engagement funnel and create content that brings others back?
The defining tool is the **engagement funnel** — a ladder of deepening participation: **visitor → voyeur** (consumes content) **→ commenter/voter** (lightweight interaction) **→ creator** (posts original content) **→ sharer** (distributes it outward). Participation is radically unequal — in the spirit of the 90-9-1 rule, lurkers vastly outnumber contributors, who vastly outnumber creators — so the metrics that matter are the **tier sizes and the transition rates between tiers**, not the total account count.
| Metric | Formula | Why it matters |
|--------|---------|----------------|
| Tier transition rates | users moving up a tier ÷ tier population | The health of the participation ladder |
| Content per active user | items created ÷ active users | Density of the content economy |
| Time between contributions | median gap per creator | Creator habit strength |
| Notification effectiveness | re-engagement CTR ÷ opt-out rate | Your main lever, and its cost |
| Spam/fraud rate | bad content ÷ total content | The tax on everything else |
**Instrumentation notes:** define each tier by concrete actions and make tiers mutually exclusive per period so funnel math adds up; instrument content *quality* signals (votes, reports, removals) alongside volume; measure notification effects as a pair — every re-engagement campaign gets its opt-out cost attached.
**Measurement failure modes:** counting registered accounts when creators are what the system runs on; celebrating engagement minutes that are pure passive scroll while creation flatlines (the voyeur tier grows, the ladder is broken); excluding spam from no reports so content volume looks healthy while the real feed rots; and over-notifying — DAU rises for a month, then the opt-out wave caps your re-engagement channel permanently.
## Two-Sided Marketplace
**Core question:** liquidity — do listings meet buyers fast enough that both sides come back?
| Metric | Formula | Why it matters |
|--------|---------|----------------|
| Fill rate / sell-through | listings transacting within window ÷ listings | The supply side of liquidity |
| Search-to-fill | searches ending in a transaction ÷ searches | The demand side of liquidity |
| Time-to-transaction | median time from listing (or search) to deal | Liquidity's speed dimension |
| Buyer/seller ratio | active buyers ÷ active sellers | Which side is the constraint |
| Take rate | platform revenue ÷ GMV | How much of the flow you capture |
| GMV | total transaction value | Context only — vanity until × take rate |
| Repeat rate per side | % of each side transacting again in period | Whether liquidity converts to habit |
| Fraud/dispute rate | disputed transactions ÷ transactions | Trust, the marketplace's real product |
**Instrumentation notes:** define the transaction window per category — 30 days is meaningless if cars take 60 and concert tickets take 2; instrument both sides' funnels separately (seller: list → inquiry → transact; buyer: search → view → contact → transact) because the constraint side changes over time; measure liquidity *per market* (city, category), never as a global blend — a marketplace is a portfolio of local liquidity pools.
**Measurement failure modes:** GMV as the headline while take-rate revenue stays trivial; subsidized transactions (coupons, free credits) counted as organic liquidity; concentration blindness — one power seller or one corporate buyer making aggregate liquidity look healthy while the long tail gets nothing; growing the easy side (usually supply) while the constraint is demand, so listings balloon and fill rate collapses; and blended liquidity across markets hiding the fact that you have one working city and five dead ones.
## Hybrid Models
Most real businesses mix archetypes: a subscription box is e-commerce acquisition with SaaS retention economics; a marketplace may sell SaaS tools to its sellers; a media site may run a UGC community. The rule: **pick ONE primary model — the one whose failure kills the company — and let it own the OMTM.** The secondary model contributes supporting metrics and counter-metrics, not a second north star.
Deciding the primary: ask where the riskiest economics live. A subscription box dies from churn, not from cart abandonment → primary model SaaS, with AOV and cart metrics as supporting. A marketplace with seller tools dies if liquidity fails, no matter how good tool attach rates look → primary model marketplace until liquidity is proven, then revisit. If you'd describe yourself to an investor as "X with a bit of Y," X is the primary.
Two failure patterns to avoid: **dual-OMTM paralysis**, where two teams optimize two north stars and trade wins back and forth invisibly (the counter-metric structure exists precisely so the secondary model constrains rather than competes); and **model drift**, where the company quietly becomes the other model — the day your marketplace's SaaS-tool revenue exceeds take-rate revenue and retains better, re-run the model choice deliberately instead of letting the dashboard decide by inertia.
references/case-studies.md
# Case Studies: Lean Analytics in Practice
## Table of Contents
- [Case Study 1: From 34 Widgets to One Number](#case-study-1-from-34-widgets-to-one-number)
- [Case Study 2: The Marketplace That Measured the Wrong Thing](#case-study-2-the-marketplace-that-measured-the-wrong-thing)
- [Case Study 3: The App That Almost Bought Growth](#case-study-3-the-app-that-almost-bought-growth)
- [Key Takeaways](#key-takeaways)
## Case Study 1: From 34 Widgets to One Number
### Context
A 20-person B2B SaaS company sells proposal software to creative agencies. Two years in: ~$70K MRR, growing slowly. The leadership dashboard, assembled tile by tile over two years, shows 34 widgets — MRR, signups, sessions, NPS, feature usage charts, support volume, Twitter followers, blog traffic, and more.
### The Problems
**Meetings argued about interpretation, not action.** Every Monday, an hour dissolved into debating why sessions dipped or which definition of "active" was correct. Three teams used three different churn formulas. Nobody could say which number, if it halved, would constitute an emergency.
**The metrics never changed a decision.** Reviewing six months of roadmap choices, the founders found exactly zero that had been triggered by a dashboard number. Features shipped because customers asked loudly or competitors had them. The dashboard was a spectator.
**The flattering numbers masked the dangerous one.** Total signups and cumulative revenue climbed reliably up and to the right. Buried in a drill-down: monthly customer churn was 7.2%, meaning the company replaced more than half its customer base every year just to stay still.
### The Application
**Step 1: Audit every widget with the four tests.** Comparative, understandable, ratio or rate, behavior-changing. Nine of 34 survived; the rest were totals, cumulative curves, or numbers nobody could attach a decision to. The deleted tiles moved to an archive report — available, not ambient.
**Step 2: Name the model and the stage.** Model: SaaS, unambiguous. Stage: the team assumed Revenue ("we have MRR"). The cohort table said otherwise — retention curves didn't flatten; every cohort slid toward zero by month 10. Revenue with collapsing retention is Stickiness with a billing system. Stage: Stickiness.
**Step 3: Find the value moment.** Mining cohorts showed accounts that sent at least one real proposal in their first week retained at 3x the rate of those that didn't. An onboarding experiment that pushed trials toward sending a first proposal (template gallery plus a "send your first proposal" checklist) moved week-4 retention in the test group — evidence the behavior was causal, not just a marker of motivated customers.
**Step 4: Install the OMTM and counter-metric.** OMTM: **week-4 retention of new accounts**, defined as performing the core action (proposal sent) in days 22-28, cohorted weekly. Baseline: 31%. Counter-metrics: weekly trial signups (don't strangle the funnel while optimizing depth) and monthly churn (the lagging confirmation).
**Step 5: Draw the line in the sand.** 45% week-4 retention by end of Q2. Pre-committed responses, written in the metrics doc: hit → rotate the OMTM to net MRR churn and unfreeze paid acquisition tests; miss → a two-week diagnostic sprint, and if interviews showed the product served freelancers better than agencies, pivot the ICP.
**Step 6: Rebuild the dashboard and the ritual.** One screen: week-4 retention huge, with the 45% target line drawn on the chart; five small tiles below — trial signups, activation rate, churn, MRR, support load. Monday meeting agenda became three questions: what did the number do, which experiments touched it, what's the next bet.
### Results After One Quarter
| Metric | Before | After |
|--------|--------|-------|
| Dashboard widgets | 34 | 6 |
| Week-4 retention | 31% | 47% |
| Monthly customer churn | 7.2% | 4.4% |
| Monday metrics meeting | 60 min, inconclusive | 25 min, ends in decisions |
| Decisions traceable to metrics (per quarter) | 0 | 9 |
| MRR growth rate | 3%/mo | 5%/mo |
### Lessons Learned
1. **Deleting metrics created information.** With 34 widgets, every signal was ambient noise; with one number and a target, a bad week was unmissable and meant something.
2. **The stage diagnosis was the unlock.** Calling themselves "Revenue stage" had justified acquisition spend that churn quietly canceled. Admitting Stickiness redirected the same budget to onboarding, where it compounded.
3. **The counter-metric earned its keep immediately.** The first onboarding redesign gated features aggressively and trial signups dropped 14% — the guardrail caught a "win" that was strangling the funnel, and the second iteration fixed both.
## Case Study 2: The Marketplace That Measured the Wrong Thing
### Context
A two-sided marketplace for renting camera and film gear between owners and working videographers. Eighteen months in, operating in six cities. The investor update headlines: GMV up 19% quarter-over-quarter, total listings up 31%, 11,000 registered users. The seed round is being raised on a "growth" story.
### The Problems
**Both headline numbers were vanity.** GMV grew because listings grew; listings grew because signup promos rewarded posting gear. Neither number said whether renters found what they needed or whether owners earned anything.
**The experience contradicted the dashboard.** Renters searched, found nothing available nearby, and left. Owners listed three lenses, got zero inquiries for a month, and went dormant. Support tickets said "ghost town"; the dashboard said "up and to the right."
**Growth spend amplified the dysfunction.** Acquisition budget recruited more owners (the easy side), worsening the imbalance — more shelves, same few shoppers.
### The Application
**Step 1: Define liquidity for the category.** The team adopted two definitions: **fill rate** — % of listings rented at least once in 30 days — and **search-to-fill** — % of searches leading to a completed rental within 7 days. Gear rental is time-sensitive (shoots have dates), so the windows were set accordingly rather than copied from another marketplace.
**Step 2: Measure honestly, per market.** The blended numbers were grim: fill rate 9%, search-to-fill 11%. Per-city was worse: one city accounted for most transactions; in three cities, search-to-fill was under 4% — dead pools propped up by the blend. Concentration analysis showed 9% of listings produced 78% of rentals: a few well-stocked, responsive owners *were* the marketplace.
**Step 3: Choose the OMTM and counter-metrics.** OMTM: **search-to-fill in the two strongest cities** — demand-side liquidity, since searches were abundant and dying. Counter-metrics: median rental value (to prevent juicing fills with fire-sale pricing) and dispute rate (to catch quality collapse from rushed transactions). Baseline: 11%. Line in the sand: 25% in both focus cities within two quarters; miss → contract to one city and one gear category until liquidity worked somewhere.
**Step 4: Act against the vanity metrics.** The playbook deliberately *shrank* the flattering numbers: pause all acquisition spend outside the two focus cities; delist inventory inactive for 60 days (listings fell 38%); require availability calendars so search returned only rentable gear; recruit "anchor owners" with deep, in-demand inventory near production hubs; add date-first search.
**Step 5: Survive the optics.** The investor update explained why listings and GMV would fall while the business improved, replacing the growth story with a liquidity story: "marketplaces are bought on density, and we now measure it."
### Results After Two Quarters
| Metric | Before | After |
|--------|--------|-------|
| Search-to-fill (focus cities) | 11% | 27% |
| Fill rate, 30-day (focus cities) | 9% | 31% |
| Total listings | 8,400 | 5,200 |
| Repeat renters per month | 6% | 19% |
| Owner 90-day dormancy | 71% | 44% |
| GMV | flat, then +12%/quarter | compounding from repeat usage |
### Lessons Learned
1. **The right metric was the one that got worse before better.** Listings and GMV had to fall for liquidity to rise. A team graded on vanity numbers could never have made those moves.
2. **Liquidity is local.** The blended dashboard averaged one working city with five dead ones. Measuring per market turned "we're growing" into "we work in exactly one place — make it two."
3. **The constraint side is rarely the easy side.** Promos recruited supply because supply was easy. The business was demand-constrained the whole time, and only search-to-fill made that visible.
## Case Study 3: The App That Almost Bought Growth
### Context
A free mobile habit-tracking app with premium subscriptions. A seed round just closed, and the plan in the deck allocates $60K/month to paid user acquisition starting next month. Launch buzz delivered 40,000 installs organically.
### The Problems
**Retention was catastrophic and unexamined.** D1 retention: 38%. D7: 12%. D30: 4%. DAU/MAU: 9%. Against the ~14% D30 average for casual apps, the product sat well below par — but the deck quoted total installs.
**The UA plan multiplied a leak.** At those curves, paid installs would be 96% gone within a month. Modeled honestly, the $60K/month bought a treadmill: spend stops, "growth" stops, cash gone.
**Monetization math depended on retention that didn't exist.** Premium conversion happened almost exclusively after day 14 of use. With 8% of users surviving to day 14, the LTV model in the spreadsheet was fiction.
### The Application
**Step 1: Stage check, then freeze.** Model: free mobile app. Stage: Stickiness — unambiguously, given the curves. The founders froze the UA budget with a written unfreeze condition rather than an open-ended delay: **D7 ≥ 25% and DAU/MAU ≥ 20%, held for four consecutive weekly cohorts.**
**Step 2: Mine cohorts for the value moment.** Two behaviors separated retained users: setting a reminder in the first session (3x D30 retention) and tracking 3+ habits in week one (2.5x). Interviews with retained users added the why: the reminder *was* the product — people stayed because the app interrupted their day at the right moment.
**Step 3: Test causality.** Experiment 1: onboarding redesign making reminder setup the default path → D7 rose 12% → 19% — causal, productize. Experiment 2: prompting users to add three habits → no retention effect — a marker of pre-existing intent, not a lever. The team stopped pushing habit count.
**Step 4: Install the OMTM and guardrail.** OMTM: **D7 retention**, weekly cohorts, displayed against the 25% line. Counter-metric: notification opt-out rate — the obvious failure mode of a reminder-centric strategy is spam, and opt-outs permanently destroy the channel the retention strategy depends on.
**Step 5: Iterate inside the freeze.** Smart reminder timing (learn from completion times), a streak-repair flow (lapsed users churned out of shame — "repair your streak" brought a measurable share back), and a home-screen widget. Two cohort-weeks per iteration, each judged only on D7 and the opt-out guardrail.
**Step 6: Unfreeze deliberately.** After 14 weeks: D7 at 27%, D30 at 13%, DAU/MAU at 22%, opt-outs flat. UA unfroze at $15K/month — a test, not a flood — with channel-level cohort tracking and a new line in the sand: paid cohorts must hit D7 ≥ 20% per channel, or that channel stops.
### Results
| Metric | At funding | At UA unfreeze (14 weeks) |
|--------|------------|---------------------------|
| D7 retention | 12% | 27% |
| D30 retention | 4% | 13% |
| DAU/MAU | 9% | 22% |
| Notification opt-out rate | 11% | 12% (guardrail held) |
| Day-14 survivors (premium-eligible) | 8% | 21% |
| Projected payback on $60K/mo UA | never | ~11 months |
### Lessons Learned
1. **The freeze was the highest-ROI decision of the year.** Three months of planned UA at the old curves would have spent ~$180K renting users who left. Stickiness first made the same dollars buy compounding users later.
2. **Correlation mining found two candidates; experiments kept one.** Without the causality test, the team would have spent a quarter pushing habit counts — a metric that moved nothing.
3. **The counter-metric shaped the solution space.** Knowing opt-outs were the guardrail pushed the team toward *smarter* notifications instead of *more* notifications — the lazy path was closed in advance.
## Key Takeaways
1. **The OMTM is a focusing device, not a data strategy.** All three teams kept collecting broad data; what changed was that one number, with a target and a date, decided what the company did next.
2. **Vanity numbers can shrink while the business improves.** Deleted widgets, delisted inventory, frozen install campaigns — each case required making a flattering number worse to make the real one better.
3. **Stage discipline protects runway.** Every case contains the same near-miss: spending scale-stage money on a stickiness-stage product. The gates exist to make that error visible before the wire transfer.
4. **Counter-metrics keep wins honest.** Trial signups, median rental value, notification opt-outs — in each case the guardrail caught or prevented a "win" that would have quietly cost more than it paid.
5. **Lines in the sand turn analytics into decisions.** Pre-committed targets and miss responses meant results — good or bad — executed a plan instead of starting a debate.
references/five-stages.md
# The Five Stages of Lean Analytics
## Table of Contents
- [How the Stages Work](#how-the-stages-work)
- [Stage 1: Empathy](#stage-1-empathy)
- [Stage 2: Stickiness](#stage-2-stickiness)
- [Stage 3: Virality](#stage-3-virality)
- [Stage 4: Revenue](#stage-4-revenue)
- [Stage 5: Scale](#stage-5-scale)
- [Stage, Funding, and Runway](#stage-funding-and-runway)
- [Diagnosing Your Stage](#diagnosing-your-stage)
## How the Stages Work
Lean Analytics sequences a startup's life into five stages — **Empathy, Stickiness, Virality, Revenue, Scale** — each answering one question, each gated by evidence. The gates exist because the stages compound: virality multiplies whatever retention you have (multiply a leak, get a bigger leak), and paid acquisition multiplies whatever unit economics you have (scale negative margins, get faster death). Three rules govern the system:
1. **Gates are evidence, not time.** You don't age into a stage. You exit with data: a flattening retention curve, a payback period inside tolerance.
2. **Your OMTM is stage × model.** The stage names the question; your business model names the metric that answers it.
3. **Movement is bidirectional.** A pivot, a new segment, or a collapsed metric sends you back. Going back early is cheap; refusing to is how runways end.
## Stage 1: Empathy
**The question:** have we found a real problem, painful and frequent enough that identifiable people will pay to fix it?
**What to measure:** this stage is mostly qualitative, and that's correct — at zero volume, quantitative metrics are noise with decimal points. The instrument is the problem interview: 15+ conversations per target segment, scored for pain (do they describe it with emotion and specifics?), frequency (weekly beats yearly), budget (do they already pay for or hack around a solution?), and reachability (can you find more people like this?). Weak quantitative signals — landing-page conversion, waitlist signups from a concept ad — are useful smoke tests, not proof. Track interview findings in a simple tally: how many of the last 15 interviewees confirmed the problem unprompted?
**Exit criteria:**
- [ ] 15+ problem interviews in one named segment, without pitching the solution
- [ ] A majority describe the problem as painful, frequent, and currently costing them money or time
- [ ] They've tried workarounds — spreadsheets, hires, competing tools (apathy is the kiss of death; existing hacks are demand)
- [ ] Solution interviews produce real commitment: time, data, a pilot, a deposit — not compliments
- [ ] You can describe exactly who has the problem and where to find a thousand more of them
**Premature-scaling symptoms:** writing code for months before the first interview; buying ads to a problem nobody confirmed; hiring sales for a pitch that hasn't survived 15 conversations; mistaking friends' politeness for validation.
**Funding/runway:** the cheapest stage — burn conversation hours, not cash. Pre-seed money here buys interviews, prototypes, and smoke tests. Raising a large round at Empathy converts unvalidated guesses into payroll.
## Stage 2: Stickiness
**The question:** do people use the product repeatedly, of their own accord?
**What to measure:** retention cohorts on the core value action (not logins), DAU/MAU for habit-shaped products, time-to-value for new users, and frequency of the core action among retained users. Two cross-checks help calibrate: the Sean Ellis product-market-fit survey ("how would you feel if you could no longer use this?" — around 40%+ answering "very disappointed" is the classic threshold), and for apps, the 30/10/10 heuristic — roughly 30% of signups active monthly, 10% daily is a strong showing.
**Exit criteria:**
- [ ] Cohort retention curves flatten — they stop decaying and hold at a floor that supports the business model
- [ ] Newer cohorts retain as well as or better than older ones (the product is improving, not just the audience)
- [ ] Users return without prompting — organic return visits, not notification-driven spikes
- [ ] You know your value moment: the early behavior causally linked (tested, not just correlated) to long-term retention
- [ ] Engagement is concentrated in the segment you intend to build for
**Premature-scaling symptoms:** spending on acquisition or building referral loops while the bucket leaks; shipping breadth (new features, new platforms) when depth (the core loop) hasn't proven habit-forming; celebrating MAU growth driven entirely by top-of-funnel while cohort curves slide to zero.
**Funding/runway:** the longest stage for most companies — budget runway for several product iterations, not one. This is the worst stage to raise a growth round: money arrives with growth expectations the retention can't support, and the spend pressure starts the leaky-bucket fire.
## Stage 3: Virality
**The question:** do users bring other users — sustainably and cheaply enough to change your acquisition math?
**What to measure:** the viral coefficient k = invitations sent per user × conversion rate of invitations, and **viral cycle time** — how long a generation takes. Cycle time is the under-appreciated half: growth compounds per cycle, so shortening the cycle from weeks to days often outgrows raising k. Distinguish three kinds of virality: **inherent** (the product works better when shared — documents, payments, multiplayer), **artificial** (incentivized invites — bought, and it shows in invited-cohort quality), and **word-of-mouth** (untracked praise — survey "how did you hear about us?" to estimate it). Even k < 1 is valuable: each acquired user yields 1/(1−k) total users, so k = 0.5 doubles every acquisition channel's efficiency.
**Exit criteria:**
- [ ] k measured from instrumented invite flows, not inferred from growth wishes
- [ ] Invited users retain comparably to organic users (counter-metric — incentives often recruit tourists)
- [ ] Virality meaningfully discounts blended CAC, with cycle time short enough to compound within a quarter
- [ ] The viral loop is inherent to product use, or its incentive costs are sustainable at scale
**Premature-scaling symptoms:** paying for incentivized invites that bring low-retention users and poison cohort data; bolting share buttons onto a product nobody's attached to (virality is a multiplier on love, not a substitute); optimizing k while cycle time stays at six weeks.
**Funding/runway:** virality work is cheap relative to paid acquisition — mostly product iterations. The risk isn't burn; it's time lost polishing loops on top of weak stickiness. If k stalls below ~0.2-0.3 after honest attempts, take the answer: your growth will be paid or content-led, and that's a Revenue-stage problem, not a failure.
## Stage 4: Revenue
**The question:** does a dollar in produce more than a dollar out — soon enough to survive the gap?
**What to measure:** revenue per customer (or ARPU/ARPA), conversion to paid, CAC fully loaded, **CAC payback** (months of gross-margin contribution to recover CAC — under 12 months is the standard SaaS heuristic), LTV:CAC (>3 as the health line), gross margin, and churn's effect on revenue (net vs gross). The mindset shift: before this stage you optimized for learning and love; now you optimize a machine — money goes in via acquisition, comes out via margin, and the ratio plus the cycle speed is the whole game.
**Exit criteria:**
- [ ] Unit economics positive at realistic volume assumptions, not best-case ones
- [ ] CAC payback inside your runway tolerance (an 18-month payback with 12 months of cash is a death sentence in slow motion)
- [ ] Pricing has been tested — at least one deliberate experiment, not a launch-day guess carried forever
- [ ] Revenue concentration is survivable (no single customer or channel whose loss ends the company)
- [ ] Margins hold after support, infrastructure, and discounting are honestly allocated
**Premature-scaling symptoms:** scaling ad spend while payback exceeds runway; discounting to manufacture growth that evaporates at renewal; hiring a sales team before founders have repeatedly closed deals themselves; reporting GMV or bookings growth while contribution margin stays negative and unexamined.
**Funding/runway:** the stage where "default alive" becomes computable — model whether current growth and margins reach profitability before zero cash. Raise to *accelerate working economics*, not to discover them. Investors at this stage buy your unit economics plus a credible multiplication plan; a raise that papers over negative unit economics just buys a bigger crater.
## Stage 5: Scale
**The question:** can the machine grow through new channels, partners, and markets without breaking what made it work?
**What to measure:** channel-level economics (CAC, payback, and churn *per channel* — blended numbers hide dying channels inside growing ones), partner- and platform-sourced revenue, market share within the segment, expansion revenue, and operational health: support load per customer, uptime, margin at volume, hiring velocity vs quality. Analytics itself changes shape — from one company-wide OMTM to a metrics hierarchy where each team owns a number that ladders into the top-line goal, with reporting discipline (definition docs, owners, cadences) keeping the numbers trustworthy as headcount grows.
**Exit criteria** (Scale doesn't exit upward — these confirm you belong here):
- [ ] At least two acquisition channels with independently healthy economics
- [ ] Growth doesn't degrade the core: retention and NPS-style signals hold as volume rises
- [ ] Operations scale sub-linearly — support tickets and infra cost grow slower than customers
- [ ] New-market entries are deliberate experiments with their own lines in the sand
**Premature-scaling symptoms — and the inverse failure:** classic premature scaling is arriving here early (the symptoms listed in every prior stage). The inverse failure is real too: a company that has passed every gate but keeps tinkering with onboarding instead of opening channels is hiding from execution risk. Passing the Revenue gate creates an obligation to scale.
**Funding/runway:** growth rounds belong here — the money multiplies proven loops. Diligence will probe exactly what this framework tracks: cohort retention, channel-level CAC/payback, net churn. A company managed by these stages walks into diligence with the data room already true.
## Stage, Funding, and Runway
| Stage | Sane funding posture | What the money buys | Red flag |
|-------|---------------------|---------------------|----------|
| Empathy | Pre-seed / none | Interviews, prototypes, smoke tests | Big raise pre-validation → payroll on guesses |
| Stickiness | Seed | Product iterations toward flat retention curves | Growth round arrives, growth pressure starts the burn |
| Virality | Seed / bridge | Loop experiments, instrumentation | Buying incentivized invites to fake organic growth |
| Revenue | Seed extension / Series A | Pricing tests, channel tests, payback proof | Scaling spend while payback > runway |
| Scale | Series A/B+ | Channel expansion, team, new markets | Raising to hide deteriorating cohort economics |
The general law: **raise on a passed gate, spend on the next one.** Each stage's evidence is the next round's pitch, and runway should cover 2-3 full iteration cycles of the current stage's loop — one cycle of cash means one roll of the dice.
## Diagnosing Your Stage
Walk the gates bottom-up; the first one you cannot evidence is your stage, regardless of what the org chart or the fundraising deck says.
1. Can you show 15+ interviews proving a painful, paid-for problem? No → **Empathy.**
2. Do cohort retention curves flatten at a viable floor? No → **Stickiness.**
3. Is there revenue with measured CAC, payback, and margin? No → **Revenue** (visit Virality on the way only if users plausibly bring users; many fine businesses skip it).
4. All of the above at volume, with channel economics holding? → **Scale.**
Common misdiagnoses: *"We have revenue, so we're at Revenue"* — revenue with collapsing retention means you're at Stickiness with a billing system; *"Growth stalled at Scale"* — usually a Stickiness regression in a new segment or channel, so cohort the new population separately and re-walk the gates for it; *"We're raising a growth round, so we're at Scale"* — funding stage and evidence stage are independent variables, and the gap between them is exactly the danger zone this framework exists to close.
references/good-metrics.md
# Good Metrics vs Vanity Metrics
## Table of Contents
- [The Four Tests of a Good Metric](#the-four-tests-of-a-good-metric)
- [The Vanity Rewrite Table](#the-vanity-rewrite-table)
- [The Four Lens Pairs](#the-four-lens-pairs)
- [Cohort Analysis: Building and Reading a Retention Table](#cohort-analysis-building-and-reading-a-retention-table)
- [Segmentation Discipline](#segmentation-discipline)
- [From Correlation to Causation: The Experiment Loop](#from-correlation-to-causation-the-experiment-loop)
- [The Metric Definition Doc](#the-metric-definition-doc)
## The Four Tests of a Good Metric
Apply these four tests to every number on your dashboard. Archive anything that fails two or more. A metric that fails the fourth test fails, period.
**1. Comparative.** A good metric lets you compare — to last week, to another cohort, to a competitor's benchmark, to your line in the sand. "Conversion is up 0.4 points week-over-week" is a sentence you can act on; "we have 3,000 signups" is not, because there is nothing to compare it to. In practice: attach a comparison frame to every metric before it goes on a dashboard. If you can't name the frame (previous period, other cohort, other segment, target), the metric is decoration.
**2. Understandable.** If people can't remember the metric and argue about it at lunch, it won't change the culture. A "blended weighted engagement index v3" might be statistically sophisticated, but nobody will fight for it. Test: ask three people on the team to define the metric without looking it up. If you get three definitions, either simplify the metric or write a definition doc (see the template below) and socialize it until the definitions converge.
**3. A ratio or rate.** Ratios and rates are the analyst's power tool for three reasons. They are inherently comparative: a daily conversion rate compared to the monthly average shows movement immediately. They are operable: you drive a car by speed (distance per time), not by total miles driven. And they expose tensions: LTV:CAC compresses "are customers worth what they cost?" into one number; sales-per-rep against refunds-per-rep shows whether you're selling well or just selling hard. Convert totals to rates as a habit: signups → signup-to-active conversion; revenue → revenue per customer; content → posts per active creator.
**4. Behavior-changing.** The master test: *what will we do differently based on this number?* If you cannot answer, stop measuring it. The strongest form is a pre-registered decision rule: "If day-7 retention is under 20% we rework onboarding; if it's over 30% we start the referral experiment." A metric attached to a decision is an instrument; a metric without one is a souvenir.
**Quick audit checklist for any metric:**
- [ ] Named comparison frame (period, cohort, segment, or target)
- [ ] One-sentence definition the whole team repeats identically
- [ ] Expressed as a ratio or rate, not a running total
- [ ] An "if it goes up/down, we will..." decision attached
- [ ] Cohorted where time-based, segmented where mixed populations exist
- [ ] Counter-metric identified if anyone is incentivized to move it
## The Vanity Rewrite Table
Vanity metrics share a signature: they rise with time and spend, regardless of whether the business is improving. Rewrite each one into its actionable counterpart instead of deleting it — the underlying data is usually fine; the framing is the problem.
| Vanity metric | Why it flatters | Actionable counterpart |
|---------------|-----------------|------------------------|
| Hits / page views | Counts loads, not people or value | Conversion or goal completion per visit |
| Visits | Can be one person refreshing | Return visitor rate; visits per active user |
| Unique visitors | Says nothing about what they did | Visitor → signup → active funnel rates |
| Total signups | Cumulative; includes the dead | % of signups active in last 7/30 days |
| Total registered users | Only ever goes up | Active users (DAU/WAU/MAU) and their trend |
| Downloads | Installation isn't usage | Launch rate; day-1/7/30 retention |
| Followers / likes | Audience, not behavior | Click-through and conversion from social |
| Emails collected | A list isn't demand | Open → click → purchase rate of the list |
| Time on site | Could be engagement or confusion | Engaged time on key pages; task completion rate |
| Cumulative revenue | Hides slowing growth | MRR growth rate; revenue per customer per cohort |
The rewrite pattern is consistent: **totals become rates, audiences become behaviors, collection becomes usage.** When an investor or executive asks for the vanity number, give it — followed immediately by the ratio that says whether it matters.
## The Four Lens Pairs
Good analytics alternates between complementary lenses. Knowing which lens you're using prevents category errors like running a survey to get a conversion rate or A/B testing your way to a vision.
**Qualitative vs quantitative.** Qualitative input (interviews, support conversations, watching sessions) is unstructured, anecdotal, and revealing — it tells you *why*. Quantitative data tells you *how much* and *how many*. Early on (Empathy stage), qualitative dominates because you have no volume; later, every quantitative anomaly should trigger a qualitative investigation. Rule of thumb: numbers tell you where to look; conversations tell you what you're looking at.
**Exploratory vs reporting.** Reporting metrics keep the lights on: they're the known numbers you track on a cadence. Exploratory analysis digs through data for unexpected patterns — the "unknown unknowns" where unfair advantages hide. The classic example from the book: Circle of Friends, a general social network, explored its data and found that mothers had wildly outsized engagement — message lengths, attachment rates, return visits. The company pivoted to Circle of Moms and found traction. Schedule exploratory time deliberately; dashboards never volunteer surprises.
**Leading vs lagging.** A lagging metric (churn, quarterly revenue) reports history; a leading metric (complaint volume, day-3 trial activity, NPS-style intent) predicts it. Lagging metrics are easier to trust and useless to steer by. To find leading indicators: take a lagging outcome (churned vs retained at day 90), look backward at what each group did in their first week, and find the early behaviors that separate them — then validate with an experiment, because separation alone is just correlation.
**Correlated vs causal.** Ice cream sales correlate with drownings; summer causes both. Correlations are valuable — they tell you where to dig — but acting on them as if they were causes wastes whole quarters. Causality requires changing one variable for a randomized group while holding everything else steady. Correlation is good; causality is a superpower.
## Cohort Analysis: Building and Reading a Retention Table
Blended averages lie. If January's users retain terribly and April's retain well, the monthly active number can look flat while the product is actually improving fast. Cohorts fix this by comparing groups at the same age.
**Procedure:**
1. **Pick the cohort key.** Usually signup week or month. Weekly for young products with volume; monthly when volume is thin (cohorts under ~100 users produce noise, not signal).
2. **Define "retained."** Prefer the core value action over mere login — "created a document," "sent a message," "placed an order." Logins measure habit at best, crash loops at worst.
3. **Build the table.** Rows are cohorts, columns are periods since signup (month 0, 1, 2...), cells are the percentage of that cohort performing the action in that period.
4. **Read down the columns.** Same-age comparison: is month-1 retention better for newer cohorts than older ones? If yes, your product changes are working — this is the single clearest signal that the product is improving.
5. **Read across the rows.** Decay shape: where does each cohort lose people, and does the curve flatten? A flattening curve means a durable core audience exists; a curve sliding to zero means a leaky bucket no matter how fast the top of the funnel grows.
**Worked example** (cell = % of cohort performing the core action in that month):
| Signup cohort | M0 | M1 | M2 | M3 | M4 |
|---------------|-----|-----|-----|-----|-----|
| January | 100% | 42% | 31% | 26% | 24% |
| February | 100% | 45% | 33% | 28% | — |
| March | 100% | 51% | 40% | — | — |
| April | 100% | 58% | — | — | — |
Reading it: M1 retention climbs 42% → 58% across cohorts — the onboarding changes shipped in February and March are working. January's curve flattens around 24% — there is a real retained core, and ~24% is your current baseline for "long-term retained." Meanwhile the blended MAU chart for the same period looked flat, because shrinking old cohorts offset improving new ones. This is why cohort tables, not aggregate actives, are the honest view.
**Common cohort mistakes:** cohorts too small to be stable; counting logins instead of value actions; comparing a complete month against a partial one (the current period always looks like a crash); and changing the "retained" definition mid-stream without re-computing history.
## Segmentation Discipline
A segment is any group sharing a characteristic: acquisition channel, plan, device, geography, persona, company size. Segmentation is how you stop averages from lying about populations the way cohorts stop them lying about time.
- **Segment before you celebrate or panic.** A flat aggregate frequently decomposes into one segment soaring and another collapsing. The action implied by "flat" is wrong in both directions.
- **Standard first cuts:** acquisition channel, platform/device, price tier, geography, and use case. Run these five before inventing exotic segments.
- **Hunt 2x differences.** A segment converting 10% better is probably noise or seasonality; a segment retaining 2-3x better is a strategy (double down, reposition, or re-target).
- **Set a minimum segment size** (a few hundred users or events, depending on base rates) before drawing conclusions — small segments generate impressive, meaningless ratios.
- **Watch for Simpson's paradox:** the aggregate trend can point the opposite direction of every individual segment when the mix shifts. If paid traffic (low conversion) grows faster than organic (high conversion), overall conversion can fall while both channels improve.
- **Tie segments to action.** A segment is only worth tracking if you would treat it differently: different onboarding, different pricing, different ad spend, or an explicit decision to ignore it.
## From Correlation to Causation: The Experiment Loop
The loop that turns observed patterns into reliable levers:
1. **Mine.** Find behaviors correlated with the outcome you want. Example: "users who import contacts on day 1 retain 3x better at day 30."
2. **Hypothesize both directions.** Either importing *causes* retention (it creates value), or intent causes both (serious users import *and* retain). Write both down; the second is usually true more often than teams hope.
3. **Design the controlled test.** Randomly assign new users to a variant that drives the behavior (an onboarding step, a prompt, an incentive) and a control that doesn't. Change only this one thing.
4. **Pre-register the call.** Success metric, minimum effect size worth shipping, sample size, and end date — written before launch. Decide now what result kills the idea.
5. **Decide and iterate.** Effect appears → the behavior is (at least partly) causal; productize the nudge and re-measure. No effect → the behavior was a marker of intent; go back to mining, and consider targeting the *value moment* the behavior was standing in for.
In the example: forcing contact imports moved imports from 18% to 64% of new users — and day-30 retention didn't budge. Importing was a proxy for pre-existing intent. The team instead identified the actual value moment (first reply received within 48 hours) and engineered toward that, which did move retention.
**Hygiene rules:** don't peek and stop early when the numbers look good; one experiment tests one variable; expect novelty effects to fade (re-measure after a few weeks); and if the split of users between variants doesn't match what you configured, distrust the entire result.
## The Metric Definition Doc
When two teams compute "churn" differently, every meeting becomes an argument about arithmetic instead of a decision. Write a one-page definition for every metric that appears on a shared dashboard — especially the OMTM and its counter-metric.
```
Metric name: Week-4 Retention (New Accounts)
Plain-English: Of accounts created in a given week, the share still
performing the core action during week 4 after signup.
Formula: accounts with ≥1 core action in days 22-28 ÷ accounts
created in cohort week
Core action: "proposal_sent" event (not login)
Source: events.proposal_sent, accounts.created_at (warehouse,
dbt model retention_weekly)
Cohorting: weekly signup cohorts; exclude internal/test accounts
Segments tracked: acquisition channel, plan tier, team size
Owner: Head of Product (reviews weekly, signs off changes)
Counter-metric: weekly trial signups (must not fall >10% while we
optimize retention)
Current baseline: 31% (trailing 6-cohort average)
Line in the sand: 45% by end of Q2; if missed → rebuild onboarding
around first-proposal moment and revisit ICP
Review cadence: Monday metrics email + monthly deep dive
```
Two rules make the doc effective. First, **any change to a formula requires recomputing history** — otherwise trend lines silently mix definitions and become fiction. Second, **the owner owns the definition, not the result** — punishing the owner for the number's direction guarantees creative arithmetic. The doc's job is to make the number boring, stable, and trusted, so the arguments can be about what to do next.
references/omtm.md
# The One Metric That Matters
## Table of Contents
- [Why One Metric](#why-one-metric)
- [Choosing the OMTM Step by Step](#choosing-the-omtm-step-by-step)
- [The Stage × Model Matrix](#the-stage--model-matrix)
- [Counter-Metric Pairing](#counter-metric-pairing)
- [Drawing the Line in the Sand](#drawing-the-line-in-the-sand)
- [Communicating the OMTM](#communicating-the-omtm)
- [Rotation Triggers](#rotation-triggers)
- [Worked Examples](#worked-examples)
## Why One Metric
The One Metric That Matters is the single number you optimize above all others at your current stage. Four things happen when a team commits to one:
1. **It answers the most important question you have.** A startup is a stack of risky assumptions; the OMTM measures the riskiest one still unproven. Choosing it forces the team to name that risk out loud.
2. **It forces a line in the sand.** One metric invites one target. "Improve engagement" survives forever; "week-4 retention to 45% by June 30" can succeed or fail.
3. **It focuses the entire company.** When everyone knows the number, every project pitch, support policy, and design debate gets evaluated against the same question: does this move it?
4. **It builds a culture of experimentation.** A visible number that must move invites bets, measurements, and honest post-mortems instead of opinion battles.
Two clarifications prevent misuse. *One metric that matters* does not mean *collect only one metric* — you instrument broadly and drill into many numbers; you *watch* one. And the OMTM is temporary by design: it's the metric that matters **now**, and graduating past it is the goal.
## Choosing the OMTM Step by Step
1. **Name your business model.** One of the six archetypes: e-commerce, SaaS, free mobile app, media, user-generated content, two-sided marketplace. Hybrids pick a primary (see business-model-metrics reference).
2. **Name your stage.** Walk the five gates — Empathy, Stickiness, Virality, Revenue, Scale — from the bottom; the first gate you haven't passed is your stage (see five-stages reference).
3. **Read the candidate from the matrix below** and adapt it to your product's actual mechanics: the "core action" in a retention metric must be *your* value moment, not a generic login.
4. **Make it pass the four tests.** Comparative, understandable, a ratio or rate, behavior-changing. Almost always this means a rate over a recent window, cohorted.
5. **Pair a counter-metric** so the OMTM can't be gamed (next section).
6. **Draw the line in the sand** — target, date, miss response — and publish all of it in one place the whole company sees.
If the team cannot agree on step 1 or 2, stop: the disagreement is not about analytics, it's about what business you're in and what could kill it. That conversation is worth more than any dashboard, and it must end in a decision.
## The Stage × Model Matrix
Empathy-stage companies share the same OMTM regardless of model — validated problem signal from interviews (count of interviewees confirming pain, frequency, and willingness to pay). Scale-stage companies converge too — channel-level unit economics and operational health. In between, the model differentiates:
| Model | Stickiness OMTM | Virality OMTM | Revenue OMTM |
|-------|-----------------|---------------|--------------|
| E-commerce | Repeat-purchase rate; cart completion | Shares/referrals per buyer that convert | Revenue per customer; AOV × repurchase |
| SaaS | Trial activation rate; week-4 retention | Invites per account × acceptance rate | MRR growth; net churn; LTV:CAC |
| Free mobile app | D1/D7/D30 retention; DAU/MAU | k-factor; invite cycle time | ARPDAU; % paying |
| Media | Return-visitor rate; engaged time | Shares per article; social referral % | RPM; sell-through of inventory |
| UGC | Voyeur → creator conversion; content per user | Invites/embeds per creator | Premium conversion; ARPU |
| Marketplace | Repeat listing/buy rate per side | Seller- and buyer-referred signups | Net take-rate revenue per transaction |
Use the matrix as a menu, not a mandate. The right cell still needs translating into your product's vocabulary, and occasionally the honest answer sits one cell over — a marketplace whose sellers churn instantly has a stickiness problem even if its dashboard says "virality stage."
## Counter-Metric Pairing
Any metric a team optimizes hard will be hit — sometimes by improving the business, sometimes by quietly damaging it. The counter-metric is the guardrail that catches the second case. Choose it by asking: *how would a cynical team hit the OMTM while hurting the company, and which number would betray them?*
| OMTM | Gaming risk | Counter-metric |
|------|-------------|----------------|
| Signup growth | Buy junk traffic, inflate top of funnel | 30-day retention of new cohorts |
| Activation rate | Force users through hollow checklist steps | Week-4 retention; support tickets per new user |
| Sales velocity | Overselling, discount abuse | Refund/return rate; 90-day churn of new deals |
| Engagement (time in app) | Dark patterns, infinite feeds | Task completion time; session value rating |
| Email-driven revenue | Send more, burn the list | Unsubscribe + spam-complaint rate |
| Marketplace fill rate | Delist anything slow, hide breadth | Listing growth in target categories; dispute rate |
| Cost per acquisition | Chase cheap, low-intent users | LTV of acquired cohorts by channel |
Display the counter-metric next to the OMTM, always — same dashboard, same weekly email. A win that breaches the guardrail is not a win, and the team should hear that from the dashboard before they hear it from customers.
## Drawing the Line in the Sand
A line in the sand converts a metric into a falsifiable bet. It has three parts, all written **before** you start optimizing:
1. **The target.** Derive it from three inputs: your current baseline (measure it first — even if embarrassing), external benchmarks for your model (as starting heuristics), and *need* — the number at which the next stage, the next funding round, or default-alive economics become real. When the three conflict, need wins: a benchmark can't pay your bills.
2. **The date.** Tie it to runway and iteration speed. A useful target is reachable within 2-3 experiment cycles, not one heroic quarter.
3. **The pre-commitment.** What happens if you hit it (advance to the next stage's OMTM; unfreeze the growth budget) and what happens if you miss (iterate with a specific focus, pivot the segment, or kill the initiative). Writing the miss response in advance is the entire point — after the fact, every miss can be rationalized into "almost."
Template:
```
OMTM: Week-4 retention (new accounts, core action basis)
Today: 31%
Line in the sand: 45% by June 30
If we hit it: Move OMTM to net MRR churn; unfreeze paid acquisition tests
If we miss it: Two-week diagnosis sprint; if interviews show wrong ICP,
pivot target segment; no new feature work until decided
Counter-metric: Weekly trial signups must stay within 10% of current
Owner: CEO (reviewed in Monday metrics email)
```
"Good enough" deserves emphasis. Perfectionists keep optimizing a passed gate; optimists declare victory at any uptick. The pre-committed target defines *enough* so the company knows when to stop polishing one stage and start risking the next.
## Communicating the OMTM
A chosen-but-hidden OMTM changes nothing. Make it environmental:
- **Dashboard design: one big, 4-6 small.** The OMTM renders as a single large number with its trend and the line-in-the-sand target drawn on the chart. Below it, 4-6 supporting metrics in small tiles — the counter-metric always among them, plus the 3-5 drivers the team can directly move. Everything else lives in drill-down reports. If your dashboard tool shows 30 tiles, your dashboard is a filing cabinet, not a scoreboard.
- **The weekly metrics email.** One paragraph in plain language: the OMTM's value, the delta, the experiments that touched it, and the single biggest thing happening next. Written by the owner, readable by a new hire.
- **Experiment review anchored on the OMTM.** Every experiment proposal states its predicted effect on the OMTM (or explicitly claims counter-metric/infrastructure status). Every review starts with what the OMTM did.
- **Pitch hygiene.** Roadmap items, sales promises, and design debates get one standard question: "what does this do to the number?" Not everything must move it — but everything must answer the question.
## Rotation Triggers
The OMTM rotates when the question it answers stops being the riskiest one. Legitimate triggers:
- **You passed the line in the sand and held it** for several consecutive cohorts or weeks — graduation, the happy path. Move to the next stage's metric.
- **You pivoted.** New model or segment means re-deriving model × stage from scratch; yesterday's OMTM is now someone else's metric.
- **The metric saturated.** It's high, stable, and experiments barely move it while a different constraint visibly throttles the business. Rotate toward the constraint.
- **It stopped changing behavior.** If three consecutive reviews produced no decision tied to the number, either re-attach decisions or admit the risk lives elsewhere.
One illegitimate trigger, named explicitly: **the number looks bad and the date is near.** Rotating away from a failing OMTM is goalpost-moving. The pre-commitment exists precisely for this moment — execute the miss response instead.
Expect a healthy early-stage company to rotate every one to three quarters. Faster usually means thrashing; a year on one metric usually means nobody is looking at it anymore.
## Worked Examples
**1. B2B SaaS — CRM for landscaping companies (14-day trial).** Model: SaaS. Stage: stickiness — trials sign up but churn after converting. Mining showed trials that scheduled ≥5 jobs in week 1 converted and retained at 3x the average. An onboarding experiment that drove job-scheduling moved retention, so the behavior is causal enough to bet on. **OMTM:** % of new trials scheduling ≥5 jobs in week 1 (currently 22%). **Counter-metric:** trial-to-paid conversion and 60-day churn (to catch hollow activation). **Line in the sand:** 40% by quarter end; miss → rebuild setup flow around importing existing client lists, the step where most trials stall.
**2. Two-sided marketplace — vintage furniture.** Model: marketplace. Stage: stickiness/liquidity — GMV grows from new listings, but buyers search and leave. **OMTM:** % of new listings that sell within 30 days (currently 14%). **Counter-metrics:** median sale price (to prevent hitting the target by forcing fire-sale pricing) and dispute rate. **Line in the sand:** 35% in the two launch cities by Q3; hit → expand to two more cities with the same playbook; miss → narrow to the three categories with proven demand and delist the rest.
**3. Free mobile app — habit tracker.** Model: free mobile app. Stage: stickiness, despite investor pressure to spend on installs. **OMTM:** D7 retention (currently 12%). **Counter-metric:** notification opt-out rate — the obvious gaming path is spamming reminders. **Line in the sand:** D7 ≥ 25% and DAU/MAU ≥ 20% before any paid acquisition; miss after three onboarding iterations → revisit the core loop (the product, not the marketing, is the problem). The pre-commitment here is mostly a *spending freeze*: it protects the runway from buying users who would leave.
Three different companies, one shape: a single rate, cohorted, with a guardrail, a number, a date, and a decision already made about both outcomes.