What metrics to track and how to define them

1 The bottom line

Pick one main metric that tracks the value users get, define it in plain terms, and recheck it as the product changes. The main metric steers everything the team builds, so a wrong one pulls the whole product off course.

2 Key takeaways

• Choose your main metric on purpose. Once it is set, the team works on little else.
  • At Replika, revenue became the main metric, and Eugenia says it became very hard to focus on anything else. Eugenia Kuyda, Wabi47:51
• Tie the main metric to the outcome the user wants. If users get the outcome, they stay.
  • WeWard A/B tests every feature against steps, and a feature that locks social apps until a step goal is reached lifted steps by 10%. Yves Benchimol, WeWard17:45
• Define engagement as coming back, and write the core value as a short phrase. Time in app can mislead, and a clear phrase makes the metrics measurable.
  • For a VPN utility, typical engagement is about half a minute or less per session. Tanuj Chatterjee, Super Unlimited5:06
  • Snap's core value was the fastest way to share a moment with your best friends. Peter Sellis, Snap and Discord1:13:36
• Define each metric so it cannot fool you. A loose definition inflates the number and hides the real result.
  • Flame believed it had a 15% conversion rate in its first month of monetizing, but the figure included renewals. Anup, Flame20:30
• Revisit your metrics as the product and market change. A metric that once made sense can start to block good decisions.
  • A seller-count gate would have blocked a Facebook Marketplace expansion in a region where buyers were finding items. Fiona Fung, Anthropic42:42

the metric you choose as the your main metric drives everything else.

Eugenia Kuyda, Wabi

2.1 Pick one main metric that matches user value

The main metric sets what the team works on, so choose it for the value users get.

• Match the main metric to how your product grows. Eugenia Kuyda, Wabi37:02
  • Wabi is multiplayer, so it watches retention and the K factor most closely.
  • People use apps with others, share them, and creators can bring their fan base.
  • After a decade on a single-player product that was hard to market, Eugenia sees built-in sharing as the growth strategy.
• Track the user's outcome, not time spent. Yves Benchimol, WeWard17:45
  • WeWard does not track time in the app, so it does not become a team target.
  • It tracks steps instead.
  • A user who does not improve their lifestyle within six months stops using the app. In Yves's words: "the success of the number of steps of people is the success of the company."
• Add a depth metric if some users adopt the core behavior far more than others. Weber Wong, FLORA37:28
  • FLORA uses nodes generated as its north star. It still watches weekly active users but weights them less.
  • Each week it also watches signups, conversion to paid and weekly churn.
  • Users who treat FLORA as a creative system have 100 to 200x more usage than those using it as a better Midjourney.
• If your goal is vague, build a metric for it. Eugenia Kuyda, Wabi16:19
  • Today, AI companion builders have to lean on clinical psychology questionnaires to measure wellbeing.
  • Separate long-term flourishing from short-term happiness.
  • "as soon as you have a metric and some observability around it then you can optimize that."

2.2 Define engagement and core value in plain words

Say what a returning user does, then turn each part into something you can count.

• Define engagement as intent to return. Tanuj Chatterjee, Super Unlimited5:06
  • Measure whether people come back, not how long they stay.
  • Remove login and heavy paywalls so the second visit is one tap.
  • For a utility, the goal is a quick connect, and then the user leaves for Netflix or the browser. In Tanuj's words: "It's not time spent on the app. It's actually the intention to come back to the app."
• Write the core value as a short phrase, then break it into metrics. Peter Sellis, Snap and Discord1:13:36
  • The phrase says why people come back every day.
  • Break each part of the phrase into measurable terms, and tie the metrics causally to retention.
  • Discord's phrase was talking and hanging out with friends before, during and after games.
• Use the definition to run the team. Peter Sellis, Snap and Discord1:13:36
  • Judge roadmap items against it.
  • Use it to onboard new hires and data scientists.
  • Peter gave the Snap core product value presentation to every new hire for seven years, and Discord's metric definitions went into its growth playbooks.
• Grow by lifting engagement of your existing core users. Peter Sellis, Snap and Discord50:50
  • Focus on people already using the product for what it is for, and make it faster, more relevant and easier.
  • Tie everything to daily use, because one daily user equals about 30 monthly users.
  • Moving DAU/MAU a few basis points can beat acquiring new MAU and hoping they retain.
  • After Snap's 2018 redesign flattened DAU, a laser focus on Android performance for existing users started a growth renaissance that ran for multiple years.

2.3 Define funnel and money metrics so they cannot mislead

Report each step and each revenue number in a way that shows what really changed.

• Report progression and drop-off together when testing onboarding. Virgin Mobile UAE
  • Track the share of users progressing to the next step, and the drop-off rate.
  • Together they show how much of the gain comes from more people moving forward versus fewer leaving.
  • In the Virgin Mobile UAE test, progression to registration rose 9% and drop-off decreased by nearly 18%.
• Calculate conversion to paid on new users only. Anup, Flame20:30
  • Measure the share of new users who convert within their cohort.
  • Keep renewals out of the numerator.
  • Anup's own check on the inflated figure: "It's not really conversion of the new users."
• Use one umbrella metric for monetization tests, such as 7-day ARPU. Michal Parizek, Mojo5:54
  • Mojo tracked 7-day revenue per user for its paywall and pricing experiments.
  • Pick a window long enough to cover your trial. Mojo had a 3-day trial.
  • Optimizing new revenue supports the user acquisition loop and allows higher ad spend, and most revenue arrives in the first days anyway.
  • RevenueCat also offered 7-day as an available ARPU view.
• Judge messaging and content changes by return on investment. Beckett Simonon
  • Report the sales lift and the annualized return together, so you see whether the effort paid off against its cost.
  • A 5% sales increase from brand storytelling translated to a 237% annualized return on investment.

2.4 Read acquisition signals with care

Organic and creator traffic is hard to attribute, so use overlays and simple benchmarks.

• Overlay posting spikes on installs and revenue to find which videos convert. Canyon, Sideshift38:42
  • Lay your posting spikes against App Store installs and Stripe or RevenueCat revenue.
  • Check creator audience geography to detect view botting.
  • At scale you only feel the million-view bombs. In Canyon's words: "it's always the attribution piece that is so foggy, that is so gray."
• Use profile-to-link clickthrough as a benchmark, with 15% as the target. Nick and Ivan, Superwall48:01
  • Divide link clicks by profile activity. In the example, about 5,000 profile activities produced about 750 clicks.
  • Bad campaigns land near 3%, and engaging content with a well set up profile reached 50 to 60%.
  • One campaign got about 7,000 clicks from about 300,000 comments and projects a 4x return.

2.5 Check quality and revisit metrics as the product changes

Sort problems by how much they hurt, and keep asking whether each metric still serves the outcome you want.

• Classify quality problems as bad or sad on each surface. Fiona Fung, Anthropic45:24
  • Bad is an irrecoverable error, such as a crash that loses work. Sad is a recoverable pain point, such as flickering.
  • Stacked sads can turn into bad.
  • Let each team define bad and sad for its surface and set its own goal. This makes it easier to judge whether a raw number is good across many surfaces.
  • On the CLI, crash rate is a bad metric and flickering is a sad one.
• Recheck every metric against the outcome it was meant to serve. Fiona Fung, Anthropic42:42
  • Early Facebook Marketplace region launches were gated partly on number of sellers.
  • The first region had few sellers but power sellers, so the team updated the metric to account for them.
  • Fiona's warning: "make sure that you're not just having blinders on that's blindly following a metric that used to make sense"

2.6 Where they disagree

The right main metric and the right amount of usage depend on the product.

• Revenue, retention or a user outcome can each be the main metric.
  • Replika's revenue focus pushed out other work, and Wabi now focuses on retention and K factor instead. Eugenia Kuyda, Wabi47:51
  • Mojo uses 7-day ARPU, but as the umbrella for monetization tests only. Michal Parizek, Mojo5:54
  • WeWard picks steps, an outcome for the user, as the measure of company success. Yves Benchimol, WeWard17:45
• Heavy usage is a good sign for some products and not for others.
  • For a utility, a short session is the goal, and returning is what counts. Tanuj Chatterjee, Super Unlimited5:06
  • FLORA treats heavy usage as the sign of real adoption, and uses nodes generated as its north star. Weber Wong, FLORA37:28