All people

Guest · Flo

Mike Charniatsou

Senior growth product manager at Flo, formerly first product hire and CPO at Luna

1 source18 tips

Where to hear and read about Mike Charniatsou1 episode or test

Sub ClubWhat 80M users taught Flo about A/B testing — Mike Charniatsou
In short

Mike Charniatsou is a senior growth product manager at Flo, which has nearly 80 million monthly users, and was the first product hire and later CPO at Luna. The thing most worth learning from him is to judge a growth win by what it does to the business over the full subscription lifecycle, and to follow evidence rather than explanations when a test fails.

Explanation usually tells you what you want to be true.

What 80M users taught Flo about A/B testing — Mike Charniatsou

Mike Charniatsou: the story and the lessons

1Mike Charniatsou's path from Luna to Flo

  • Charniatsou was Luna's first product hire and grew into its CPO. He joined Luna as its first product hire and was promoted to CPO as the team grew. Luna went from nothing to almost $6M ARR and won an App Store design award.

    The idea was to turn sleep stories, which were very popular then, into a more interactive experience. He is proud of that story as a growth leader.

  • He left a CPO job to build hands-on again at Flo. He wanted to build things as an individual contributor while AI was growing, and Flo offered a scale Luna could not. A CPO role is mostly managing people and setting strategy.

    Flo's web division looked like Luna a year earlier, so he knew what to do. He called the move a "no-brainer".

  • He worked out web onboarding from first principles at both companies. No bigger sleep or meditation app existed to copy at Luna, and Flo is the biggest in its niche. The open questions were the goal of the onboarding, its length and its friction points.

    Web funnels are pitched as a way around App Store and Google Play commissions, but "nobody told us how expensive it would be to build this thing". No single platform existed, so the team made many mistakes, and RevenueCat then had no web funnels or web payments.

  • He runs growth differently at Flo's scale than at an early-stage startup. A startup works in broad strokes, such as how the onboarding journey should work. At Flo a proven product sits behind every feature, so growth tunes the details of specific parts of the funnel.

    A large ad budget lets marketing test many hypotheses at scale and pass validated insight to product. At Luna he had only market insights, gut feeling, educated guesses and investor tips, which he calls a completely different paradigm.

2How Charniatsou separates paywall wins from business wins

  • He treats a subscription as a lifecycle, not a paywall conversion event. Trial start, activation and retention are steps in one journey, so Flo builds monetization around the whole user lifecycle. A paywall conversion uplift does not guarantee the business grows.

    Flo has seen conversion wins that did not become business wins once it kept analyzing experiments after rollout. In his words, the onboarding paywall "is just the beginning of a whole journey for the user."

  • He re-checks every rolled-out win at 3, 6 and 12 months. Flo goes back after rollout to see whether a reported win still holds in retention and revenue.

    Teams are pushed to announce wins early, for LinkedIn and for management, so only the later look shows which wins were real.

  • He asks whether a year of reported uplifts compounded into growth. A team may ship tens of experiments with uplifts of 5, 7 or 10%. At year end Flo asks whether the business grew by the same percentage.

    The gap shows how many wins were short-term conversion bumps. Relative uplifts mean little when you zoom out to the health of the business.

  • He checks refunds before trusting a hard paywall win. BoldVoice removed the X button to make a hard paywall and conversion jumped. Then refunds spiked massively and wiped out the gain.

    A paywall variant can win on conversion and still lose money later in the funnel.

  • Flo waits for revenue because 40% of its subscriptions come after six months. Flo does not try to monetize on day one or day seven, and basics like cycle tracking stay free. Users' needs and willingness to pay shift over a lifetime.

    He calls this a luxury, not advice for apps that need growth from day zero or carry high inference costs. "We don't feel the need to try and monetize the user on day one and day seven."

  • He times paid offers to a new life stage. Flo is moving from period tracker to guide through stages such as understanding periods, trying to conceive and following pregnancy. It offers something tailored when a new need appears.

    This converts free users 3, 6 or 12 months later, and Flo avoids intrusive paywalls that would spoil the relationship.

  • He judges free trial tests early with proxy events. Trials delay the conversion event, so Flo uses an early event that correlates with future user value. A good proxy must predict what happens 30 or 40 days later.

    Early startups often lack the historical data to build one. "In proxies we trust."

3How Charniatsou learns from failed experiments

  • He follows evidence after a failed test, not a new explanation. The first instinct is to blame copy, slow execution or heavy design. He searches for what users actually experienced and only reruns when the evidence supports it.

    Several competing explanations start a vicious cycle of experimenting. "The evidence tells you where to look."

  • He runs a short postmortem before any retry. Flo checks that treatment users saw the change as planned, then finds where the biggest drop or uplift sat and whether a segment drove it.

    Flo fails often because its funnels are well optimized and its win rate is slowing.

  • He turned a perimenopause step from a loss of about 10% into a 23% win. The step was meant to name the user's perimenopause phase. The first test lost about 10%, and a second run with a design tweak lost again.

    Dissecting the audience showed about 75% of users were assigned no phase and got an "Oops, sorry" screen, which clashed with an ad promising to explain their phase. The medical team rewrote the question, users with no phase skipped to the next section, and the third version won 23%. He names it his biggest win of the year, an unexpected blockbuster.

  • He runs blackout tests to confirm wins and trim funnels. Flo removed the perimenopause step in a test and it lost, which confirmed the 23% uplift was real.

    Funnels keep gaining steps, so removing one is sometimes the better move. He cites a line he heard: the biggest app feature is less features.

  • He learned that a popular feature is not automatically a good onboarding step. Partner Mode is one of Flo's most requested features, used by more than 10 million pairs. Putting it in onboarding did not help conversion and was his biggest fail of the year.

    He says there were multiple reasons without detailing them. His lesson is that popularity and onboarding fit are separate questions.

4How Charniatsou works with marketing and borrows ideas

  • He built an onboarding branch around an ad narrative that overperformed. Ads about enhancing your sex life did well on Meta even with the old funnel. Product made it its own user goal with tailored questions so the funnel acknowledged the ad.

    Marketing validates what resonates much faster than a product funnel can. The branch became a great revenue driver for Flo.

  • He shares every ad and funnel change in weekly syncs. Marketing reports each ad campaign change and product reports its own, even ones that seem minor. Marketing tends to blame the funnel and product the audience.

    He says it is not a happy marriage. "You never know what insights will be valued by a different team."

  • He borrows the idea behind another team's win, not the solution. Flo copied pixel for pixel a paywall change another team reported as a 10% win, and it did nothing. He had no reasonable explanation and would not build strategy on it.

    Many teams borrow the solution before understanding the problem, and Flo and Luna both did. Success elsewhere is not proof it works in your context.

  • He found the Blinkist reminder paywall failed at both Luna and Flo. It looks like a bulletproof pattern, yet it did not work at Flo at all. The trial toggle also worked for some teams and not for Flo.

    The lasting idea is making people comfortable that they will be reminded to cancel, so teams where it failed iterated on how to make people comfortable starting a free trial.

  • He rejects streaks as a fix for missing intrinsic motivation. Teams add streaks to everything, from daily logins to meditation audio. He calls it a cargo cult accidentally created by Duolingo's success.

    His test: "does forcing this engagement contribute to the user value or it contributes to boosting your product metrics?" Some apps have intrinsic motivation and others do not.

More from Flo

Guest

Dmitry Gurski

Cofounder and CEO of Flo, interviewed by Phil Carter

Flo
1 source · 15 tips
Guest

Ryan Thorpe

Founder of Flo, former owner of a 30-app mobile portfolio (Reflectly, Do Habit)

Flo
1 source · 17 tips