How to learn from users and customers

1 The bottom line

Combine interviews, surveys, emails, usage data and your own use of the product, and treat each as a hint to be checked by the others. Every source misleads in its own way, so no single one should decide what you build.

2 Key takeaways

• Spend real time with users before you build, and keep doing it. Hearing the problem in their words beats guessing at it.
  • Walking every pontoon on the south coast with a clipboard turned a niche racing-yacht idea into a market of all boaters, and new needs still turn up Jelte Liebrand, Savvy Navvy6:15.
• Watch what users do, because it often contradicts what you say about them. Behavior is data nobody has to remember or politely soften.
  • Grammarly's site was student-only, yet about 50% of the content checked was non-academic, which drove the pivot to knowledge workers Yuriy Timen, Grammarly.
• Read requests for the pain underneath, not as orders. Users describe problems well and solutions badly.
  • Users asked for years to talk to a human tutor, but nobody used the tier Duolingo built Cem Kansu, Duolingo1:02:43.
• Dogfood to move fast, but remember your team is not your user. Builders are usually far ahead of, or unlike, the people they serve.
  • OpenAI's internal signal favored a cloud product, while the wider market adopted the local IDE and CLI first Alexander Embiricos, OpenAI20:44.
  • The onX lead signed up in 2014 and found today's new users are completely different Brandon Gador, onX29:55.
• Let sources check each other, and doubt any neat explanation. People are polite, and context changes what feedback means.
  • A deal said to be lost on price may be lost on weak differentiation Bret Taylor, Sierra20:35.
  • Early monday.com offered no free trial, so feedback came from paying users, a proxy for real value Daniel Lereya, monday.com58:20.

We didn't build exactly what they asked for. We empathized and then came up with something new.

Evan Spiegel, Snap

2.1 Learn the problem in person before you build

Spend time with potential customers until you can feel their problem, not just list it.

• Do the customer's job yourself, or shadow someone who does. Advice from Rob Snyder18:50.
  • Skip whiteboard brainstorms and ten 30-minute Zoom calls, and ride along or embed until you feel where they are stuck Rob Snyder18:50.
  • Rob estimates about 10% of the ideas he has seen came from the founder's own experience and 90% from firsthand work Rob Snyder18:50.
  • A friend shadowed solar installers, a bad fit, then watched independent physical therapists over their shoulders for days, and found the pull there Rob Snyder18:50.
• Talk to hundreds of people when the problem is still unclear.
  • Sam spoke to four or five hundred teachers by emailing lists, attending meetups and teaching summer school in exchange for feedback Sam Chaudhary, ClassDojo12:12.
  • StudyFetch interviewed 120 to 150 college students before launch Esan Durrani, StudyFetch7:15.
  • The top teacher problem, classroom management, was non-obvious and unsexy, so the first product became a positive-feedback points tool Sam Chaudhary, ClassDojo12:12.
  • The students showed the gap was not only saving time but understanding material presented poorly, which supported a tutor that teaches rather than answers Esan Durrani, StudyFetch7:15.
• Ask about their problems, and about why people do not use you.
  • Early Behance focus groups said they did not need another social network, but asking what they suffered from revealed missing attribution, out-of-date portfolios and no reach to hirers Scott Belsky, Adobe13:08.
  • Talk to churned and never-adopted users, a question most teams never ask Marty Cagan10:25.
  • Cagan churns from many products he tries, and almost nobody follows up to ask why Marty Cagan10:25.
• Be in the room, and listen to emotion as well as tasks.
  • No phones, hear the words and intonation, and do not swap interviews for an AI summary, because you can't empathize with a summary Peter Deng1:32:33.
  • Chrome's team watched users navigate their tabs and heard frustration and apologies, which led to moving tabs untouched for 21 days into a folder instead of closing them Nesrine Changuel45:21.
  • Peter leased a car to drive for Uber for two weeks before joining Peter Deng1:32:33.

2.2 Test early versions with real users, without leading them

Watch people use a rough version, with as little help from you as possible.

• Recruit strangers, not friends, and let them struggle.
  • Recruit good prospective users with zero skin in the game through panels like Voice Panel or UserTesting, give little context and have them think aloud Grant Lee, Gamma1:05:35.
  • About 20 people run each prototype, and Gamma can go from idea to results in a day or so Grant Lee, Gamma1:05:35.
• Also let users try the product alone.
  • Behavior with Speak's team in the room differed greatly from behavior when users tried it alone Connor Zwick, Speak12:56.
  • Alone, people had no fear of judgment and spoke freely, which exposed the core benefit of speaking with an AI rather than a human Connor Zwick, Speak12:56.
• Onboard your first users by hand and fix every failure by the next call.
  • The core team joined onboarding calls for about two weeks and avoided leading users, so natural usage patterns showed up Roman Ugarte, Grok Bot12:15.
  • Pick a few people who owe you a favor, watch them, and read the logs when it breaks Maor Shlomo, Base4449:23.
  • Roughly 200 to 300 people were onboarded manually over about two weeks Roman Ugarte, Grok Bot12:15.
  • Even at 20K to 100K users, Maor ran focus groups or small hackathons of 20 to 30 people every other week Maor Shlomo, Base4449:23.
• Run a fixed-format sprint when you need answers fast.
  • Show five bullseye customers the same three simple prototypes in five one-hour interviews, clumped into one day Michael Margolis, GV13:32.
  • Score each part of the hypothesis red, yellow or green: right customer, real problem, right approach, chosen over competition, differentiation Jake Knapp and John Zeratsky, Character Capital1:20:50.
  • After four or five interviews, teams say they hear the same things repeatedly, across over 300 sprints Michael Margolis, GV13:32.
  • Latchet's scorecard went from lots of red in week one to every box green by week three Jake Knapp and John Zeratsky, Character Capital1:20:50.

2.3 Use surveys for narrow questions

Surveys work when the question is specific and the people asked are the right ones.

• Ask about one concrete behavior users already have.
  • Ask how often people do something that already exists elsewhere, and do not accept an anecdote that a competitor tried it and failed Francesco Polizzi, Discord21:48.
  • A quick survey of some Discord users found 70 to 80% used voice messages weekly, and the shipped feature raised overall communication by a single-digit percent Francesco Polizzi, Discord21:48.
  • Surveys fail when the question is vague Francesco Polizzi, Discord21:48.
• Put a few questions inside onboarding to catch people in the right mindset.
  • Noom adds screens or questions to onboarding, alongside surveys and usability tests, because survey targeting may reach the wrong people in the wrong mindset Brent Zajaczkowski, Noom19:08.
• Ask users to rank candidate features.
  • Flo offered about 20 possible features in user research, and the partner feature came out most desirable Dmitry Gurski, Flo44:10.
  • Partner mode launched in October 2023, and Flo expects about 10% of users, around 10 million, to be connected to partners Dmitry Gurski, Flo44:10.
• Survey through in-app chat when you have it.
  • Ladder's chat gives direct user relationships, so it surveys often Greg Stewart, Ladder40:40.
  • One platform-wide survey got 5,000 replies, with an average response time of 45 minutes Greg Stewart, Ladder40:40.

2.4 Keep always-on channels open and read them yourself

Emails, in-product forms, calls, reviews and complaints give a steady stream of signal at low cost.

• Ask new customers how it went, by email and inside the product.
  • Jimmy emailed every customer who placed a first DoorDash order in San Francisco, and later did the same for first grocery orders Jimmy Liu, DoorDash4:56.
  • Add a feedback button with a short inline form and four emojis, and route every response into Slack Guillermo Rauch, Vercel1:01:35.
  • The DoorDash replies surfaced themes of selection, delivery quality and affordability, and seeded DashPass Jimmy Liu, DoorDash4:56.
  • Early Vercel got tens or hundreds of entries a day, sometimes just a couple Guillermo Rauch, Vercel1:01:35.
• Call users yourself, and let them opt in.
  • On the third app open, Brawler shows a pop-up offering a call with a founder, picks random numbers and sets aside about an hour a week George, Brawler42:36.
  • Ten to fifteen calls with about 2,500 active subscriptions revealed features that took weeks to build that users never knew about George, Brawler42:36.
  • A niche app can offer a reward its audience values, such as a Rubik's cube or a $30 gift card, then offer a call Steve P. Young, Migraine Buddy8:26.
• Mine reviews, DMs and public posts for repeating themes.
  • Fix friction in your own reviews, then read competitor reviews for problems you can solve better Tanuj Chatterjee, Super Unlimited3:31.
  • Read your five-star reviews for the theme that shows what users pay for Ben Gammon, Ladder2:03.
  • Ask publicly for the worst parts of your product, and re-ask later Tyler Denk, beehiiv13:54.
  • The review-driven approach took the app to number one for the VPN keyword in 67 of 71 tracked iOS countries Tanuj Chatterjee, Super Unlimited3:31.
  • Ladder's five-star reviews pointed to getting results, which became a consistency target of workouts per week Ben Gammon, Ladder2:03.
  • One post asking for the worst part of beehiiv drew 250 critical comments Tyler Denk, beehiiv13:54.
  • Repeated postpartum core rehab questions in comments, DMs and Facebook groups became a program with over a million users, and would never have shown up in Shopify reports Nancy Anderson, Natal32:12.
• Turn complaints into test ideas, and count complaints as a second success metric.
  • Blinkist fixed confusion about how its trial and billing worked Blinkist.
  • Chick-fil-A tested new layouts and CTAs after customers wrongly believed they could only pay with the loyalty card Chick-fil-A.
  • Blinkist's write-up credits the fix with a 23% conversion increase and 55% fewer complaints Blinkist.
  • The payment complaints at Chick-fil-A fell to 0 Chick-fil-A.

2.5 Dogfood, and treat it as one signal

Using your own product daily catches problems dashboards miss, as long as you add outside eyes.

• Use the product every day and ship only what is useful to you.
  • Cursor's team used the tool daily and refused to ship anything not useful to them, which kept humans in the driver's seat Michael Truell, Cursor20:04.
  • Do your own work in the product even when it is worse, so it becomes the best tool Andrew Ambrosino, OpenAI23:05.
  • Cursor's first prototype took about five weeks until the team lived in it full time, and it launched about three months after the first line of code Michael Truell, Cursor20:04.
• Check retention and delight internally before a wide release.
  • Ask whether it retains, whether people find something delightful, and whether it unlocks a new use of the models. Weak internal uptake is a signal not to ship Tara Seshan and Nan Yu, OpenAI3:20.
  • Widen the circle from internal use to beta groups to general availability, and read no engagement as a miss Nan Yu, Linear14:04.
• When you cannot use the product the way customers do, go to them.
  • Do not get lost in metrics, dashboards and presentations Fiona Fung, Anthropic1:04:20.
  • A trip to Chile with a few Android phones showed Marketplace's feed loaded poorly on slow LTE, a blocker no metric had explained Fiona Fung, Anthropic1:04:20.

2.6 Check what people say against what they do

Use behavior data to confirm, correct or explain what interviews and requests told you.

• Look for uses you did not plan, and build for them.
  • When people hack your product to do something it was not designed for, build a special-purpose product for that use Boris Cherny, Claude Code47:25.
  • Opal noticed many people using it for sleep without a sleep feature, and quietly launched a sleep tab Kenneth Schlenker, Opal31:40.
  • 40% of Facebook group posts were buying and selling, which led to Marketplace Boris Cherny, Claude Code47:25.
  • After people used Claude Code for tomato plants, genome analysis and wedding photos, Cowork was built in about 10 days Boris Cherny, Claude Code47:25.
• Test your assumptions about when and why users act.
  • Chess.com built game review expecting use after losses, but data showed otherwise, so it flipped the loss experience to show brilliant moves and an encouraging coach message Albert Cheng, Chess.com11:41.
  • 80% of reviews happen after a win, and the flip grew reviews 25% and subscriptions 20% Albert Cheng, Chess.com11:41.
• When a feature flops, segment the data until you find the root cause.
  • Split by region, device, demographic and use case until you see where the break lives Robby Stein, Google54:00.
  • Close Friends flopped because the list name was mistranslated as "best friend" in many markets, so people added only one person. Redesigning around larger lists made it one of Stories' most popular features Robby Stein, Google54:00.
• Read requests for the pain behind them.
  • Snap skipped surveys and talked for one or two hours about how technology fits a life Evan Spiegel, Snap26:25.
  • Stories combined requests for a send-all button, less social pressure and confusion over reverse chronological feeds. It added no public likes or comments, a 24-hour expiry and chronological order Evan Spiegel, Snap26:25.
• Use AI to find themes, then read the details yourself.
  • Ask for themes, then have the model list every specific detail under each theme with the customer who said it. Cancellation volume is often low enough to just read Jason Cohen, WP Engine30:32.
  • Paste interviews and your strategy into a model and ask where the strategy does not match what customers Shaun Clowes, Confluent15:39.

2.7 Where they disagree

Some operators distrust customer talk or favor their own taste, and the right mix depends on who your customer is.

• Interviews may suit enterprise only.
  • Purchase decisions are subconscious, so use customer development only for enterprise, such as 30 must-win accounts, and pressure test your own insight for mass products Keith Rabois, Khosla Ventures52:10.
  • Ask a Porsche driver why they bought it and 99% of the time you get every reason except the real one, and Airbnb would have gotten a no from 9 of 10 random people Keith Rabois, Khosla Ventures52:10.
  • Consumer builders above still got their core insight from talking to users, so the safer use is to find pain, not to collect verdicts on your idea Sam Chaudhary, ClassDojo12:12.
• Building for yourself gives conviction, but also blind spots.
  • Fitbod's founders built the planner they wanted and reached product market fit quickly, and used their own gym use to decide which requests to ignore Allen Chen, Fitbod5:11.
  • Intercom's founder built for himself and felt people would want it without needing customer feedback Eoghan McCabe, Intercom1:20:48.
  • Gamma's Jon read feedback mostly from product managers like himself and missed consultants already using it for client pitches. He advises knowing your top three to five segments Jon Noronha, Gamma13:53.
• How much feedback to act on is contested.
  • FLORA adds quality of life features only when it hears the same need multiple times in one week, and says core innovation comes from first-principles thinking Weber Wong, FLORA35:55.
  • Focus on somewhat-disappointed users whose favorite benefit matches your core benefit, and deliberately not act on the rest Rahul Vohra, Superhuman53:54.
  • Users cannot ask for capabilities they do not know are possible, and Bolt's native mobile app support, ignored in requests, was its best-received launch Eric Simons, Bolt41:15.