CaddieOpen in CaddieDownload for Mac

Spot a surprising result worth chasing

Shared by Caddie Library · Updated
Chapters show their main points
Chapter 27 of 29 · How a field works and shifts
← The growth course
1. Finding Product-Market Fit
Test for real customer pull earlyPick a narrow niche to start inLaunch a simple first version fastLearn from users and customersRead early signs of product-market fit
2. Onboarding and Activation
Structure onboarding as a storyAsk and use onboarding questionsGet users to the aha moment fastDecide how long onboarding should beBuild quiz funnels for paid traffic
3. Paywalls and Pricing
When to choose a hard paywall over freemiumDesign a paywall that convertsPrice annual and monthly plansSet, raise and localize pricesRun free trials and free tastes
4. Business Models and Revenue
Structure subscription tiersKeep a free tier that pays offCharge for usage and AI creditsPair hardware with a subscriptionAdd ads, affiliates and partners
5. Retention and Engagement
Measure retention honestlyBuild habit loops that bring users backUse notifications and lifecycle messagesReduce churn and handle cancellationsBuild social loops and community
6. Brand and Positioning
Position against bigger incumbentsName and describe your productEarn trust in high-stakes categoriesBorrow credibility from celebrities and experts
7. Organic Content and Creators
Find, vet and recruit creatorsPay creators fairly and safelyBrief and manage creatorsFind and copy viral formatsScale one format across many accountsRank in AI search and SEO
8. Paid, App Store and Viral Growth
Make ads that convertTurn organic winners into paid adsStructure and scale Meta campaignsTell if paid acquisition is profitableWin in the App StoreBuild viral and referral loops
9. Experimentation and Analytics
Run trustworthy experimentsJudge test results correctlyRun more experiments fasterWhat metrics to track and how to define them
10. Building AI Products
Build for the models of tomorrowDesign AI interfaces beyond chatDesign AI agents people can trustMeasure AI quality with evalsControl AI costs and pricingBuild products with AI coding tools
11. Teams, Hiring and Funding
Hire people who thrive on ambiguityOrganize small product teamsRun exploratory bets and kill themWork as a team with AI agentsWhen to raise money or sell your company
12. Founder Principles
Pick a mission worth building a company aroundWhy to trust your own judgment over the consensusTurn failure and setbacks into the next stepLearn from rivals and copy what works
13. Choosing Where to Compete
Make low cost your competitive advantageCreate a market instead of fighting for shareWhy scarcity and brand can beat volumeBuild an advantage that feeds itselfAttack a bigger rival where it cannot followWhen to stay focused on the core business
14. Building and Leading the Company
Hire great people and give them roomAlign pay and ownership with the missionKeep your culture and standards as you growBuild candor and fast decision-makingWhy leaders must stay close to the work and customersChoose and prepare a successor
15. Capital and Deals
Fund growth without losing controlAcquire companies and keep what makes them workWhat to do with cash: reinvest, buy back or holdWhy to keep a cash cushion and avoid heavy debtStructure ownership to protect your missionNegotiate deals that keep your leverage
16. Finding Ideas and Thinking
How to find ideas worth writing aboutWhy writing is how you find out what you thinkHow to research so the writing carries feelingHow to read like a writer
17. Structure and Argument
How to open so readers keep goingHow to find the shape before you draftHow to keep readers curious with questions and cluesHow to argue without preachingHow to end so the last page lingers
18. Clear Sentences and Voice
How to write plainly for a reader who knows nothingHow to choose specific details over generic onesHow to sound like yourself on the pageHow to shape rhythm and phrases people remember
19. Characters and Storytelling
How to build characters readers care aboutWhat makes a scene move: want, obstacle and changeHow to write dialogue that sounds real but stays shortHow to make readers feel without telling themHow to turn information into a story
20. Revising, Habits and Readers
How to cut and rewrite your own draftHow to get feedback you can useHow to build a writing routine that lastsHow to get unstuck when a draft stallsHow to use AI without losing your voiceHow to write for one specific reader
21. Why people play games
How to design around the feeling you wantHow to find a game idea with a hookHow to decide who your game is forWhy rewards and motivation need careful designHow to use mystery and curiosity to pull players on
22. Core mechanics and loops
How to build a game around one core verbHow to give each mechanic several jobsHow to add depth with holds, timing and chainsHow to use feedback loops to keep games balancedHow to use randomness without frustrating playersHow to make synergies feel powerful and earned
23. Challenge, difficulty and feel
How to make hard games feel fairHow to tune difficulty with many leversHow to offer assists without breaking your intentHow to make death and retries painlessHow to make movement and controls feel goodHow to add camera and juice that sell impact
24. Teaching players and designing levels
How to teach mechanics without a tutorialHow to introduce mechanics one at a timeHow to guide players without markersHow to build levels around one mechanicHow to design puzzles with a catch and a revelationHow to design dungeons from locks and keys
25. Progression, systems and story
How to gate exploration without trapping playersHow to design unlocks and skill treesHow to let systems create emergent playHow to design enemy AI players can readHow to tell stories through mechanicsHow to make player choices and consequences matter
26. Prototyping, testing and accessibility
How to prototype before you commitHow to scope and cut a game to finish itHow to run playtests that reveal real problemsHow to turn player feedback into the right fixHow to build accessibility in from the start
27. How a field works and shifts
Make steady progress inside a working frameworkSpot a surprising result worth chasingWhen to abandon a model that mostly worksWhat your tests and metrics quietly assumeWin agreement for a new approach
28. Organising a place for discovery
Build a team where disciplines collideFund open-ended research and keep it usefulMove a promising result into real useLearn from missed chances and outside threats
29. Discovering together in a small group
Share work, credit and priority in a groupFind the cause when results behave erraticallyGet a new result noticed and adopted

1 The bottom line

Write down your expectations and doubts, with dates, before you run anything. Surprises only show up against a clear prediction, and only the ones that persist or hit core assumptions deserve escalation.

2 Key takeaways

  • Write a precise expectation before every experiment. A surprise only stands out against a clear prediction.
    • Roentgen saw a screen glow and spent seven weeks testing it Thomas S. Kuhn.
  • Date your doubts about your current model before you have evidence. Prior suspicion prepares you to see what others miss.
    • In 1772 Lavoisier deposited a sealed note with the French Academy saying phlogiston theory was flawed Thomas S. Kuhn.
  • Log most anomalies and keep working, and escalate only some. Most discrepancies yield to normal practice.
    • Newtonian predictions long disagreed with the speed of sound and Mercury's motion Thomas S. Kuhn.
  • Run a hard test before you tell leadership. A false alarm upward is the worst outcome.
    • Ralph Bown had Bardeen and Brattain make their amplifier oscillate, and Mervin Kelly was not told for about a month Jon Gertner, Bell Labs.
  • Look at odd signals again and again before filing them in old categories. Familiar categories absorb new things without anyone noticing.
    • Astronomers recorded a star where Uranus must have been on at least seventeen occasions between 1690 and 1781 Thomas S. Kuhn.

Anomaly appears only against the background provided by the paradigm.

Thomas S. Kuhn, The Structure of Scientific Revolutions

2.1 Write down what you expect

A result can only surprise you if you have said in advance what you thought would happen.

  • Record the expected range before each experiment. This is the first step in Thomas S. Kuhn's account of how anomalies get noticed.
    • Write the range down first and date it.
    • Treat results outside the range as the start of exploration, not noise.
    • Roentgen noticed the glowing screen during a normal cathode ray investigation. Another investigator had seen the glow and discovered nothing.
  • Make your model as precise and far reaching as you can. A sharper model is a better detector.
    • The more precise and far reaching the model, the more sensitive it is as a detector of anomaly Thomas S. Kuhn.

2.2 Keep a dated record of doubts

Writing down what you suspect is wrong, before the evidence arrives, makes you faster at recognizing it.

  • Write each doubt down with a date before you have proof. It gives shape to later findings.
    • Prior suspicion that something is wrong prepares you to see what others miss when the anomaly arrives Thomas S. Kuhn.
    • Lavoisier's 1772 sealed note let him see in Priestley's experiments what Priestley could not Thomas S. Kuhn.
  • Reread the doubt log when a strange result comes in. It helps you recognize the finding fast.
    • Match the new result against the dated doubts to see whether it fits one of them Thomas S. Kuhn.
    • Kuhn's line: "Discovery commences with the awareness of anomaly" Thomas S. Kuhn.

2.3 Decide which odd results deserve attention

Log every anomaly, but escalate only the few
Kuhn: there is no research without counterinstances, so the filter matters more than the alarm.

Log every discrepancy, keep working, and escalate only when the anomaly is serious or will not go away.

  • Log discrepancies and keep working. Most yield to normal practice.
    • Do not stop the main work for a single odd result Thomas S. Kuhn.
    • The speed of sound gap was later closed by heat experiments done for other purposes Thomas S. Kuhn.
    • Mercury's gap vanished with general relativity Thomas S. Kuhn.
  • Escalate when the anomaly meets one or more of four tests. Usually several combine before a real crisis starts.
    • It hits a fundamental generalization.
    • It blocks an important use.
    • New measurement tools give it more weight.
    • It resists the best people for a long time Thomas S. Kuhn.
    • Kuhn's reminder: "There is no such thing as research without counterinstances" Thomas S. Kuhn.

2.4 Look again, and expect your own resistance

Old categories swallow new things quietly, so hold your labels loosely and keep watching.

  • Hold back on labeling an observation that looks slightly off. Watch it over time.
    • Familiar categories absorb new things without anyone noticing Thomas S. Kuhn.
    • A small shift in categories can prepare a team for a run of further finds.
    • After the shift on Uranus, astronomers identified twenty minor planets with standard instruments in the first fifty years of the nineteenth century Thomas S. Kuhn.
  • Expose people to the anomaly repeatedly, and expect resistance, including your own. Categories shift with exposure, but not for everyone.
    • People first force an anomaly into an existing category without noticing. With more exposure they hesitate, grow confused, then often switch suddenly Thomas S. Kuhn.
    • In the Bruner and Postman card experiment, even at forty times the exposure needed for normal cards, more than 10 per cent of anomalous cards went unidentified Thomas S. Kuhn.

2.5 Test hard before you escalate

Pass a hard test before news moves up
Bell Labs moved the transistor news up one convinced supervisor at a time.

Agree a demanding test first, pass it, then move the news up one level at a time.

  • Agree on a demanding test that proves the result is real, and run it before telling leadership. The test should be one the result could easily fail.
  • Move the news up one level at a time, once each supervisor is convinced. A false alarm upward is the worst outcome.

2.6 Where the books differ

Match the method to the kind of anomaly
Kuhn: watch over timeGertner: test once, hard
Main riskIgnoring the anomalyRaising it too early
Works overYears of persistenceA single decisive test
Use forLong-standing gapFresh, testable result
ActionWait for persistence and weightDesign the hard test, run it first
ExampleMercury's orbit, speed of soundAmplifier made to oscillate
Long-standing gaps call for patience, and fresh testable results call for a hard test first.

Kuhn warns against ignoring anomalies, while Gertner's Bell Labs story warns against raising them too early.

  • Kuhn's filter works over time, and Gertner's gate works on a single test. Pick the one that fits the kind of anomaly you have.
    • For a long standing gap, wait and watch for the signs of persistence and weight Thomas S. Kuhn.
    • For a fresh result you can test directly, design the hard test and run it first Jon Gertner, Bell Labs.
  • Guard against both errors: a false alarm and your own resistance. The sources stress different risks.
    • Gertner's account calls a false alarm upward the worst outcome Jon Gertner, Bell Labs.
    • Kuhn says some people never adjust to an anomaly, so expect resistance, including your own Thomas S. Kuhn.

Sources

Part of How a field works and shifts in the growth library.