Chapter 27 of 29 · How a field works and shifts
1 The bottom line
Write down your expectations and doubts, with dates, before you run anything. Surprises only show up against a clear prediction, and only the ones that persist or hit core assumptions deserve escalation.
2 Key takeaways
- Write a precise expectation before every experiment. A surprise only stands out against a clear prediction.
- Roentgen saw a screen glow and spent seven weeks testing it Thomas S. Kuhn.
- Date your doubts about your current model before you have evidence. Prior suspicion prepares you to see what others miss.
- In 1772 Lavoisier deposited a sealed note with the French Academy saying phlogiston theory was flawed Thomas S. Kuhn.
- Log most anomalies and keep working, and escalate only some. Most discrepancies yield to normal practice.
- Newtonian predictions long disagreed with the speed of sound and Mercury's motion Thomas S. Kuhn.
- Run a hard test before you tell leadership. A false alarm upward is the worst outcome.
- Ralph Bown had Bardeen and Brattain make their amplifier oscillate, and Mervin Kelly was not told for about a month
Jon Gertner, Bell Labs.
- Ralph Bown had Bardeen and Brattain make their amplifier oscillate, and Mervin Kelly was not told for about a month
- Look at odd signals again and again before filing them in old categories. Familiar categories absorb new things without anyone noticing.
- Astronomers recorded a star where Uranus must have been on at least seventeen occasions between 1690 and 1781 Thomas S. Kuhn.
Anomaly appears only against the background provided by the paradigm.
Thomas S. Kuhn, The Structure of Scientific Revolutions
2.1 Write down what you expect
A result can only surprise you if you have said in advance what you thought would happen.
- Record the expected range before each experiment. This is the first step in Thomas S. Kuhn's account of how anomalies get noticed.
- Write the range down first and date it.
- Treat results outside the range as the start of exploration, not noise.
- Roentgen noticed the glowing screen during a normal cathode ray investigation. Another investigator had seen the glow and discovered nothing.
- Make your model as precise and far reaching as you can. A sharper model is a better detector.
- The more precise and far reaching the model, the more sensitive it is as a detector of anomaly Thomas S. Kuhn.
2.2 Keep a dated record of doubts
Writing down what you suspect is wrong, before the evidence arrives, makes you faster at recognizing it.
- Write each doubt down with a date before you have proof. It gives shape to later findings.
- Prior suspicion that something is wrong prepares you to see what others miss when the anomaly arrives Thomas S. Kuhn.
- Lavoisier's 1772 sealed note let him see in Priestley's experiments what Priestley could not Thomas S. Kuhn.
- Reread the doubt log when a strange result comes in. It helps you recognize the finding fast.
- Match the new result against the dated doubts to see whether it fits one of them Thomas S. Kuhn.
- Kuhn's line: "Discovery commences with the awareness of anomaly" Thomas S. Kuhn.
2.3 Decide which odd results deserve attention
Log every discrepancy, keep working, and escalate only when the anomaly is serious or will not go away.
- Log discrepancies and keep working. Most yield to normal practice.
- Do not stop the main work for a single odd result Thomas S. Kuhn.
- The speed of sound gap was later closed by heat experiments done for other purposes Thomas S. Kuhn.
- Mercury's gap vanished with general relativity Thomas S. Kuhn.
- Escalate when the anomaly meets one or more of four tests. Usually several combine before a real crisis starts.
- It hits a fundamental generalization.
- It blocks an important use.
- New measurement tools give it more weight.
- It resists the best people for a long time Thomas S. Kuhn.
- Kuhn's reminder: "There is no such thing as research without counterinstances" Thomas S. Kuhn.
2.4 Look again, and expect your own resistance
Old categories swallow new things quietly, so hold your labels loosely and keep watching.
- Hold back on labeling an observation that looks slightly off. Watch it over time.
- Familiar categories absorb new things without anyone noticing Thomas S. Kuhn.
- A small shift in categories can prepare a team for a run of further finds.
- After the shift on Uranus, astronomers identified twenty minor planets with standard instruments in the first fifty years of the nineteenth century Thomas S. Kuhn.
- Expose people to the anomaly repeatedly, and expect resistance, including your own. Categories shift with exposure, but not for everyone.
- People first force an anomaly into an existing category without noticing. With more exposure they hesitate, grow confused, then often switch suddenly Thomas S. Kuhn.
- In the Bruner and Postman card experiment, even at forty times the exposure needed for normal cards, more than 10 per cent of anomalous cards went unidentified Thomas S. Kuhn.
2.5 Test hard before you escalate
Agree a demanding test first, pass it, then move the news up one level at a time.
- Agree on a demanding test that proves the result is real, and run it before telling leadership. The test should be one the result could easily fail.
- Choose the test with your team before you run it
Jon Gertner, Bell Labs. - The test for an amplifier: "The acid test of an amplifier is whether it can be made to oscillate"
Jon Gertner, Bell Labs. - When the amplifier did oscillate, Bown called it a basically new thing in the world
Jon Gertner, Bell Labs.
- Choose the test with your team before you run it
- Move the news up one level at a time, once each supervisor is convinced. A false alarm upward is the worst outcome.
- Do not skip a level, even for exciting news
Jon Gertner, Bell Labs.
- Do not skip a level, even for exciting news
2.6 Where the books differ
| Kuhn: watch over time | Gertner: test once, hard | |
|---|---|---|
| Main risk | Ignoring the anomaly | Raising it too early |
| Works over | Years of persistence | A single decisive test |
| Use for | Long-standing gap | Fresh, testable result |
| Action | Wait for persistence and weight | Design the hard test, run it first |
| Example | Mercury's orbit, speed of sound | Amplifier made to oscillate |
Kuhn warns against ignoring anomalies, while Gertner's Bell Labs story warns against raising them too early.
- Kuhn's filter works over time, and Gertner's gate works on a single test. Pick the one that fits the kind of anomaly you have.
- For a long standing gap, wait and watch for the signs of persistence and weight Thomas S. Kuhn.
- For a fresh result you can test directly, design the hard test and run it first
Jon Gertner, Bell Labs.
- Guard against both errors: a false alarm and your own resistance. The sources stress different risks.
- Gertner's account calls a false alarm upward the worst outcome
Jon Gertner, Bell Labs. - Kuhn says some people never adjust to an anomaly, so expect resistance, including your own Thomas S. Kuhn.
- Gertner's account calls a false alarm upward the worst outcome
Sources
- The Idea Factory: Bell Labs and the Great Age of American Innovation · Books
- The Structure of Scientific Revolutions: 50th Anniversary Edition · Books
Part of How a field works and shifts in the growth library.