Acquired · Aug 26Google Part II: Alphabet
Its launch made owning a browser urgent for Google, since no payment could keep Google as the default if Microsoft made Bing the Internet Explorer default. It later powered Yahoo search.
Tips
App · Other
Bing
Microsoft's search engine, launched in 2009.
What was said about Bing
Experimentation and analytics3
- Expect most experiments to fail and plan for it▶ 13:16Failure rates were about 66% at Microsoft, about 85% at Bing, and 92% in Airbnb search relevance, with Google Ads and Booking reporting 80 to 90%.
Lenny's Podcast · The ultimate guide to A/B testing | Ronny Kohavi (Airbnb, Microsoft, Amazon) - Define an OEC with a countervailing metric, not just revenue▶ 27:59Amazon's email team credited all email-driven purchases, so they spammed; after modeling the cost of an unsubscribe (a few dollars), more than half of campaigns turned out negative.Lenny's Podcast · The ultimate guide to A/B testing | Ronny Kohavi (Airbnb, Microsoft, Amazon)
- Test small, cheap ideas even when the backlog ranks them low▶ 4:29The change raised Bing revenue about 12%, worth over $100 million at the time, with no significant guardrail damage.Lenny's Podcast · The ultimate guide to A/B testing | Ronny Kohavi (Airbnb, Microsoft, Amazon)
Competition and markets1
Capital and deals1
Product strategy and PMF1
Teams and execution1
Acquired · Aug 26Acquired · Jul 21Microsoft Volume II: The Complete History and Strategy of the Ballmer Years
A Yahoo search deal gave Bing scale, and Microsoft sold ads for both. Satya Nadella briefly led it, and it earns roughly a billion dollars a year in profit.
Tips
Lenny's Podcast · Jul 27The ultimate guide to A/B testing | Ronny Kohavi (Airbnb, Microsoft, Amazon)
About 85% of Bing experiments failed. Moving the second ad line up to the title line raised revenue about 12%, worth over $100 million, and became its biggest revenue win. A year and a half of social integration experiments with Twitter and Facebook came out flat to negative before the team aborted it.
Tips
- Expect most experiments to fail and plan for it▶ 13:16Failure rates were about 66% at Microsoft, about 85% at Bing, and 92% in Airbnb search relevance, with Google Ads and Booking reporting 80 to 90%.
- Define an OEC with a countervailing metric, not just revenue▶ 27:59Amazon's email team credited all email-driven purchases, so they spammed; after modeling the cost of an unsubscribe (a few dollars), more than half of campaigns turned out negative.
- Run an ongoing portfolio: many small bets plus some big ones▶ 20:44Bing relevance targeted about 2% yearly gains from 0.1 to 0.2 steps; 250 Airbnb search experiments added up to a 6% revenue gain.
- Test small, cheap ideas even when the backlog ranks them low▶ 4:29The change raised Bing revenue about 12%, worth over $100 million at the time, with no significant guardrail damage.
- Start experimentation in a team that ships often with a clear OEC▶ 1:07:43Bing was the beachhead at Microsoft, and Office later adopted experiments after seeing its ideas fail too.
More in Other
OtherApple
A consumer technology company that makes the Mac, iPod, iPhone and software.
74 sources · 200 tips
OtherGoogle's web search product, now including AI Overviews, AI Mode and Lens.
47 sources · 97 tips
OtherGoogle Play
Google's app store for Android, with its own listing rules and search ranking.
18 sources · 49 tipsStandard Oil
John D. Rockefeller's oil refining company that came to dominate American kerosene and oil distribution.
5 sources · 48 tips
OtherTSMC
The world's leading contract chip maker, a pure-play foundry that manufactures chips for fabless designers.
6 sources · 32 tips
OtherTesla
An electric car maker that builds its own key components and factories.
13 sources · 23 tips
OtherIntel
A semiconductor company that made memory chips and the first microprocessor.
10 sources · 20 tips
OtherSpaceX
A private aerospace company building reusable rockets, the Dragon capsule, Starship and the Starlink satellite network.
4 sources · 27 tips