Shopify

Shopify requires a working demo instead of slides for every product presentation. Shopify has banned static PowerPoint and Keynote product presentations for over two years. Every presentation must be a working demo, and executives refuse to look at descriptions of features that don't yet work.

Rabois offers this as evidence for replacing year-long roadmaps with fast noticing and exploiting of new model capabilities, since things impossible in November are easy by March. He also argues conventional PMs as intermediaries make less sense: "the idea of a PM makes no sense basically in the future."

David Senra▶ 1:21:35

Shopify says it still loves junior talent despite the shift to AI. Shopify publicly said it still loves junior talent. This is cited in support of the case against never hiring juniors again.

The argument is that juniors can be very productive fast if taught the AI-native way on day one. They have no bias against AI coding tools, and strong CS grads are more available as big companies cut hiring. Mix them with experienced people. Quote: "they can come in and like crush way beyond their years if you're willing to do like AI native onboarding for them".

Lenny's Podcast▶ 1:03:10

Shopify found about a third of A/B test effects vanished against a holdout. Shopify's team checked test effects later against a holdout group and found that about a third of them disappeared.

Cohen uses this to argue against routine A/B tests of small details. Real wins are rare, so most positive results are false positives even with 95% accurate tools, and teams can stack winners for a year and see no change in conversion. Lenny notes it can still pay off at millions-of-users scale. Quote: "AB testing doesn't work very well and it doesn't work on most things."

Lenny's Podcast▶ 1:19:00