All people

Guest · OpenAI

Kevin Weil

Chief Product Officer at OpenAI, former head of product at Instagram and Twitter

2 sources34 tips

Where to hear and read about Kevin Weil2 episodes and tests

Lenny's PodcastOpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin WeilLenny's PodcastA conversation with OpenAI's CPO Kevin Weil, Anthropic's CPO Mike Krieger, and Sarah Guo
In short

Kevin Weil is Chief Product Officer at OpenAI and formerly head of product at Instagram and Twitter. The one thing most worth learning from him is to build for the model arriving in two months, and to steer it with evals, fine-tuning and fast public iteration.

The AI models that you're using today is the worst AI model you will ever use for the rest of your life.

OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil

Kevin Weil: the story and the lessons

1Kevin Weil's path from Instagram to OpenAI

  • Weil built big consumer products before joining OpenAI. He was head of product at Instagram and Twitter, worked at Planet, and co-created the Libra cryptocurrency at Facebook. He sits on the boards of Planet, Strava, the Black Product Managers Network and the Nature Conservancy.

    At OpenAI he oversees ChatGPT, enterprise products and the API.

  • Weil joined OpenAI through an existing tie to Sam Altman. He had planned a summer off with his kids when he called Sam, and Vinod helped reconnect them. He met most of the management team within days, and Sam came to dinner.

    Sam told him that if the final round went well, "we're basically there".

  • Nine days of silence taught Weil not to over-read a hiring process. After the last interview he heard nothing for nine days and replayed every answer. OpenAI had a lot happening internally, and then said it had gone well.

    His takeaway: silence usually reflects the company's chaos, not your performance, so don't jump to conclusions and keep a little chill.

  • Weil calls the CPO job the hardest of his career. He describes it as the most challenging, sleepless product role of his life. OpenAI sells consumer, enterprise and developer products at once.

    At earlier companies the tools were stable and the work was about problems and users. At OpenAI the foundation changes every two months, where a database might improve 5% in two years.

2What Weil learned from Libra

  • Libra tried to make sending money as easy as sending a text. Remittances are regressive: people with little money pay around 20% to send it home, and it takes days. WhatsApp has 3 billion users, many of them friends and family who would send each other money.

    The aim was to let people send 50 cents in WhatsApp for free. David Marcus led the project, and Weil calls him a visionary, a mentor and a friend.

  • Weil says Libra changed too much at once. It bundled a new blockchain, a basket of currencies, and integration into WhatsApp and Messenger. It landed when Facebook's reputation was at its lowest, and Messenger was not the app people wanted for this.

    He would introduce the changes one at a time and owns the decisions with the team. He still counts it as probably the biggest disappointment of his career.

  • The Libra technology lived on after the product died. The tech was open-sourced, and Aptos and another company built on it. Both are doing well.

    Weil thinks Meta could build it now, since the current administration is friendly to crypto and Meta's reputation has recovered.

3Weil's model maximalism: build for the next model

  • Weil tells builders today's model is the worst they will use. He says models gain about 10x a year, a steeper curve than Moore's Law, which doubled transistors every 18 months. A new o-series model arrives roughly every three to four months.

    Before the o-series, a new GPT model came every six to nine months. The original GPT-3.5 cost about 100 times what GPT-4o mini costs in the API today, for much less intelligence.

  • He builds products right at the edge of what models can do. OpenAI tells developers to keep going if the product barely works, because a better model in about two months will blow past the limits. Bolt's builders spent years on a product that failed until Claude Sonnet 3.5 made it work.

    In his words: "if the product that you're building is right on the edge of the capabilities of the models, keep going, because you're doing something right".

  • Weil builds scaffolding only for errors nobody can tolerate. OpenAI sometimes builds guardrails, but not around flaws a better model will fix. The test is whether the mistake is one you simply don't want to make.

  • Weil expects users to get used to magic within minutes. A first Waymo ride goes from panic to checking email or a phone in about 10 to 15 minutes. He expects today's magic to look like garbage in 12 months.

    Waiting a minute for an AI image of your family already feels slow. Looking back at GPT-3, he says, you would be disgusted by how bad it was.

  • Weil plans around learning because capabilities arrive unpredictably. Every two months computers can do something they have never done before, and the product has to change with it. Nobody knows whether a new capability will work 60%, 90% or 99% of the time.

    He describes a new capability as an emergent property "coming through the mist".

4How Weil builds with evals, fine-tuning and ensembles

  • Weil turns hero use cases into evals before building. Evals are quizzes for a model, like unit tests. Deep research, which can work 25 to 30 minutes on a task that would take a person a week, was built by listing hero questions with ideal answers and fine-tuning until scores rose.

    Rising eval scores are the signal: "Okay, I think we have a product here."

  • He designs the product around the model's real accuracy rate. A model right 60% of the time needs a very different product than one right 95% or 99.5% of the time. These rates keep changing, so he keeps re-measuring as models improve.

  • Weil designs for a human in the loop when accuracy is partial. A model right only 60% of the time can still be valuable if the product expects the human to edit or confirm. GitHub Copilot launched on a small, imperfect model and still saved typing.

    A model that flags where it is unsure and asks for help lifts the combined result well above 60%.

  • He says to read the outputs, not trust the score. A new model scoring 78% against 80% may only have a worse grader. Some golden answers in model-card evals are doubtful, so 100% is hard to reach.

    Mike Krieger cites Andrej Karpathy on the point: nothing beats looking at the data.

  • Weil argues models are eval limited, not intelligence limited. Claude can write and grade evals, but the team must say what success looks like. That step can move a task from 60% to 85%.

    In pilots, one company said a model solved a three-month problem while another, the same day, said it was worse than the older model. Customers use their own data and prompts, so real use is the test.

  • He splits problems across an ensemble of specialized models. Ten problems might use 20 model calls, with models of different sizes, custom prompts and fine-tunes. Support for 400 million weekly users runs with only 30 to 40 people because most flows are automated.

    Slower o-series models handle reasoning and 4o mini handles quick, cheap checks. He compares it to a company of people with different skills.

  • Weil wants researchers or ML engineers on every product team. Fine-tuning will be part of the core product workflow, and he is surprised how few companies use it. OpenAI now starts every product with the loop of building evals, gathering data and fine-tuning.

    He says separating research from product turns product teams into API customers.

  • He recommends examples and role framing in prompts. Pasting example questions with good answers is a poor man's fine-tuning and works far better than none. Telling the model its role, like the world's greatest marketer, shifts it into a useful mindset.

    He expects prompt engineering to fade over time.

5How OpenAI ships under Weil

  • Weil plans lightly and lets teams drive bottoms up. Quarterly roadmapping is a pause to ask what worked and to check dependencies like infrastructure and research partnerships. Roadmaps often get thrown out halfway through the quarter.

    "Plans are useless. Planning is helpful." Engineers, PMs, designers and researchers who build the products know what the models can really do.

  • He ships early and iterates in public instead of waiting for reviews. No review ritual is allowed to block a launch, and only the biggest, highest-priority projects get a close look. Some launches get rolled back, and Sam Altman pushes hard to move fast.

    He calls it iterative deployment: learning about the models together and co-evolving with society. Model naming is "absolutely atrocious", yet ChatGPT is still the top product.

  • Weil keeps the PM team small at about 25 people. Each PM gets slightly too many engineers so they cannot micromanage. That leaves decisions with high-agency, product-minded engineers.

    "Too many PMs causes problems. We'll fill the world with decks and ideas versus execution."

  • Research and product work as one team at OpenAI. ChatGPT began as a low-key research preview. Weil says OpenAI should never be a pure product company and must be world-class at both.

    Early experiments with research aim at demos, not finished products. Artifacts and Canvas were co-designed, co-researched and co-fine-tuned.

  • Heavy internal use is Weil's signal a social product will work. Staff used the new image model for about two months, and its internal gallery generated nonstop buzz. Instagram Stories felt the same inside the company.

    A social product that doesn't take off among close colleagues who know it should make you question it.

  • Weil believes in steady good work over silver bullets. He keeps a poster of Zuckerberg's earnings-call line that growth is not one thing, just good work consistently over a long period. As an ultra marathon runner, he sees small daily gains compounding over years.

    People get dismayed when they see no change over a week or a month, and they stop.

6Weil on hiring and being a PM in the AI era

  • Weil hires PMs for high agency and comfort with ambiguity. OpenAI wants people who start things on their own, make calls when nobody else will, and execute fast. Nobody hands a new PM a defined area, so junior PMs sometimes struggle.

    Problems are ill-formed and the team is figuring them out "as we go".

  • He wants PMs to lead through influence and still be decisive. Researchers are self-directed, so rapport and EQ matter and a PM must earn trust. If no one is deciding, the PM makes sure a call gets made.

    The decider need not be the PM, and Weil dislikes the PM as CEO of the product idea.

  • Weil treats writing evals as a core PM skill. Feature quality is now gated on the evals and the prompts. OpenAI ran a boot camp on evals for every PM, and Anthropic tests eval writing in interviews.

    At Anthropic, the code analysis feature was 80% built by one PM, then handed to another who wrote the evals, which showed the roles had merged.

  • He wants prototypes in place of Figma mocks. Teams should show vibe-coded prototypes built in about 30 minutes with tools like Windsurf or Cursor. OpenAI's chief people officer, Julia, vibe coded an internal tool, so "we have no excuse".

    Vibe coding suits demos and proofs of concept, not tight production code yet. Anthropic's best PMs also prompt Claude for A/B comparisons of UIs before designers open Figma.

  • Weil says PMs should go deeper into the tech stack. Not every PM must be a researcher, but they should learn the language and build intuition. He compares it to database work, where 2005 demanded more depth than today as abstraction layers grew.

    He says the depth needed will likely shift over the years. He would still recognize his day job, and thinks he shouldn't.

7How Weil designs AI product experiences

  • Weil designs AI wait times by asking what a person would do. A person wouldn't go mute for 20 seconds and wouldn't voice every thought, so the model gives short status updates. For long tasks like deep research, users leave and come back.

    After DeepSeek showed its full reasoning, OpenAI saw a novelty effect. At 400 million users it settled on one or two sentence summaries, since three paragraphs is wrong.

  • He keeps chat as the open-ended catch-all interface. Chat's lack of structure is its strength, and it never worked before because no model could handle the nuance of human speech. For high-volume, prescribed tasks, faster dedicated interfaces can beat it.

    Chat remains the baseline for anything outside the vertical.

  • Weil thinks of model behavior as a product role. Users notice a model's personality and care about it. Krieger says you are shipping intelligence, so an upgrade changes a relationship.

    The open question is how much a model should customize to each user versus keep one personality.

  • He designs feedback loops and guardrails for non-deterministic output. Same input, different output breaks 25 years of intuition about how computers behave. Teams must decide how to spot a model going astray, collect feedback fast and watch the aggregate.

    Krieger says teams must get "Zen" about letting go of control.

  • Weil uses the model to teach users and enterprise power users to spread adoption. Claude now links to exact documentation at launch, after early users got generic advice about its own features. Organizations build thousands of custom GPTs, and internal power users act as evangelists.

8Where Weil sees AI and opportunity going

  • Weil sees startup room in verticals with private data. No single company can serve every industry, and most data and process knowledge sits behind company or government walls. About 3 million developers use the API, and he expects far more.

    He quotes Ev Williams: "there are way more smart people outside your walls than there are inside your walls".

  • He expects proactive, asynchronous AI that works in the background. With permission, a model could read your email and start your day with a recap, research for your next meeting and a draft presentation. o1 thinks 30 to 60 seconds now, and Weil expects hours or days later.

    He says OpenAI is at the GPT-1 phase of this new way to scale intelligence. Quick answers still have their place.

  • Weil points computer use at drudgery first. Early fits are UI testing and moving data between silos in support and finance teams. During a beta, an employee had it order a pizza, and Domino's showed up at the office.

  • He plans for enterprise buyers who have their own goals. A deal can take six months to reach deployment, and a product every employee likes may still not matter to the buyer. One large customer asked for 60 days' notice before any launch.

    The upside is that deployed customers will tell you what works, while consumers have little incentive to.

  • Weil ranks personalized AI tutoring among AI's most important jobs. He says studies show multiple standard deviation gains in learning speed, the models are good enough, ChatGPT is free and Android phones are everywhere. Nothing yet serves 2 billion kids.

    He puts it alongside faster scientific discovery, and says society must make the transition as graceful and well-supported as it can.

More from OpenAI

Guest

Tara Seshan

Product lead for ChatGPT Work and Codex at OpenAI; ex-Stripe, Watershed, Sutter Hill EIR

OpenAI
2 sources · 33 tips
Guest

Alexander Embiricos

Product lead for Codex at OpenAI

OpenAI
1 source · 18 tips
Guest

Andrew Ambrosino

Product and engineering lead for the Codex desktop app at OpenAI

OpenAI
1 source · 18 tips
Guest

Ian Silber

Head of product design at OpenAI (ChatGPT, Codex); formerly Artifact and eight years at Instagram

OpenAI
1 source · 16 tips
Guest

Krithika Shankarraman

Executive in Residence at Thrive Capital; first marketing hire at OpenAI and Stripe, former marketing lead at Retool

OpenAI
1 source · 19 tips
Guest

Logan Kilpatrick

Head of developer relations at OpenAI

OpenAI
1 source · 17 tips