Tulsee Doshi
Head of Product for Google's frontier AI models (Gemini, Nano Banana, Veo, Genie, Lyria, Gemma)
Where to hear and read about Tulsee Doshi1 episode or test
Tulsee Doshi is Head of Product for Google's frontier AI models, including Gemini, Nano Banana, Veo, Genie, Lyria and Gemma, and spent six years before that leading product for Google's responsible AI team. The thing most worth learning from her is to judge what a model actually does through small hands-on evals, and to build products that grow as the model improves.
what you should be providing is the insight. What the model should be providing is the data
Why Google wants to win the frontier conversation | Tulsee Doshi (Head of Product for Gemini)
Tulsee Doshi: the story and the lessons
1Tulsee Doshi's path to running Google's frontier model product
Doshi leads product across Google's frontier models, not only Gemini. Her portfolio covers Gemini, gen media models for video and image (Nano Banana) and music (Lyria), the open Gemma family, and audio models for dialogue, understanding, text to speech and translation.
She counts all of these as frontier models and says audio gets less attention than it deserves.
She spent six years leading product for Google's responsible AI team. Her safety view is to monitor a model's chain of thought to catch misalignment, and to use models to find and patch vulnerabilities.
Gemini 4 work includes partnering with cyber defenders such as Wiz. Its Scan for Good program helped a hospital system find and patch a vulnerability.
Her Stanford research on flash teams now applies to agent swarms. She researched flash teams with Michael Bernstein, using crowdsourced mTurk workers who were assembled for a task and then disbanded.
She sees the same principles in systems of sub agents. She cites agents that assigned themselves roles, including a leader, without being trained to.
She argues Google competes through its full stack, and admits one gap. Owning hardware, models and products with a billion users lets Google build for every point on the cost to quality curve. She cites about 14 products with over a billion users, a number she says to double check.
She concedes Google has no standout personal assistant yet despite having the data for one: "there is more work we need to do". Her message is to stay tuned.
2How Doshi judges and shapes a frontier model
Doshi studies what a model became, because training never lands as planned. Teams set goals and watch training runs closely, but she expects some magic. A PM's job is to look at what comes out and find where the model shines in surprising ways.
She likens it to Snape's potions class: you add the ingredients, but expect some magic. Then she shapes those strengths into something great for users.
She treats model behavior as a kind of UX. Transparency, how well the model communicates and which questions it asks back all count. PMs turn a spec for good behavior into rubrics, then data, then reward modeling.
She works with a dedicated model behavior team and calls it a nice collaboration between PM and research. Customer and user feedback feeds the spec.
She starts evals with five to twenty prompts, not a giant benchmark. A mini eval of 5 to 20 prompts, with you as the rater, can show signal between model A and model B.
Its main payoff is clarity about what good looks like. Only after that do you figure out how to scale it up. She says rubrics and vibes are both needed.
She stress tests new models on tasks that are almost solved. Pick a task you care about where models show signs of life but are not perfect, such as 3D animation, creative writing, essays or hard financial analysis.
Imperfection lets you see how different models handle the same problem. PRDs or strategy docs work too.
She evaluates the whole stack, not just the model. Judges, mitigations, filters and serving all change what users get, and a weak judge gives a false picture even of a good model.
Audio is the clearest case: serving can hurt naturalness, tenor and inflection even when the model is good. She wants a clean end-to-end setup.
Her team replaced a skewed skin tone scale with a better one. The six-tone Fitzpatrick scale was built for healthcare and skewed toward light skin. Her team redid evaluations around the 10-tone Monk scale, built by Harvard professor Ellis Monk from years of interviews and research.
It improved Pixel camera work and image generation. Her point is to check whether an eval's yardstick was built for a different purpose.
3Doshi's 60-20-20 rule for building ahead of the model
Doshi builds 60% that works now, 20% almost works, 20% is a bet. About 60% of the product should work today, 20% should work well enough to see where models are heading, and the last 20% bets on glimmers of light.
She says the exact numbers are fungible and the thesis is what matters. She argues the classic MVP that fully works falls short when models keep changing.
She uses product proxies instead of guessing where the model will be. Common advice is to build three months ahead of the models. She finds it hard to say where the model will be in three months, even working on it.
Instead she asks where in the product you see glimmers of light. It works like a portfolio, with the bold bet now being one that works in six months.
She retests the harness with every new model and strips old scaffolding. Her harness is the system prompt, tools, context and skills built around a model. Rigid systems built around an older model can hamper a better one.
She cites Boris Cherny's advice to remove all your skills once in a while and see what happens. Often the model is now as good as what you built in.
She matches UX flexibility to how expert the user is. AI Mode must stay accessible to billions of search users, while Antigravity can assume slash commands across terminal and IDE.
Labs products like Flow and NotebookLM keep testing new form factors.
4What Doshi looks for in PMs and how she builds her team
She says taste and consensus-building matter more as building gets cheap. When anyone can build anything, the scarce skill is separating signal from noise and finding the insight that unlocks a great experience.
She notes people are building things nobody wants. Speed without structure creates chaos, and someone has to bring people together. She admits she is biased as a PM.
She hires PMs who play with models daily and hold opinions. She wants PMs with their own tests for what makes a model good, who stress test capabilities and are technical enough to dig into data and build an eval.
She also wants agility, since a roadmap from two months ago gets rewritten. "I want PMs who have an opinion."
She asks 'can AI do this for me' before starting a task. One example that works well changes your mindset, and confidence grows from there. Her rule of thumb is that if you do something twice, consider outsourcing it.
She excludes work like writing a strategy, which stays hers. She found it hard to leave her habit of doing what she already knew how to do.
She is building a shared skills repository for her PM team. PMs who lean in build technical skills that others reuse and improve, which she calls an iteration flywheel. Workshops teach eyeballing data and reading long traces and trajectories.
The aim is confidence with the tools.
She wants protected time for PMs to learn AI tools. Constant sprinting leaves no room to step back, and she says learning the tools accelerates people 10x. A two-day November workshop will pull all PMs out of their day jobs to build and play.
She admits the team is not there yet. She doesn't know if Google's 20% time still exists, and her PMs are working 150%.
She lets AI supply data in docs and keeps insight human. A post-launch update is a good AI doc because it pulls dashboards, metrics and social feedback faster than a person can.
She says the reading burden has replaced the writing burden, so her team invests in skills that pull the right data and structure artifacts well.
She treats burnout as a problem of churn and joyless work. She sees burnout coming from chaos and grunt work, not only long hours. Passion work is still exhausting but brings more joy.
A leader's job is to learn what motivates each person. She admits sustainability for her team is still an open question.
Why Google wants to win the frontier conversation | Tulsee Doshi (Head of Product for Gemini)
Google's head of product for Gemini explains how PMs shape frontier models, from writing a first eval to building ahead of model capabilities with her 60-20-20 rule. She also covers what makes PMs more valuable as building gets cheap and how her team builds AI habits.
Doshi leads product across Google's frontier models, not only Gemini. Her portfolio covers Gemini, gen media models for video and image (Nano Banana) and music (Lyria), the open Gemma family, and audio models for dialogue, understanding, text to speech and translation.
She counts all of these as frontier models and says audio gets less attention than it deserves.
She spent six years leading product for Google's responsible AI team. Her safety view is to monitor a model's chain of thought to catch misalignment, and to use models to find and patch vulnerabilities.
Gemini 4 work includes partnering with cyber defenders such as Wiz. Its Scan for Good program helped a hospital system find and patch a vulnerability.
Her Stanford research on flash teams now applies to agent swarms. She researched flash teams with Michael Bernstein, using crowdsourced mTurk workers who were assembled for a task and then disbanded.
She sees the same principles in systems of sub agents. She cites agents that assigned themselves roles, including a leader, without being trained to.
She argues Google competes through its full stack, and admits one gap. Owning hardware, models and products with a billion users lets Google build for every point on the cost to quality curve. She cites about 14 products with over a billion users, a number she says to double check.
She concedes Google has no standout personal assistant yet despite having the data for one: "there is more work we need to do". Her message is to stay tuned.
Doshi studies what a model became, because training never lands as planned. Teams set goals and watch training runs closely, but she expects some magic. A PM's job is to look at what comes out and find where the model shines in surprising ways.
She likens it to Snape's potions class: you add the ingredients, but expect some magic. Then she shapes those strengths into something great for users.
She treats model behavior as a kind of UX. Transparency, how well the model communicates and which questions it asks back all count. PMs turn a spec for good behavior into rubrics, then data, then reward modeling.
She works with a dedicated model behavior team and calls it a nice collaboration between PM and research. Customer and user feedback feeds the spec.
She starts evals with five to twenty prompts, not a giant benchmark. A mini eval of 5 to 20 prompts, with you as the rater, can show signal between model A and model B.
Its main payoff is clarity about what good looks like. Only after that do you figure out how to scale it up. She says rubrics and vibes are both needed.
She stress tests new models on tasks that are almost solved. Pick a task you care about where models show signs of life but are not perfect, such as 3D animation, creative writing, essays or hard financial analysis.
Imperfection lets you see how different models handle the same problem. PRDs or strategy docs work too.
She evaluates the whole stack, not just the model. Judges, mitigations, filters and serving all change what users get, and a weak judge gives a false picture even of a good model.
Audio is the clearest case: serving can hurt naturalness, tenor and inflection even when the model is good. She wants a clean end-to-end setup.
Her team replaced a skewed skin tone scale with a better one. The six-tone Fitzpatrick scale was built for healthcare and skewed toward light skin. Her team redid evaluations around the 10-tone Monk scale, built by Harvard professor Ellis Monk from years of interviews and research.
It improved Pixel camera work and image generation. Her point is to check whether an eval's yardstick was built for a different purpose.
Doshi builds 60% that works now, 20% almost works, 20% is a bet. About 60% of the product should work today, 20% should work well enough to see where models are heading, and the last 20% bets on glimmers of light.
She says the exact numbers are fungible and the thesis is what matters. She argues the classic MVP that fully works falls short when models keep changing.
She uses product proxies instead of guessing where the model will be. Common advice is to build three months ahead of the models. She finds it hard to say where the model will be in three months, even working on it.
Instead she asks where in the product you see glimmers of light. It works like a portfolio, with the bold bet now being one that works in six months.
She retests the harness with every new model and strips old scaffolding. Her harness is the system prompt, tools, context and skills built around a model. Rigid systems built around an older model can hamper a better one.
She cites Boris Cherny's advice to remove all your skills once in a while and see what happens. Often the model is now as good as what you built in.
She matches UX flexibility to how expert the user is. AI Mode must stay accessible to billions of search users, while Antigravity can assume slash commands across terminal and IDE.
Labs products like Flow and NotebookLM keep testing new form factors.
She says taste and consensus-building matter more as building gets cheap. When anyone can build anything, the scarce skill is separating signal from noise and finding the insight that unlocks a great experience.
She notes people are building things nobody wants. Speed without structure creates chaos, and someone has to bring people together. She admits she is biased as a PM.
She hires PMs who play with models daily and hold opinions. She wants PMs with their own tests for what makes a model good, who stress test capabilities and are technical enough to dig into data and build an eval.
She also wants agility, since a roadmap from two months ago gets rewritten. "I want PMs who have an opinion."
She asks 'can AI do this for me' before starting a task. One example that works well changes your mindset, and confidence grows from there. Her rule of thumb is that if you do something twice, consider outsourcing it.
She excludes work like writing a strategy, which stays hers. She found it hard to leave her habit of doing what she already knew how to do.
She is building a shared skills repository for her PM team. PMs who lean in build technical skills that others reuse and improve, which she calls an iteration flywheel. Workshops teach eyeballing data and reading long traces and trajectories.
The aim is confidence with the tools.
She wants protected time for PMs to learn AI tools. Constant sprinting leaves no room to step back, and she says learning the tools accelerates people 10x. A two-day November workshop will pull all PMs out of their day jobs to build and play.
She admits the team is not there yet. She doesn't know if Google's 20% time still exists, and her PMs are working 150%.
She lets AI supply data in docs and keeps insight human. A post-launch update is a good AI doc because it pulls dashboards, metrics and social feedback faster than a person can.
She says the reading burden has replaced the writing burden, so her team invests in skills that pull the right data and structure artifacts well.
She treats burnout as a problem of churn and joyless work. She sees burnout coming from chaos and grunt work, not only long hours. Passion work is still exhausting but brings more joy.
A leader's job is to learn what motivates each person. She admits sustainability for her team is still an open question.
More from Google
Founder storyLarry Page
Founders of Google
Google
3 sources · 58 tips
GuestItamar Gilad
Product coach and author of Evidence-Guided, former product manager at Google (Gmail, YouTube) and Microsoft
Google1 source · 19 tips
GuestRobby Stein
VP of Product, Google Search (formerly head of product at Instagram)
Google1 source · 16 tips
Founder storySergey Brin
Founders of Google
Google1 source · 20 tips