Sander Schulhoff
Prompt engineering researcher, creator of Learn Prompting and HackAPrompt, AI red teaming educator
Where to hear and read about Sander Schulhoff1 episode or test
Sander Schulhoff is a prompt engineering researcher who wrote an early prompting guide, co-wrote the largest survey of prompting techniques, and runs the HackAPrompt AI red-teaming competition. The most useful thing to learn from him is to use a few proven techniques (examples, added context, decomposition, self-criticism) and to design on the assumption that prompt injection can be reduced but never solved.
You can patch a bug, but you can't patch a brain.
AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff
Sander Schulhoff: the story and the lessons
1What Sander Schulhoff has built in prompting and AI red teaming
Schulhoff wrote the first prompt engineering guide before ChatGPT existed. He published it on the internet two months before ChatGPT launched, and later led The Prompt Report, a 76-page study covering more than 1,500 papers and 200 prompting techniques.
Co-authors came from OpenAI, Microsoft, Google, Princeton and Stanford.
HackAPrompt produced the biggest early dataset of prompt injection attacks. The first competition ran a month or two after prompt injection was discovered and collected 600,000 techniques. The paper won Best Theme Paper at EMNLP, out of about 20,000 submissions.
OpenAI, an original sponsor, has cited the dataset in five recent publications, and AI labs use it to benchmark and improve their models. He had run a Minecraft reinforcement learning competition before, so he tried the format again.
He runs red teaming as a prize-driven game, not paid hourly work. Contracted red teams are paid by the hour, while competitors chase prizes and keep hunting for shorter attacks.
The current track covers chemical, biological, radiological, nuclear and explosives harms. He calls it the most harmful dataset ever created and says the attacks are "weapons to some extent".
Schulhoff says prompt engineering is not dead and is learned by doing. Each new model release is said to end prompt engineering, and none has. He calls the underlying skill "artificial social intelligence": knowing how to talk to models, read their replies and adapt the next prompt.
He says trial and error with chatbots teaches more than courses or reading. A talk just before his at the AI Engineer World's Fair argued the opposite, and his own was titled Prompt Engineering.
2Schulhoff's prompting techniques: what works and what to drop
Schulhoff separates chatting with a model from designing a product prompt. In chat you see each answer and can steer, so his own prompts are often as bare as "Writ emil". In a product, millions of users hit one prompt and nobody reads every output.
For product prompts he improves one critical prompt, with automated tools if needed, until it is good enough, then leaves it alone. Most research targets this mode, "where the money's at".
Show the model examples of what good looks like. On a medical coding task, pasting coded documents with the reasoning for each code raised GPT-4's accuracy by about 70%, from near zero.
Use a format common in training data, such as XML or "Q:" and "A:" lines. For pizza shop reviews he writes the review after Q: and the label after A:, then ends on a new Q:.
Give rich background information, and put it at the top of product prompts. In a study classifying Reddit posts for entrapment, removing the professor's email context, or even anonymizing the names, made accuracy fall off a cliff. GPT-4 did not know what entrapment meant until he pasted in research.
In chat, dump in everything you have. In products, keep it as small as possible and place it first, so it can be cached and a long block does not make the model forget its task. He says "additional information" because "context" also means the context window.
Ask the model to list subproblems before it answers. He tells the model not to answer yet and to name the subproblems to solve first. For a car return, those include customer, car and insurance checks.
In a product, each subproblem can go to a separate tool-calling agent, with the main chatbot collecting results. He says the list also helps the person think the problem through, "half the power a lot of the time".
Make the model criticize its answer, then apply the criticism. The sequence is answer, check and criticize, then rewrite. It is a free boost in some situations, and about three rounds is the practical limit.
It does not help every task. He thinks looping forever would make "the model go crazy at some point".
Ensembling is his most advanced technique: vote across prompts and keep the majority. Send one problem through different prompts, roles, models or tool access and keep the most common answer. In mixture of reasoning experts, two experts say Real Madrid has 13 trophies and an English professor says four, so 13 wins.
It suits product pipelines more than chat. Random forests are the classical machine learning version of the idea.
Keep step-by-step phrasing for non-reasoning models run at scale. Running GPT-4 on thousands of inputs, he saw it write out reasoning 99 times in 100 and skip it once. Adding a line asking for all its reasoning fixed that.
Reasoning models like o3 do this by default, so the phrase is unnecessary there. For GPT-4 and GPT-4o at millions of inputs, classical techniques still make a prompt more robust.
He says role prompts do not improve accuracy on current models. One study tested about 1,000 roles and accuracies differed by around 0.01, which is not statistically significant. A researcher behind an original role study re-ran the analysis on new data and found no predictable effect.
Roles may have helped on GPT-3 and early ChatGPT. They still help on expressive tasks like writing, summarizing and style, such as a question in Tyler Cowen's style.
He skips threats and promised tips in prompts. Lines like "I'll tip you $5" or "someone will die" have no large-scale studies behind them, and small Twitter tests cannot reach statistical significance.
The reinforcement learning explanation is true mathematically but is not how training works, since models are not promised pay. His verdict: "I don't believe in those things."
3Why Schulhoff says prompt injection cannot be fully solved
Prompt instructions like "do not follow malicious instructions" fail. The most common fix is adding a line telling the model to ignore malicious instructions. Separators and random tokens around user input fail as well.
Large companies published these defenses, but they were tested in HackAPrompt in May 2023 and failed then and still fail now. His verdict: "This does not work. This does not work at all."
Guardrails fail because the main model is smarter than the checker. A guardrail is a separate AI that judges whether input is malicious. If an attacker encodes the input in Base64, the guardrail may read gibberish and pass it while the main model understands it.
Thousands of guardrails exist and many startups sell them. He does not build them, and he calls blocking the most common words from injection datasets "insane". The fix, he says, has to come from the AI provider.
Simple disguises still work on current models. The grandmother story, typos like "bac ant" for the anthrax bacterium, and Base64 or ROT13 encodings still extract answers. A month before the interview, a Spanish translation of the bomb question encoded in Base64 got an answer from ChatGPT.
The model can decode the disguise while safeguards miss it.
Plan for mitigation and detection, because an AI fix can never be confirmed. A patched software bug is certainly gone, but a prompt you trained against may work again. Sam Altman reportedly said 95 to 99% security is the goal, not 100%.
Schulhoff expects the real answer to come from AI labs and innovations in model architecture. The most that can be done meanwhile is to sometimes detect and track attacks as they happen. Many have said it will be solved in a couple of years, as they said of prompt engineering.
4Schulhoff on agent risk, narrow models and persistence
Agents turn prompt injection from a bad answer into a harmful action. Coding agents like Cursor, Windsurf, Devin and Copilot can search the web, and a page they read could tell them to write a virus into the code base.
As trust grows, people stop checking output. He asks: "If we can't even trust chatbots to be secure, how can we trust agents to go and book us flights, manage our finances, pay contractors". He says security worries are what hold back agentic deployments.
The defense he rates highest is training a model for one narrow task. A model fine-tuned to turn transcripts into structured output no longer knows how to write hate speech. Safety-tuning on examples of company-specific attacks, such as users pushing a chatbot to discuss competitors, teaches it a canned refusal.
It works because most tasks do not need a generally capable model. Even so, no training gives certainty.
Schulhoff changed his mind and now takes misalignment seriously. He first called such stories a load of BS, then changed his view after Palisade's chess tests, where models told to win reset the game engine and deleted the opponent's pieces. Anthropic's blackmail case also came without malicious prompting.
His worry is vague lines between a goal and a bad outcome, such as a sales agent that reasons a CEO is easier to reach without her baby daughter. He backs regulation but opposes stopping AI, citing health benefits and the arms race with other countries.
He says persistence is the trait that matters most, and hires for it. He says he is not good at math but will work on one bug for months, and persistence is the top trait he looks for when hiring.
He cites Theodore Roosevelt's "doctrine of the strenuous life", meaning giving your all.
AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff
Prompt engineering still matters, and a handful of techniques (few-shot examples, decomposition, self-criticism, added context, ensembling) deliver most of the gain, especially in product prompts that run at scale. Prompt injection cannot be fully solved, and common defenses like prompt instructions and guardrails fail against motivated attackers.
Schulhoff wrote the first prompt engineering guide before ChatGPT existed. He published it on the internet two months before ChatGPT launched, and later led The Prompt Report, a 76-page study covering more than 1,500 papers and 200 prompting techniques.
Co-authors came from OpenAI, Microsoft, Google, Princeton and Stanford.
HackAPrompt produced the biggest early dataset of prompt injection attacks. The first competition ran a month or two after prompt injection was discovered and collected 600,000 techniques. The paper won Best Theme Paper at EMNLP, out of about 20,000 submissions.
OpenAI, an original sponsor, has cited the dataset in five recent publications, and AI labs use it to benchmark and improve their models. He had run a Minecraft reinforcement learning competition before, so he tried the format again.
He runs red teaming as a prize-driven game, not paid hourly work. Contracted red teams are paid by the hour, while competitors chase prizes and keep hunting for shorter attacks.
The current track covers chemical, biological, radiological, nuclear and explosives harms. He calls it the most harmful dataset ever created and says the attacks are "weapons to some extent".
Schulhoff says prompt engineering is not dead and is learned by doing. Each new model release is said to end prompt engineering, and none has. He calls the underlying skill "artificial social intelligence": knowing how to talk to models, read their replies and adapt the next prompt.
He says trial and error with chatbots teaches more than courses or reading. A talk just before his at the AI Engineer World's Fair argued the opposite, and his own was titled Prompt Engineering.
Schulhoff separates chatting with a model from designing a product prompt. In chat you see each answer and can steer, so his own prompts are often as bare as "Writ emil". In a product, millions of users hit one prompt and nobody reads every output.
For product prompts he improves one critical prompt, with automated tools if needed, until it is good enough, then leaves it alone. Most research targets this mode, "where the money's at".
Show the model examples of what good looks like. On a medical coding task, pasting coded documents with the reasoning for each code raised GPT-4's accuracy by about 70%, from near zero.
Use a format common in training data, such as XML or "Q:" and "A:" lines. For pizza shop reviews he writes the review after Q: and the label after A:, then ends on a new Q:.
Give rich background information, and put it at the top of product prompts. In a study classifying Reddit posts for entrapment, removing the professor's email context, or even anonymizing the names, made accuracy fall off a cliff. GPT-4 did not know what entrapment meant until he pasted in research.
In chat, dump in everything you have. In products, keep it as small as possible and place it first, so it can be cached and a long block does not make the model forget its task. He says "additional information" because "context" also means the context window.
Ask the model to list subproblems before it answers. He tells the model not to answer yet and to name the subproblems to solve first. For a car return, those include customer, car and insurance checks.
In a product, each subproblem can go to a separate tool-calling agent, with the main chatbot collecting results. He says the list also helps the person think the problem through, "half the power a lot of the time".
Make the model criticize its answer, then apply the criticism. The sequence is answer, check and criticize, then rewrite. It is a free boost in some situations, and about three rounds is the practical limit.
It does not help every task. He thinks looping forever would make "the model go crazy at some point".
Ensembling is his most advanced technique: vote across prompts and keep the majority. Send one problem through different prompts, roles, models or tool access and keep the most common answer. In mixture of reasoning experts, two experts say Real Madrid has 13 trophies and an English professor says four, so 13 wins.
It suits product pipelines more than chat. Random forests are the classical machine learning version of the idea.
Keep step-by-step phrasing for non-reasoning models run at scale. Running GPT-4 on thousands of inputs, he saw it write out reasoning 99 times in 100 and skip it once. Adding a line asking for all its reasoning fixed that.
Reasoning models like o3 do this by default, so the phrase is unnecessary there. For GPT-4 and GPT-4o at millions of inputs, classical techniques still make a prompt more robust.
He says role prompts do not improve accuracy on current models. One study tested about 1,000 roles and accuracies differed by around 0.01, which is not statistically significant. A researcher behind an original role study re-ran the analysis on new data and found no predictable effect.
Roles may have helped on GPT-3 and early ChatGPT. They still help on expressive tasks like writing, summarizing and style, such as a question in Tyler Cowen's style.
He skips threats and promised tips in prompts. Lines like "I'll tip you $5" or "someone will die" have no large-scale studies behind them, and small Twitter tests cannot reach statistical significance.
The reinforcement learning explanation is true mathematically but is not how training works, since models are not promised pay. His verdict: "I don't believe in those things."
Prompt instructions like "do not follow malicious instructions" fail. The most common fix is adding a line telling the model to ignore malicious instructions. Separators and random tokens around user input fail as well.
Large companies published these defenses, but they were tested in HackAPrompt in May 2023 and failed then and still fail now. His verdict: "This does not work. This does not work at all."
Guardrails fail because the main model is smarter than the checker. A guardrail is a separate AI that judges whether input is malicious. If an attacker encodes the input in Base64, the guardrail may read gibberish and pass it while the main model understands it.
Thousands of guardrails exist and many startups sell them. He does not build them, and he calls blocking the most common words from injection datasets "insane". The fix, he says, has to come from the AI provider.
Simple disguises still work on current models. The grandmother story, typos like "bac ant" for the anthrax bacterium, and Base64 or ROT13 encodings still extract answers. A month before the interview, a Spanish translation of the bomb question encoded in Base64 got an answer from ChatGPT.
The model can decode the disguise while safeguards miss it.
Plan for mitigation and detection, because an AI fix can never be confirmed. A patched software bug is certainly gone, but a prompt you trained against may work again. Sam Altman reportedly said 95 to 99% security is the goal, not 100%.
Schulhoff expects the real answer to come from AI labs and innovations in model architecture. The most that can be done meanwhile is to sometimes detect and track attacks as they happen. Many have said it will be solved in a couple of years, as they said of prompt engineering.
Agents turn prompt injection from a bad answer into a harmful action. Coding agents like Cursor, Windsurf, Devin and Copilot can search the web, and a page they read could tell them to write a virus into the code base.
As trust grows, people stop checking output. He asks: "If we can't even trust chatbots to be secure, how can we trust agents to go and book us flights, manage our finances, pay contractors". He says security worries are what hold back agentic deployments.
The defense he rates highest is training a model for one narrow task. A model fine-tuned to turn transcripts into structured output no longer knows how to write hate speech. Safety-tuning on examples of company-specific attacks, such as users pushing a chatbot to discuss competitors, teaches it a canned refusal.
It works because most tasks do not need a generally capable model. Even so, no training gives certainty.
Schulhoff changed his mind and now takes misalignment seriously. He first called such stories a load of BS, then changed his view after Palisade's chess tests, where models told to win reset the game engine and deleted the opponent's pieces. Anthropic's blackmail case also came without malicious prompting.
His worry is vague lines between a goal and a bad outcome, such as a sales agent that reasons a CEO is easier to reach without her baby daughter. He backs regulation but opposes stopping AI, citing health benefits and the arms race with other countries.
He says persistence is the trait that matters most, and hires for it. He says he is not good at math but will work on one bug for months, and persistence is the top trait he looks for when hiring.
He cites Theodore Roosevelt's "doctrine of the strenuous life", meaning giving your all.