Chapter 26 of 26 · Prototyping, testing and accessibility
1 The bottom line
Treat every suggestion as a symptom, find the root cause, and retest each fix for side effects. Players report where it hurts, not what is wrong, and fixes often break something else.
2 Key takeaways
- Treat a suggested fix as a symptom, not an instruction. Players see the pain, but not the cause or the knock-on effects.
- On Dying Light, the reported problem was weapons breaking too fast. The real one was that players could kill only a few zombies per weapon, so lowering zombie health fixed it
Mark Brown, Game Maker's Toolkit1:47. - Edmund McMillen says a complaint about controls could mean one of hundreds of things
Mark Brown, Game Maker's Toolkit8:25.
- On Dying Light, the reported problem was weapons breaking too fast. The real one was that players could kill only a few zombies per weapon, so lowering zombie health fixed it
- Filter feedback through your goal and through who is speaking. Loud voices skew the picture, and answering everything makes a forgettable game.
- Insomniac's Ryan Schneider says Resistance 2 suffered from listening too intently to a small noisy group
Mark Brown, Game Maker's Toolkit5:32. - Half-Life 2 testers loved the short intro, but Valve delayed combat anyway to serve its goal Mark Brown, Game Maker's Toolkit14:53.
- Insomniac's Ryan Schneider says Resistance 2 suffered from listening too intently to a small noisy group
- Ask what a bad-looking feature is for before you remove it. Friction can be carrying the design.
- Echoes of Wisdom's long echo list is meant to make players stumble on echoes they forgot Mark Brown, Game Maker's Toolkit17:09.
- DOOM 3's torch-or-gun choice was fiddly but drove the atmosphere, and the BFG Edition lost some of it
Mark Brown, Game Maker's Toolkit11:35.
- Hold out for a fix that solves several problems at once. Narrow patches tend to leave the cause in place.
- New Super Mario Bros. Wii's respawn bubble handles waiting players and lets struggling novices set their own difficulty
Mark Brown, Game Maker's Toolkit11:10. - Astroneer's team found two designers were describing two different crafting problems, then fixed both with one overhaul
Mark Brown, Game Maker's Toolkit2:57.
- New Super Mario Bros. Wii's respawn bubble handles waiting players and lets struggling novices set their own difficulty
- Retest after every fix. A fix can create a new problem somewhere else.
- In Mind Over Magnet, a fix created a bug elsewhere, so the game went through several QA rounds Mark Brown, Game Maker's Toolkit4:34.
- The Witcher 3's cow fix spawned a monster that players farmed for profit, so it had to be patched again Mark Brown, Game Maker's Toolkit15:15.
When responding to feedback, ask specific questions and try to find the root of the problem
Mark Brown, Game Maker's Toolkit
2.1 Decide whether the feedback deserves action
Before you fix anything, check how many people said it, who they are, and whether it serves your goal.
- Weigh how many players complain and who they are. Frustrated players speak up far more than players with no opinion
Mark Brown, Game Maker's Toolkit5:32.
- Remember that vocal critics tend to want games harder and more complex, which can alienate casual fans who never visit forums.
- Steam forum anger over DOOM Eternal's Marauder is one example of a hatred that can look bigger than it is.
- Act when different sources line up. A trend across sources is stronger than one loud voice Dana Nightingale and David Di Giacomo, Game Maker's Toolkit12:59.
- Dishonored 2's original clockwork soldier shipped in early trailers for a long time. Feedback said it looked goofy, not intimidating, so the design changed.
- Rank the problems and work from the worst. Score each issue from 1 (trivial visual glitch) to 5 (game-breaking) in a spreadsheet and sort it Mark Brown, Game Maker's Toolkit3:47.
- This keeps limited energy on what hurts the player most.
- Mind Over Magnet had almost 100 reported issues.
- Its MVP feedback became a checklist: shaky controls, mixed platforming and puzzles, few aha moments, and no sense of being built around the magnet. Demo 2.0 targeted all of it Mark Brown, Game Maker's Toolkit11:22.
- Filter feedback through your goal. Bending to every tester gives design-by-committee sludge Mark Brown, Game Maker's Toolkit14:53.
- Go in with a specific game and audience, then test whether you hit that goal.
- Rare's Sean Davies warns that responding to all feedback converges on a middle of the road game that surprises no one
Mark Brown, Game Maker's Toolkit7:07.
2.2 Turn the suggestion into a symptom
A complaint shows where something feels wrong, and your job is to work out why.
- Treat a proposed fix like "nerf this" or "remove him" as a symptom. Players cannot foresee the knock-on effects
Mark Brown, Game Maker's Toolkit8:25.
- Ask specific questions and drill down to the root problem, then find a solution that works.
- Edmund McMillen says a complaint about controls could mean one of hundreds of things.
- Use feedback to generate questions for yourself. Nioh director Fumihiko Yasuda argued the complaint shows where it feels wrong, and the designer works out why and how to fix it
Mark Brown, Game Maker's Toolkit9:06. - Look past the reported symptom to the real cause. The symptom is often not the issue
Mark Brown, Game Maker's Toolkit1:47.
- On Dying Light, the director said weapons broke too fast. Lead designer Maciej Binkowski saw that players could kill only a few zombies per weapon, so he lowered zombie health.
- That left durability and the economy untouched, and basic combat felt better too.
- Make sure the whole team agrees on what the problem is. Designers can argue for solutions while describing different problems
Mark Brown, Game Maker's Toolkit2:57.
- On Astroneer, one designer saw crafting as too simple, from a systems view. Another saw it as too fiddly, moment to moment.
2.3 Watch what players do, not only what they say
Behavior and post-play comments often name a problem that the suggestion misses.
- Treat manual workarounds as a sign of missing information. Then ask whether the workaround is tedious busywork or part of the intended experience
Mark Brown, Game Maker's Toolkit6:27.
- Balatro playtesters tracked played cards by hand. After polling users, LocalThunk added a deck peek.
- Left 4 Dead testers struggled to find teammates in trouble, which led to survivor outlines. Half-Life testers broke every crate, so Valve stuffed some with ammo and health Mark Brown, Game Maker's Toolkit6:59.
- Compare how different players behave to find the fix. Gears of War's active reload helped experts, but novices were punished
Mark Brown, Game Maker's Toolkit11:57.
- Epic noticed experts reload before the clip is empty while novices shoot until it is dry.
- They secretly boosted the last few bullets in a clip, so novices got a similar advantage automatically.
- Interview testers after play. Talking can expose what watching alone might not have named Mark Brown, Game Maker's Toolkit0:06.
- Portal testers said it was a great tutorial and they could not wait to play the real game. Valve added GLaDOS to give motivation and frame the puzzles as training.
2.4 Choose the fix that treats the cause
The best fix often lives in the design, the level order or another system, not in a patch or a hint.
- Revise the design before adding spoken hints. When many testers get stuck at the same spot, change the layout, visual clarity or camera Mark Brown, Game Maker's Toolkit2:21.
- This costs more once art is finished, but it solves the confusion instead of covering it.
- In Metroid Prime, testers missed morphball tunnels, so Retro added spotlights over the entrances.
- Portal's cluttered rooms hid which objects mattered. One player spent half an hour pushing a shelf onto a button, so Valve moved to sterile white walls Mark Brown, Game Maker's Toolkit2:36.
- Change the level order when the problem is what the player knows. Sometimes the level is fine and the player's knowledge at that point is not Mark Brown, Game Maker's Toolkit2:25.
- Moving objects and adding a hint lump failed in Mind Over Magnet. Putting a ride-up-twice level and a too-small-gap level first got playtesters through.
- Move the feature to another system when the UI is the problem. The fix can live somewhere else
Mark Brown, Game Maker's Toolkit9:38.
- The Last of Us had an inventory upgrade tab that cluttered the UI and led to buying the cheapest upgrade at once. Workbenches in the world made players save and plan.
- Hold out for a fix that solves several problems. Miyamoto's view is that a good idea solves multiple problems at once
Mark Brown, Game Maker's Toolkit11:10.
- New Super Mario Bros. Wii returns dead players in a bubble their teammates pop. Struggling novices can also bubble themselves on purpose.
- Shovel Knight reversed its pay-to-save checkpoints. They became free and automatic, and you earn money by breaking one, which removed the novice penalty and kept the risk for experts
Mark Brown, Game Maker's Toolkit8:47.
2.5 Check the fix against the intended experience
A fix that removes friction can also remove the reason the feature existed.
- Ask what the friction is for before you remove it. A bad-looking menu may be a deliberate lever Mark Brown, Game Maker's Toolkit17:09.
- Echoes of Wisdom's 120-plus echo list is meant to surface forgotten echoes while you scroll. Co-director Satoshi Terada says it is designed to make you less efficient but more creative.
- DOOM 3 forced a choice between torch and gun. Fans asked for both, and the BFG Edition delivered, arguably spoiling the survival horror theme
Mark Brown, Game Maker's Toolkit11:35.
- Reject a usability fix that breaks the design intent. A favourites ring would be faster, but it lets players lean on a few creatures Mark Brown, Game Maker's Toolkit16:14.
- Look for a design that keeps every echo available instead.
- Keep a mechanic's purpose when players demand its removal. Redesign the mechanic instead
Mark Brown, Game Maker's Toolkit8:36.
- Players asked Team Ninja to remove item durability in the Nioh alpha. It existed to stop one-weapon play, so the team turned it into item familiarity, which still pushes players to swap gear.
2.6 Retest each fix for side effects
Every fix is a new change, so test it as you would any other.
- Run several rounds of QA after fixing. Keep going until testers stop finding new problems Mark Brown, Game Maker's Toolkit4:34.
- Some Mind Over Magnet bugs looked solved, but the fix had created a bug somewhere else in the game.
- Check second-order effects of balance and economy fixes. Fixing a problem in one place can create another elsewhere
Mark Brown, Game Maker's Toolkit12:59.
- Ubisoft nerfed the Rainbow Six Siege shotgun, but it is a key defensive weapon, so defenders could no longer hold attackers back. Shortening the round time restored balance.
- The Witcher 3's cow exploit was first fixed by spawning a level 27 monster after six cows died. Higher-level players farmed its head for profit, so the monster now spawns only once Mark Brown, Game Maker's Toolkit15:15.
- Retest with playtesters without telling them what changed. Telling them how you fixed it biases their experience
Mark Brown, Game Maker's Toolkit14:26.
- If the fix fails, use what you learned to reframe the problem and start again.
- Keep replaying and keep watching after launch. More plays of a level before shipping usually make it better Mark Brown, Game Maker's Toolkit16:13.
- A Hitman (2016) Paris section was fixed because Viktor Novikov was alone for too long and became an easy kill.
- Steam data showed many players stuck in Half-Life 2: Episode One, so Valve patched a tricky siege battle to lower its difficulty Mark Brown, Game Maker's Toolkit7:57.
2.7 Where designers disagree
How far to follow player complaints depends on whether the complaint hits a real flaw or the game's identity.
- Fix what players hate, or hold the line. Some complaints point at real flaws, and others at what makes the game distinctive.
- Gods Will Be Watching launched at 20% positive reviews. Its Mercy Update fixed difficulty and randomness problems and lifted reviews to 70% positive
Mark Brown, Game Maker's Toolkit2:50. - Resident Evil 3's Nemesis was limited to boss fights and set pieces, possibly in response to stress complaints about Mr. X, and critics found him neutered. DUSK programmer Dillon Rogers argues devs must not let fans ruin their games
Mark Brown, Game Maker's Toolkit9:26. - Respawn's David Bocek says that at some point a designer has to be brave, or nothing interesting ships
Mark Brown, Game Maker's Toolkit12:06.
- Gods Will Be Watching launched at 20% positive reviews. Its Mercy Update fixed difficulty and randomness problems and lifted reviews to 70% positive
- Make it easier, or keep the challenge. Tedium is a different problem from difficulty.
- Patrick's Parabox maker Patrick Traynor advised simplifying levels even if that made the game easier. Mark ignored him until his dad's playtest, then found that easing off was right and he had not gone far enough Mark Brown, Game Maker's Toolkit10:20.
- Mark also cut levels that relied on quick reflexes, and one with an exploit he could not easily fix Mark Brown, Game Maker's Toolkit4:06.
- Explain your reasons, or frame players as helping your vision. Both aim to keep players on side without handing over the design.
- Chris Avellone found that frankness and a structured argument for a feature created buy-in. Warframe uses weekly streams and designers on forums to keep dialogue open
Mark Brown, Game Maker's Toolkit12:27. - After the Darkest Dungeon corpse backlash, Red Hook made corpses optional and changed its message to "help us make the game we're aiming for"
Mark Brown, Game Maker's Toolkit15:06. - Designer Tyler Sigman admits the studio could have communicated the change better
Mark Brown, Game Maker's Toolkit13:47.
- Chris Avellone found that frankness and a structured argument for a feature created buy-in. Warframe uses weekly streams and designers on forums to keep dialogue open
Sources
Part of Prototyping, testing and accessibility in the growth library.