CaddieOpen in CaddieDownload for Mac

How to turn player feedback into the right fix

Shared by Caddie Library · Updated
Chapters show their main points
Chapter 26 of 26 · Prototyping, testing and accessibility
← The growth course
1. Finding Product-Market Fit
Test for real customer pull earlyPick a narrow niche to start inLaunch a simple first version fastLearn from users and customersRead early signs of product-market fit
2. Onboarding and Activation
Structure onboarding as a storyAsk and use onboarding questionsGet users to the aha moment fastDecide how long onboarding should beBuild quiz funnels for paid traffic
3. Paywalls and Pricing
When to choose a hard paywall over freemiumDesign a paywall that convertsPrice annual and monthly plansSet, raise and localize pricesRun free trials and free tastes
4. Business Models and Revenue
Structure subscription tiersKeep a free tier that pays offCharge for usage and AI creditsPair hardware with a subscriptionAdd ads, affiliates and partners
5. Retention and Engagement
Measure retention honestlyBuild habit loops that bring users backUse notifications and lifecycle messagesReduce churn and handle cancellationsBuild social loops and community
6. Brand and Positioning
Position against bigger incumbentsName and describe your productEarn trust in high-stakes categoriesBorrow credibility from celebrities and experts
7. Organic Content and Creators
Find, vet and recruit creatorsPay creators fairly and safelyBrief and manage creatorsFind and copy viral formatsScale one format across many accountsRank in AI search and SEO
8. Paid, App Store and Viral Growth
Make ads that convertTurn organic winners into paid adsStructure and scale Meta campaignsTell if paid acquisition is profitableWin in the App StoreBuild viral and referral loops
9. Experimentation and Analytics
Run trustworthy experimentsJudge test results correctlyRun more experiments fasterWhat metrics to track and how to define them
10. Building AI Products
Build for the models of tomorrowDesign AI interfaces beyond chatDesign AI agents people can trustMeasure AI quality with evalsControl AI costs and pricingBuild products with AI coding tools
11. Teams, Hiring and Funding
Hire people who thrive on ambiguityOrganize small product teamsRun exploratory bets and kill themWork as a team with AI agentsWhen to raise money or sell your company
12. Founder Principles
Pick a mission worth building a company aroundWhy to trust your own judgment over the consensusTurn failure and setbacks into the next stepLearn from rivals and copy what works
13. Choosing Where to Compete
Make low cost your competitive advantageCreate a market instead of fighting for shareWhy scarcity and brand can beat volumeBuild an advantage that feeds itselfAttack a bigger rival where it cannot followWhen to stay focused on the core business
14. Building and Leading the Company
Hire great people and give them roomAlign pay and ownership with the missionKeep your culture and standards as you growBuild candor and fast decision-makingWhy leaders must stay close to the work and customersChoose and prepare a successor
15. Capital and Deals
Fund growth without losing controlAcquire companies and keep what makes them workWhat to do with cash: reinvest, buy back or holdWhy to keep a cash cushion and avoid heavy debtStructure ownership to protect your missionNegotiate deals that keep your leverage
16. Finding Ideas and Thinking
How to find ideas worth writing aboutWhy writing is how you find out what you thinkHow to research so the writing carries feelingHow to read like a writer
17. Structure and Argument
How to open so readers keep goingHow to find the shape before you draftHow to keep readers curious with questions and cluesHow to argue without preachingHow to end so the last page lingers
18. Clear Sentences and Voice
How to write plainly for a reader who knows nothingHow to choose specific details over generic onesHow to sound like yourself on the pageHow to shape rhythm and phrases people remember
19. Characters and Storytelling
How to build characters readers care aboutWhat makes a scene move: want, obstacle and changeHow to write dialogue that sounds real but stays shortHow to make readers feel without telling themHow to turn information into a story
20. Revising, Habits and Readers
How to cut and rewrite your own draftHow to get feedback you can useHow to build a writing routine that lastsHow to get unstuck when a draft stallsHow to use AI without losing your voiceHow to write for one specific reader
21. Why people play games
How to design around the feeling you wantHow to find a game idea with a hookHow to decide who your game is forWhy rewards and motivation need careful designHow to use mystery and curiosity to pull players on
22. Core mechanics and loops
How to build a game around one core verbHow to give each mechanic several jobsHow to add depth with holds, timing and chainsHow to use feedback loops to keep games balancedHow to use randomness without frustrating playersHow to make synergies feel powerful and earned
23. Challenge, difficulty and feel
How to make hard games feel fairHow to tune difficulty with many leversHow to offer assists without breaking your intentHow to make death and retries painlessHow to make movement and controls feel goodHow to add camera and juice that sell impact
24. Teaching players and designing levels
How to teach mechanics without a tutorialHow to introduce mechanics one at a timeHow to guide players without markersHow to build levels around one mechanicHow to design puzzles with a catch and a revelationHow to design dungeons from locks and keys
25. Progression, systems and story
How to gate exploration without trapping playersHow to design unlocks and skill treesHow to let systems create emergent playHow to design enemy AI players can readHow to tell stories through mechanicsHow to make player choices and consequences matter
26. Prototyping, testing and accessibility
How to prototype before you commitHow to scope and cut a game to finish itHow to run playtests that reveal real problemsHow to turn player feedback into the right fixHow to build accessibility in from the start

1 The bottom line

Treat every suggestion as a symptom, find the root cause, and retest each fix for side effects. Players report where it hurts, not what is wrong, and fixes often break something else.

2 Key takeaways

  • Treat a suggested fix as a symptom, not an instruction. Players see the pain, but not the cause or the knock-on effects.
  • Filter feedback through your goal and through who is speaking. Loud voices skew the picture, and answering everything makes a forgettable game.
  • Ask what a bad-looking feature is for before you remove it. Friction can be carrying the design.
  • Hold out for a fix that solves several problems at once. Narrow patches tend to leave the cause in place.
  • Retest after every fix. A fix can create a new problem somewhere else.

When responding to feedback, ask specific questions and try to find the root of the problem

Mark Brown, Game Maker's Toolkit

2.1 Decide whether the feedback deserves action

Before you fix anything, check how many people said it, who they are, and whether it serves your goal.

  • Weigh how many players complain and who they are. Frustrated players speak up far more than players with no opinion Mark Brown, Game Maker's Toolkit5:32.
    • Remember that vocal critics tend to want games harder and more complex, which can alienate casual fans who never visit forums.
    • Steam forum anger over DOOM Eternal's Marauder is one example of a hatred that can look bigger than it is.
  • Act when different sources line up. A trend across sources is stronger than one loud voice Dana Nightingale and David Di Giacomo, Game Maker's Toolkit12:59.
    • Dishonored 2's original clockwork soldier shipped in early trailers for a long time. Feedback said it looked goofy, not intimidating, so the design changed.
  • Rank the problems and work from the worst. Score each issue from 1 (trivial visual glitch) to 5 (game-breaking) in a spreadsheet and sort it Mark Brown, Game Maker's Toolkit3:47.
    • This keeps limited energy on what hurts the player most.
    • Mind Over Magnet had almost 100 reported issues.
    • Its MVP feedback became a checklist: shaky controls, mixed platforming and puzzles, few aha moments, and no sense of being built around the magnet. Demo 2.0 targeted all of it Mark Brown, Game Maker's Toolkit11:22.
  • Filter feedback through your goal. Bending to every tester gives design-by-committee sludge Mark Brown, Game Maker's Toolkit14:53.
    • Go in with a specific game and audience, then test whether you hit that goal.
    • Rare's Sean Davies warns that responding to all feedback converges on a middle of the road game that surprises no one Mark Brown, Game Maker's Toolkit7:07.

2.2 Turn the suggestion into a symptom

The complaint names the pain, not the cause
On Dying Light, the fix landed on the zombies, not on the weapons players complained about.

A complaint shows where something feels wrong, and your job is to work out why.

  • Treat a proposed fix like "nerf this" or "remove him" as a symptom. Players cannot foresee the knock-on effects Mark Brown, Game Maker's Toolkit8:25.
    • Ask specific questions and drill down to the root problem, then find a solution that works.
    • Edmund McMillen says a complaint about controls could mean one of hundreds of things.
  • Use feedback to generate questions for yourself. Nioh director Fumihiko Yasuda argued the complaint shows where it feels wrong, and the designer works out why and how to fix it Mark Brown, Game Maker's Toolkit9:06.
  • Look past the reported symptom to the real cause. The symptom is often not the issue Mark Brown, Game Maker's Toolkit1:47.
    • On Dying Light, the director said weapons broke too fast. Lead designer Maciej Binkowski saw that players could kill only a few zombies per weapon, so he lowered zombie health.
    • That left durability and the economy untouched, and basic combat felt better too.
  • Make sure the whole team agrees on what the problem is. Designers can argue for solutions while describing different problems Mark Brown, Game Maker's Toolkit2:57.
    • On Astroneer, one designer saw crafting as too simple, from a systems view. Another saw it as too fiddly, moment to moment.

2.3 Watch what players do, not only what they say

Behavior and post-play comments often name a problem that the suggestion misses.

  • Treat manual workarounds as a sign of missing information. Then ask whether the workaround is tedious busywork or part of the intended experience Mark Brown, Game Maker's Toolkit6:27.
    • Balatro playtesters tracked played cards by hand. After polling users, LocalThunk added a deck peek.
    • Left 4 Dead testers struggled to find teammates in trouble, which led to survivor outlines. Half-Life testers broke every crate, so Valve stuffed some with ammo and health Mark Brown, Game Maker's Toolkit6:59.
  • Compare how different players behave to find the fix. Gears of War's active reload helped experts, but novices were punished Mark Brown, Game Maker's Toolkit11:57.
    • Epic noticed experts reload before the clip is empty while novices shoot until it is dry.
    • They secretly boosted the last few bullets in a clip, so novices got a similar advantage automatically.
  • Interview testers after play. Talking can expose what watching alone might not have named Mark Brown, Game Maker's Toolkit0:06.
    • Portal testers said it was a great tutorial and they could not wait to play the real game. Valve added GLaDOS to give motivation and frame the puzzles as training.

2.4 Choose the fix that treats the cause

The best fix often lives in the design, the level order or another system, not in a patch or a hint.

  • Revise the design before adding spoken hints. When many testers get stuck at the same spot, change the layout, visual clarity or camera Mark Brown, Game Maker's Toolkit2:21.
    • This costs more once art is finished, but it solves the confusion instead of covering it.
    • In Metroid Prime, testers missed morphball tunnels, so Retro added spotlights over the entrances.
    • Portal's cluttered rooms hid which objects mattered. One player spent half an hour pushing a shelf onto a button, so Valve moved to sterile white walls Mark Brown, Game Maker's Toolkit2:36.
  • Change the level order when the problem is what the player knows. Sometimes the level is fine and the player's knowledge at that point is not Mark Brown, Game Maker's Toolkit2:25.
    • Moving objects and adding a hint lump failed in Mind Over Magnet. Putting a ride-up-twice level and a too-small-gap level first got playtesters through.
  • Move the feature to another system when the UI is the problem. The fix can live somewhere else Mark Brown, Game Maker's Toolkit9:38.
    • The Last of Us had an inventory upgrade tab that cluttered the UI and led to buying the cheapest upgrade at once. Workbenches in the world made players save and plan.
  • Hold out for a fix that solves several problems. Miyamoto's view is that a good idea solves multiple problems at once Mark Brown, Game Maker's Toolkit11:10.
    • New Super Mario Bros. Wii returns dead players in a bubble their teammates pop. Struggling novices can also bubble themselves on purpose.
    • Shovel Knight reversed its pay-to-save checkpoints. They became free and automatic, and you earn money by breaking one, which removed the novice penalty and kept the risk for experts Mark Brown, Game Maker's Toolkit8:47.

2.5 Check the fix against the intended experience

A fix that removes friction can also remove the reason the feature existed.

  • Ask what the friction is for before you remove it. A bad-looking menu may be a deliberate lever Mark Brown, Game Maker's Toolkit17:09.
    • Echoes of Wisdom's 120-plus echo list is meant to surface forgotten echoes while you scroll. Co-director Satoshi Terada says it is designed to make you less efficient but more creative.
    • DOOM 3 forced a choice between torch and gun. Fans asked for both, and the BFG Edition delivered, arguably spoiling the survival horror theme Mark Brown, Game Maker's Toolkit11:35.
  • Reject a usability fix that breaks the design intent. A favourites ring would be faster, but it lets players lean on a few creatures Mark Brown, Game Maker's Toolkit16:14.
    • Look for a design that keeps every echo available instead.
  • Keep a mechanic's purpose when players demand its removal. Redesign the mechanic instead Mark Brown, Game Maker's Toolkit8:36.
    • Players asked Team Ninja to remove item durability in the Nioh alpha. It existed to stop one-weapon play, so the team turned it into item familiarity, which still pushes players to swap gear.

2.6 Retest each fix for side effects

Every fix can create a new problem elsewhere
Test each fix like any other change, and keep going until testers stop finding new problems.

Every fix is a new change, so test it as you would any other.

  • Run several rounds of QA after fixing. Keep going until testers stop finding new problems Mark Brown, Game Maker's Toolkit4:34.
    • Some Mind Over Magnet bugs looked solved, but the fix had created a bug somewhere else in the game.
  • Check second-order effects of balance and economy fixes. Fixing a problem in one place can create another elsewhere Mark Brown, Game Maker's Toolkit12:59.
    • Ubisoft nerfed the Rainbow Six Siege shotgun, but it is a key defensive weapon, so defenders could no longer hold attackers back. Shortening the round time restored balance.
    • The Witcher 3's cow exploit was first fixed by spawning a level 27 monster after six cows died. Higher-level players farmed its head for profit, so the monster now spawns only once Mark Brown, Game Maker's Toolkit15:15.
  • Retest with playtesters without telling them what changed. Telling them how you fixed it biases their experience Mark Brown, Game Maker's Toolkit14:26.
    • If the fix fails, use what you learned to reframe the problem and start again.
  • Keep replaying and keep watching after launch. More plays of a level before shipping usually make it better Mark Brown, Game Maker's Toolkit16:13.
    • A Hitman (2016) Paris section was fixed because Viktor Novikov was alone for too long and became an easy kill.
    • Steam data showed many players stuck in Half-Life 2: Episode One, so Valve patched a tricky siege battle to lower its difficulty Mark Brown, Game Maker's Toolkit7:57.

2.7 Where designers disagree

Fixing real flaws lifted reviews from 20% to 70%
At launch20%
After Mercy Update70%
Some complaints point at real flaws rather than the game's identity, and those are worth fixing.

How far to follow player complaints depends on whether the complaint hits a real flaw or the game's identity.

  • Fix what players hate, or hold the line. Some complaints point at real flaws, and others at what makes the game distinctive.
    • Gods Will Be Watching launched at 20% positive reviews. Its Mercy Update fixed difficulty and randomness problems and lifted reviews to 70% positive Mark Brown, Game Maker's Toolkit2:50.
    • Resident Evil 3's Nemesis was limited to boss fights and set pieces, possibly in response to stress complaints about Mr. X, and critics found him neutered. DUSK programmer Dillon Rogers argues devs must not let fans ruin their games Mark Brown, Game Maker's Toolkit9:26.
    • Respawn's David Bocek says that at some point a designer has to be brave, or nothing interesting ships Mark Brown, Game Maker's Toolkit12:06.
  • Make it easier, or keep the challenge. Tedium is a different problem from difficulty.
    • Patrick's Parabox maker Patrick Traynor advised simplifying levels even if that made the game easier. Mark ignored him until his dad's playtest, then found that easing off was right and he had not gone far enough Mark Brown, Game Maker's Toolkit10:20.
    • Mark also cut levels that relied on quick reflexes, and one with an exploit he could not easily fix Mark Brown, Game Maker's Toolkit4:06.
  • Explain your reasons, or frame players as helping your vision. Both aim to keep players on side without handing over the design.

Sources

Part of Prototyping, testing and accessibility in the growth library.