I’ve been thinking a bit about feedback/reinforcement loops lately, so I figured I would write up a post on them.
Positive feedback loops in games are mechanics that reinforce the success or failure of the player and make future successes/failures more probable. This can be seen in numerous games, especially in RPGs. The cliche often used to explain positive feedback loops is “the rich get richer”. Having money is often a prerequisite to gaining more money: you need capital to start a business, so the ones making money are the ones already rich. And so on.
But let’s take an example from board games first. In Risk, players control armies trying to take over the world. They do so by battling other armies. When players win battles and control entire continents, they receive additional armies every turn – this in turn “feeds back” into the simple result that having more armies makes you more likely to win battles. So not only does winning a battle give a direct advantage (the opponent has fewer units) but you also feed back that victory into gaining more units, which should ensure further victories.
Quake II faces positive feedback loops. The relevant mechanic is that when you die, you restart with guns that are not as good as the ones that can be picked up on the battlefield. The player with the first kill then has an inherent advantage – he will have better weapons than the newly respawned player and will be more likely to kill him again, given equal skill.
Professional baseball in America has a somewhat muted positive feedback loop. There are no salary caps in baseball, so teams can spend as much money as they have to procure the best players. The best players (one would think) lead to the best success on the field. Success on the field leads to more money as people bid up tickets and buy merchandise. That money can then be fed into buying even better players.
The Facebook game Mafia Wars originally had a pretty severe positive feedback problem. Players could buy properties that were constantly earning rents. Players could dump all of their money into buying “Mega-Casinos” which in turn gave a lot of additional money in rents, for which players could buy additional “Mega-Casinos”. After my first burst of playing the game, I quit. I came back three months later and had hundreds of billions of dollars, enough to buy any object in the game (that could be bought with in-game money, that is).
Positive feedback isn’t always bad. For one, it helps direct player actions. Players want to become richer, more powerful, &c., so giving them rewards for succeeding makes sense. It also breaks stalemates. Consider a game like the card game War. In it, the player with the highest card value wins. It is entirely random, so on average, each player should win 50% of the time. Series of War games will always be in stalemate. No previous game affects the current game. But if you added a positive feedback loop that changed that, the game could progress towards an end state. Say that players get to turn over an additional card if they won the last hand. This gives the winning player an advantage, that should cause a positive feedback loop leading to an end-game state where one player has dominance over the other and can be concluded the winner. I’m not saying that will make War fun (Lord knows), but it can end the damn thing.
When I was assigned to NCAA Football 08, there was a positive reinforcement system being implemented called “DPR” – dynamic player ratings. The idea was to implement the most straightforward of positive reinforcements. Player events that were tied to random die rolls were affected by ability ratings (a player with 90 catching ability would catch more often than a player with 89 catching rating), as a player succeeded during the course of a single game, his ratings would be temporarily boosted, causing more successes. Player ratings converged to either essential perfection or complete inadequacy. That system had to be tweaked the entire cycle, the smallest change could spiral out of control creating perfect passers or punters that couldn’t hit the broad side of a barn. In the end, it was tuned down to look significant, but the probabilities affected were only slightly nudged. The essential nature of the positive feedback dynamic was never addressed.
This problem is endemic to many sports games as “stat boosts” seem to be the only reward structure worth pursuing.
Negative feedback loops in games are mechanics that cause to hinder succeeding players from further success or failing players from further failures. The classic example is Mario Kart.
Mario Kart is a racing game where players have weapons that they can use to speed themselves up or hinder the progress of their opponents. In a later installment, Nintendo added the Blue Shell weapon. The Blue Shell is the ultimate negative feedback loop. It homes in and destroys the player in first place no matter where they are on the map. There is nothing the first place player can do to avoid this. The blue shell is almost inevitable – there is almost one in every race. It is always on the mind of the player in first. Succeeding players are punished – no one wants to be too far ahead of the pack because the blue shell will knock them back. Instead, it encourages a dynamic where players want to be in second place until the final parts of the race – safe from blue shells but also close enough to take the lead by the finish.
Negative feedback is dangerous because it sends mixed signals. We direct our players to succeed – win the race, kill the bad guys &c., but at the same time our negative feedback mechanics do not agree with our stated intent.
Some racing games suffer from this problem. Since most racing games are more interesting when you are jockeying for position with other cars, if you do too well and leave the pack behind, the AI will cheat and make the opponents go supernaturally fast to catch up with player to encourage more nail-biting racing. This punishes the player for succeeding, which goes against all the other mechanics in many of these games which are tailored to encourage the player to race well. Players are almost unanimously against this technique – do a search in racing game reviews for “rubber-band AI” and you will see what I mean. Designers here are faced with a conundrum – have no “rubber banding” and let good players race off into the distance, effectively racing time trails, or have the AI cheat to make the race more exciting.
Even relatively simple games can use negative feedback. In the trivia game Buzz, the player in first place never gets to select the trivia category. The player selecting should naturally try to pick something that she knows that the leader doesn’t and thus this should serve as negative reinforcement.
Positive feedback loops are naturally occurring for a simple reason: players direct themselves towards actions that make them stronger. They buy swords in RPGs that let them kill bigger creatures that drop better loot that allow them to buy even better swords. There’s nothing inherently wrong with that, but it can cause problems. Negative feedback loops are much harder to come by because they can be simply contradictory and non-intuitive. We guide players to help them succeed and then behind the scenes sabotage them with negative feedback so that succeeding is bad. We spend more time figuring out solutions to positive feedback situations than negative because positive loops cause more problems:
- Interest curves. Even single player games can fall victim to this. What if the player spends all his time trying to level up instead of exploring all the great content we have for him? What if the drops he gets from such leveling up make the rest of the game too easy? How can we reward the player with one hand and take it away with the other such that he keeps progressing through the game?
- Problems of balance. Multiplayer games need to use positive feedback sparingly. If the entire game is decided by the time the first winning move enters the positive feedback loop (like the first kill in Quake II above), then why play the rest of the game? The loser will be continually beaten by the first winner, and without hope of winning himself, probably won’t be having much fun.
- Endgame problems. World of Warcraft has a problem. Players get stronger and stronger, but there is a finite amount of content they can offer. Eventually, you have to reach the strongest sword in the game. At that point, players cannot be directed to kill bigger bad guys for the purpose of better drops that have better stats. The designers there have to deal with the problem of “what to give the player that has everything”. Naturally, they’ve done a good job as they have a healthy base of players at the maximum level, but this problem only exists because of the positive feedback loops in the main gameplay.
Disgaea took a different approach to this. They simply scale up the numbers for weapons and monsters to ridiculous levels, encouraging an ever-moving treadmill of progression-escalation. World of Warcraft does not have this luxury because they have to deal with players of uneven levels cooperating in the same area. It is hard enough to do that for a Level 80 and a Level 70, let alone a Level 200 and a Level 70.
One technique to fix positive feedback getting out of control is to decouple what the reward affects and what the tasks require. Farmville and other “social” games are all about walking up a treadmill to get the next doo-dad. But the doo-dads in these games generally aren’t things that help the player succeed. Often they are cosmetic or tangential. For instance, while the harvester in Farmville allows the player to harvest in less clicks, it doesn’t make the plants grow faster. The reward is reinforcing the click-to-harvest mechanic and not the XP-gaining growth mechanic. Thus, players can lust over the harvester and harvest like silly without breaking the game. This is a fine technique and one that I recommend, but it isn’t perfect because the player has to want a reward of something other than what will give him more power to achieve. Thus it can only be applied in some situations.
When I was working on the multiplayer for Superman Returns DS, we started by drafting a pretty clever multiplayer board game where players attempted to take control of the city of Metropolis. Quickly, we fell into problems of positive feedback loops – players who controlled more of the city could move around the city easier and thus could get the bonuses that helped the players towards victory. We could have gimped it by removing the positive feedback from ease of movement around territory you control, but we chose a different method. We added a negative feedback mechanic- taking over an opponents areas gains someone more territory than taking over neutral areas. I don’t remember if this stayed in the final version or not, but since no one actually played the multiplayer of Superman Returns, even though it was far and away the best part, I’ll just make that claim and leave it at that.
This is often the best solution if the positive reinforcement is knocking the game out of balance and you cannot simply remove the offending mechanic – pair positive reinforcements with negative ones.
RPGs do this: When you level up, monsters also become tougher. So you get stronger, but your enemies do as well, likely at the same rate. Your power increase is positive feedback. Your enemy’s increase is negative feedback. This would be noticeable to players if put that explicitly, so what we do is allow the player to really beat up on some lower-level thugs after powering-up to get them feeling more invincible before throwing their ass in front of newly souped-up baddies.
But this has to be carefully balanced. In Morrowind, players could choose to level up and if they did so, stronger baddies would appear instantly, rolled from a different table. When I first played it, I stayed as a level Suck player for as long as I could, simply because the quest items I found were positively reinforcing and leveling up was a net-negative reinforcement that was under my control. Certainly, it was not what the designers intended but I’m sure many players chose that path.
“Fixing” positive feedback doesn’t even have to really fix the problem, only the appearance of a problem. Consider our Major League Baseball example above. The National Football League saw that positive feedback problem and added negative reinforcements – salary caps (limits on how much a team can spend on personnel) and reverse draft selection (the worst team gets to choose first the best players coming out of the collegiate system.) It is generally seen as more fair. In baseball, the top four teams (Yankees, Dodgers, Cardinals, Giants) combine for 50% (53/105) of all world championships. In football, the four winningest Super Bowl teams (Steelers, Cowboys, 49ers, Patriots) combine for 43% (19/44) of all Super Bowls. While football is widely considered to be more egalitarian, the statistical difference is slight.
EDIT: And I almost forgot to mention one of the most clever negative reinforcements in all of the games I’ve experienced. In Dominion, the act of buying victory points and putting them into your deck de facto makes your deck weaker since victory cards are empty draws. This means that you are always stuck with the decision of making an efficiency advantage at the cost of points versus a points advantage at the cost of efficiency.