Rebalancing a Live Game: How I Broke Veteran Players Twice and Why the Test Was Green

On Saturday a player of my idle tower game emailed me. In short: “Thanks for the update, but now I have millions of HP and no damage, I can’t beat a single enemy.”

The worst word in that email is “update”. Two days earlier the same player had complained about the opposite: after a big rebalance his upgraded tower died on the first wave. I shipped a fix, wrote a test, the test passed. And the fix broke the game again, just in the other direction.

This is how that happened. Not about balance formulas, but about how to test changing numbers in a game where people already have progress.

A rebalance is a data migration, not a config edit

In version 0.10.0 I rebuilt the progression after a reference game. Enemies on high difficulty levels (tiers) became much stronger. On tier 10 enemy health went from a ×2800 multiplier to ×40320, 14 times more. Attack grew about 5 times.

For a new player that’s fine: they start from zero and upgrade for the new numbers. But a veteran already has tier 10 unlocked. Their tower is upgraded for the old numbers. Nobody touched the save, but the world around it became 14 times tougher.

The key point I missed: a rebalance changes the meaning of data that is already saved. “Damage level 75” in a save used to mean “enough for tier 10” and now means “not enough even for tier 5”. It’s the same kind of migration as renaming a database column. You just don’t see it in the code: the save code didn’t change by a single line.

So the QA question for any rebalance is not “are the new numbers right” but “what happens to people who arrive with the old ones”.

First break: the tower dies on wave one

The first symptom was obvious. A tier 10 player starts a run and dies in seconds. Enemies hit 5 times harder, the tower has the same health.

I decided to give such saves their “tier armour” back: a multiplier that in the old balance raised the tower’s health, regen and block on high tiers. A save is flagged as “old” if its tiers were unlocked before 0.10.0, and the armour is turned back on for it.

I wrote a test: take a real tier 10 save from an old version, run the battle simulation, check that the tower survives to wave 20. It used to die on wave one, now it survives. Green. Release.

Second break: millions of HP and no kills

On tier 10 the armour works out to roughly ×2200 health. The tower really stopped dying. But armour doesn’t touch damage. And enemy health had grown 14 times.

The result was a tower with a huge health pool that could barely kill anything. Enemies don’t die, new ones arrive, there are more and more of them. That’s exactly what the player described: “millions of HP but no damage”.

Why didn’t the test catch it? Because in my game the wave number grows on a timer. A wave lasts a fixed time, then the next one starts, whether you killed anything or not. The test “survive to wave 20” checked that the tower didn’t die in 20 time slices. It didn’t check that the tower was actually playing.

When I rewrote the check to count kills, the picture was obvious:

Version 0.10.1 (armour only): on tier 10 at wave 10 there are 106 enemies alive on the field, by wave 30 there are 200, the game’s limit. In 62 waves the tower killed 311 enemies.
The same save on tier 1: 0 enemies alive at wave 10, 4166 kills in 115 waves.

The test measured a proxy (“the wave went up”) and treated it as the outcome (“the player is playing normally”). In a working game those two match. In a broken one they split, and that’s exactly where you need the test.

How I tested the second time

The second time I started not with a fix but with a table. I took the same old save and ran the simulation on every tier from 1 to 10, in two modes: with in-run upgrade purchases and without. For each run I recorded not the wave number but:

how many enemies were killed by waves 3, 10 and 30;
how many enemies were alive on the field at the same moments, which is the backlog;
on which wave and why the run ended.

And I compared four fix variants:

Armour only (as in 0.10.1). The tower doesn’t die, but enemies pile up. Backlog from wave 10.
Old enemy stats per tier, no armour. Enemies can be killed again, but the tower dies on tiers 8–10 by waves 10–15: the new balance also changed the wave curves.
Old stats plus weak armour. A bit better, but tiers 7–10 still die early.
Old stats plus the old armour. At wave 10 there are 0 enemies alive on every tier. A backlog only appears by waves 30–50 on the top tiers, which is the natural ceiling of the upgrades, not a bug.

The choice became obvious only when I saw all four variants side by side, measured by a metric that reflects what the player feels.

Two more things turned up along the way:

The armour was also applied in tournaments. A tournament is a competition, and a veteran shouldn’t get a bonus new players don’t have. The fix only applies in normal runs.
Config can arrive from the server. Part of my balance is controlled through Remote Config. If the old tier values lived in the same config, the server could overwrite them. I put the historical values in code and left the mode behind a flag that can be switched off remotely.

What I do now for any rebalance

I build save cohorts. Not “one old save” but a set: a new player, a mid-game player on the old balance, a top player on the old balance, a player who unlocked tiers on the new balance. Saves come from real versions, not handmade. It’s easy to handcraft a save that suits the test.

I check the outcome, not a proxy. For a combat game the outcome is “does the tower kill what’s coming at it” and “when and why does the run end”. Wave number, run time, “didn’t crash” are proxies. A good question for any test: can it pass while the player is having a bad time?

I check both sides. “The tower doesn’t die” only catches “too hard”. You also need “too easy”: on the top tier a run must not be endless. Otherwise the next fix will overshoot and the test will be green again.

I write a test for the symptom from the complaint. I now have a test that enables the 0.10.1 configuration and checks that there are more than 50 enemies alive at wave 10. In other words, it reproduces the bug. If someone reverts the fix, this test will say the bug is back.

I limit the scope of the fix. A compatibility mode for veterans must not leak into places where fairness matters: tournaments, leaderboards, events with rewards.

I give players a way to send their ID. The second time I could check the specific case quickly because the previous fix added a player ID in settings and a support button with a pre-filled email. Without it, “can’t beat a single enemy” would have taken a long email thread to decode.

Checklist: rebalancing a live game

List which saved values change meaning after the rebalance: upgrade levels, unlocked difficulty levels, currency, records.
Collect saves from real old versions: beginner, mid-game, top, a player who started on the new balance.
Decide for each cohort: migrate, compensate or leave as is, and write down why.
For each cohort run every affected difficulty level, with and without in-run purchases.
Measure the outcome: kills, enemy backlog, time and reason the run ended. Not the wave number and not “didn’t crash”.
Add a “too easy” test: the run isn’t endless, rewards don’t break the economy.
Add a test that reproduces the player’s complaint on the old configuration.
Check that the compatibility mode doesn’t turn on in tournaments, leaderboards and events.
Check that Remote Config won’t overwrite the historical values.
Store the migration flag in the save and set it once, so the next launch doesn’t recalculate the player.
Before release, play at least one old save on the top difficulty level by hand.

I wrote about building a simulator to test balance in the article on balance and a simulator. About why an update is new code reading old data, in the article on testing app updates. This story is where the two topics met.

A test that passes while the player suffers is worse than no test. Without a test you at least know you didn’t check.