release
3 articles
-
Testing Feature Flags and A/B Experiments: How Not to Ship a Mess of Variants
A flag isn't "on/off" — it's a live branch of code in production that you test in both states. A first-person take: why one flag becomes a matrix of states; where the nasty targeting and sticky-bucketing bugs hide; how to test an A/B experiment as a mechanism rather than a hypothesis; kill switches, zombie flags, and a pre-release checklist.
-
Update testing — the bugs only users on old versions ever see
The release was tested perfectly — on a clean install. But almost every user gets it as an update on top of an old version with old data. A first-person take: why an update means new code reading old data, the N-5 → N version matrix and migration chains, updates landing mid-session, force update and its bypasses, staged rollout and a server that must support two versions at once, downgrade as a crash loop, first launch after an update ≠ FTUE — and why an archive of old builds must exist.
-
Knight Capital: how $440M vanished in 45 minutes because of one deploy
A QA-eye breakdown of the August 1, 2012 disaster: a reused feature flag, an unnoticed 8th server, the dead Power Peg code, and 97 ignored alerts. 7 lessons and a 10-point release-process checklist.