Mega2580.Solutions
October 2021

Zero-downtime database migrations, a checklist

By A. Tanaka · 5 min read

Schema migrations are where we see the most self-inflicted outages, almost always from a change that's correct in isolation but unsafe under concurrent load. A short checklist we run before anything touches a live table:

Is the migration backward compatible with the currently-running code, not just the code being deployed? Does adding a column set a default that locks the table, or is it nullable first and backfilled separately? Is there an index being built that should run concurrently instead of blocking writes? Is there a tested rollback that doesn't require restoring from backup?

None of these are exotic. Almost every migration-related incident we've investigated skipped one specific item on this list, usually under time pressure. The checklist is boring on purpose — it's meant to survive someone being in a hurry.