DR guide
Testing DR and keeping it working
DR that has never been tested is a guess. Most problems only appear the first time someone actually fails over: a missing firewall rule, an expired certificate at the DR site, a licence server that only exists in production. Testing finds these while there is no real disaster.
Four kinds of test
| Test | What happens | Suggested frequency |
|---|---|---|
| Tabletop walk-through | The team reads the runbook step by step and checks that every step, name and phone number is still correct. | Every quarter |
| Restore test | Restore one server or database from backup and confirm it works. | Every month, rotating systems |
| Isolated failover | Bring a tier up at DR on an isolated network, without touching production users. | Twice a year for each tier |
| Full failover | Move real users to DR for an agreed window, then fail back. | Once a year for tier 1 |
Drill-day checklist
Before
- Freeze changes for the systems in scope
- Confirm replication lag is within RPO
- Tell users and the helpdesk
- Write down what “success” means, for example: users can log in and the last transaction is within the RPO
During
- Time every step
- Write down every manual fix, however small
- Have the application owner test, not only IT
- Agree a hard stop time for rolling back
After
- Report measured RPO and RTO against the targets
- Turn every issue into a fix with an owner and a date
- Update the runbook the same week
- Book the next drill
Keeping DR working between drills
- Alert on replication lag. Lag that quietly grows past the RPO is the most common silent failure.
- Make DR part of change management. Every new application, firewall change or certificate renewal should include the question “and at DR?”.
- Patch DR the same as production. Templates and standby servers fall behind fast.
- Keep immutable backups. At least one copy that cannot be changed or deleted from the network.
- Review the tiers once a year. Applications change in importance. Last year's tier 3 system may now be critical.
Not sure where your DR stands?
Send us your application list and a short note on how backups work today. We will come back with a plain gap review: which systems are exposed, what a realistic recovery time looks like, and what it would take to close the gap.