Run a restore drill: turn RPO and RTO into a recovery procedure
A successful backup job reports that a copy was created. A restore drill answers the more useful question: can the team recover a working service, with acceptable data loss, inside the time the project can tolerate?
The practical takeaway
Set explicit recovery targets, restore a coherent copy into an isolated environment, test important application behaviour and record measured results instead of assuming the schedule is enough.
Separate the data-loss target from the recovery-time target
Recovery point objective, or RPO, describes the point in time to which data needs to be recoverable. Recovery time objective, or RTO, concerns how long recovery can take before the interruption has an unacceptable effect. Translate both into a business conversation before choosing software. A portfolio website can tolerate a different interruption from a service accepting new orders every minute.
Consider an illustrative booking application whose owner accepts at most one hour of lost updates and aims to recover service within two hours. A nightly copy alone cannot demonstrate the one-hour data target: an incident near the end of the daily interval could lose much more recent work. The example targets belong to that project; they are not an HM Bastion recovery commitment.
Choose a precise measurement boundary. Record when the service became unavailable, when the incident was recognised and when tested service returned. During a planned drill, distinguish time spent rebuilding from time that would also be needed to notice an incident, obtain authorisation and retrieve credentials. Reporting only the fastest import step hides the work that delays real recovery.
Back up everything needed to recreate one consistent service
| Recovery component | What to retain | Common omission |
|---|---|---|
| Application | Release identifier, dependencies and deployment instructions | Source code exists but the required release cannot be rebuilt |
| Database | A supported consistent backup plus required roles or configuration | A copy of live database files is assumed to be restorable |
| User content | Uploads, documents and storage permissions | Database rows point to missing files |
| Private configuration | Recoverable secrets and access to their encryption keys | The key is available only on the failed server |
| Operations | DNS details, scheduled jobs and external-service requirements | The restored site unexpectedly sends live messages |
Use a backup method supported by the database engine. PostgreSQL logical dumps provide a consistent database view, while cluster-wide roles need separate consideration. File-level backup and continuous archiving follow different requirements. A logical dump is not interchangeable with the base backup and archived WAL needed for point-in-time recovery. Select and test one documented method rather than combining incompatible pieces.
Coordinate database and uploaded-file capture. If a customer uploads a document during the copy, a database backup and a much older file archive may describe different application states. Define a maintenance or consistency strategy that suits the application, and include a recent sample attachment in the restore checks. File counts alone do not establish that the right files match the restored records.
Make the recovery copy independent and reachable
Keep a recovery copy outside the failure path you are planning for. Another directory on the same disk does not help when that disk is lost. A copy controlled by the same compromised application credential may also be removable by an attacker. Choose storage access, retention and any immutability controls deliberately, then confirm their actual behaviour rather than relying on a product label.
Encrypt copies containing sensitive information and keep the recovery key available through a separately controlled procedure. Authorise the people who need to restore them, and test retrieval using the permissions they will have during an incident. Include backup storage growth and transfer time in the plan. A copy that technically exists but cannot be downloaded by the recovery operator is not ready for use.
Restore into a safe environment and run a concrete test
- Select a completed backup and record its recovery timestamp, size and integrity evidence.
- Prepare an isolated destination with the required software versions and enough temporary storage.
- Block real email, payment calls, scheduled jobs and webhooks before starting the restored application.
- Restore the database, files and private configuration using the documented procedure.
- Check login, representative records, an uploaded file and a synthetic write in the isolated copy.
- Record the last recovered data point, elapsed stages, failures and operator decisions.
For the booking example, find a known reservation immediately before the backup boundary and verify its related customer and attachment. Confirm that the expected recent reservation appears and that an intentionally later test record does not. Then create a synthetic reservation in the isolated copy. This tests relationships and write capability without contacting a real customer or altering production.
Use a stage log: credential retrieval, destination preparation, download, verification, import, application startup and acceptance checks. Suppose the import takes twenty minutes but access approval takes fifty. The next improvement is the access procedure, not necessarily a faster database plan. Preserve the measured result even when it misses the target; a failed rehearsal has identified work while the service is still healthy.
Turn the drill result into a maintained recovery record
Give each failed check an owner and repeat the affected part after fixing it. Missing uploads require a better backup set; unsupported extensions require a reproducible software inventory. Repeat a complete drill after important application or storage changes and at a frequency justified by the project. Mark the last successful recovery date in the operating notes.
Read hosting backup features separately from your own recovery procedure. Replication, snapshots and retained backups have different roles and failure boundaries. Confirm the features and restore scope attached to your purchased service with the team. A tested application recovery procedure remains valuable even when the infrastructure offers additional recovery tools.
