There’s a conversation I’ve heard more than once in architecture meetings:
“What happens if that server fails?”
“We’ve got another one.”
Great.
“What happens if the database gets corrupted?”
Ah.
It’s usually at this point that the conversation gets a bit quieter. The problem is that high availability and disaster recovery are often talked about as if they’re basically the same thing. They’re not. They’re related, but they’re both solving different problems. What’s more, having a highly available system does not necessarily mean you have robust plan for recovering from a proper disaster.
High availability is about keeping things running
High availability is mostly concerned with the sort of failures we should expect to happen now and again. Servers die. Processes crash. Hardware fails. Networks misbehave. Availability zones can have issues.
That’s all normal stuff. To cover some of this off, you might choose to run your application across several instances behind a load balancer, rather than a single server. If one instance disappears, another can keep serving traffic while you fix the first. You might even spread those instances across different availability zones. Your database might have replicas. Auto-scaling might replace unhealthy instances automatically.
The idea is simple: Individual components can fail without taking the whole service down with them. That’s high availability, and for most production systems, it’s a very sensible thing to have.
Disaster recovery starts when the normal safeguards aren’t enough
Disaster recovery asks a slightly trickier question:. What happens when the environment itself is no longer usable?
Maybe someone accidentally deletes something important. Perhaps there’s a security incident. What if a deployment corrupts data, or an entire region becomes unavailable. Or just possibly the database is happily chugging along replicating corrupted data to every replica you carefully configured for resilience.
In that situation, having three application servers instead of one isn’t much help. They’re all still pointing at the same problem. Which is where disaster recovery comes in. It’s less about keeping everything running continuously, and more about having a reliable way to recover when your normal resilience measures have failed.
RTO and RPO sound dull, but they’re actually useful
Disaster recovery conversations usually bring up two acronyms fairly quickly:
RTO – Recovery Time Objective
How long can your system be unavailable before it becomes a business-breaking problem? Five minutes? An hour? A day?
RPO – Recovery Point Objective
How much data can you afford to lose? Nothing? Five minutes? Yesterday’s data?
These are important because not every system needs the same level of protection. A payment platform probably needs very aggressive recovery targets. An internal reporting dashboard may be perfectly acceptable if it comes back online later that afternoon.
The answer should always come from the business impact, not from whichever disaster recovery architecture looks best on a diagram.
“We have backups” isn’t quite enough
Backups are obviously a vital part of disaster recovery. But having backups and having a disaster recovery plan are definitely not the same thing. A backup is only useful if you can actually restore it.
How long does the restore take? Has anybody tested it recently? Does the application work afterwards? Could someone other than the person who built the backup process actually carry it out?
If your database takes eight hours to restore but the business expects the service back within two, you haven’t met the recovery requirement. You do have a backup, you just don’t have the recovery capability you thought you had.
My general rule is that a backup becomes much more interesting once somebody has successfully restored it.
You probably don’t need the most complicated solution
There’s also a tendency to go too far in the other direction. Mention disaster recovery and suddenly someone wants active-active infrastructure running across three regions. Sometimes that’s justified, often it isn’t. A warm standby environment might be enough. You might be able to rebuild the platform quickly from Infrastructure as Code.
Regular backups, replication and a documented recovery procedure may meet your business needs perfectly well. The important thing is to ensure that the recovery strategy matches the consequences of the failure. Not every system needs to survive the apocalypse.
The simple distinction
The easiest way I’ve found to separate the two is this:
High availability asks: what happens when part of the system fails?
Disaster recovery asks: what happens when the system we were relying on is no longer recoverable through normal means?
Both matter, they’re just different conversations, and ideally, they’re conversations you have before something goes wrong. Because discovering your disaster recovery strategy for the first time during an actual disaster is rarely anyone’s idea of a good afternoon.