Disaster Recovery and Business Continuity Questions

Designing and maintaining plans, architectures, and processes to ensure service continuity and recoverability after major incidents or disasters. Topics include defining Recovery Time Objective and Recovery Point Objective, conducting business impact analysis and tiering services by criticality, dependency mapping and recovery ordering, selecting replication and backup strategies including synchronous and asynchronous replication, active active and active passive topologies, snapshots and transaction log based point in time recovery, and planning cold, warm, and hot recovery sites. Also covers failover and failback procedures, orchestration and automation of recovery workflows, runbook creation and stakeholder roles and communications, regular disaster recovery testing and exercises including tabletop, simulated failover, full recovery drills and chaos engineering, metrics tracking such as mean time to recovery and actual Recovery Time Objective achieved, off site and geographic redundancy considerations, cloud versus on premise trade offs, regulatory and data residency requirements, and postexercise reviews to close recovery gaps.

EasyTechnical

45 practiced

Define cold, warm, and hot recovery sites. For a mid-sized company with limited budget, list three practical scenarios where each site type would be appropriate and explain the cost and staffing trade-offs for each option.

MediumSystem Design

20 practiced

A multinational company expanding into new regions must comply with varying data residency laws. Describe a DR approach that supports local data residency for customer records while still enabling centralized operations and efficient DR failover. Include partitioning, replication, encryption, and audit considerations.

Sample Answer

**Clarify requirements & constraints**- I would confirm which regions require residency, RPO/RTO targets per region, allowable cross-border replicas, and regulatory audit retention/retention periods.

**High-level approach**- Partition customer records by region (logical tenancy) and store the primary copy inside the region’s sovereign boundary. Provide centralized operations via read-only aggregated indices and orchestrated failover.

**Partitioning**- Use a region key (country/region code) as primary shard key. Deploy regional database clusters (e.g., managed RDBMS or document DB) holding that region’s primary data. Metadata/global catalog contains only pointers and anonymized indices (no PII) for centralized UI/search.

**Replication & DR**- Synchronous local writes to regional primary for residency compliance. Asynchronous, encrypted replication of non-PII aggregates or tokenized records to a central operations cluster in permitted location for operational needs.- For DR, maintain a cold/warm standby in a permitted DR region that can become writable only after legal-approved escalation and with automated consistency checks. Use multi-AZ within-region replication to meet low RTO.

**Encryption**- Encrypt-at-rest with region-specific KMS keys stored in the local cloud region. Use envelope encryption; central operations only hold wrapped keys when legally allowed. Enforce TLS for in-transit; rotate keys regularly.

**Audit & Compliance**- Immutable, tamper-evident audit logs per region (WORM or cloud audit service) retained per regulation. Central operations have role-based, audited access via bastion and privileged access workflows (JIT, MFA).- Log replication (metadata-only) to central SIEM for monitoring; full PII logs never leave region unless allowed and logged.

**Operational considerations**- Automate failover runbooks, periodic DR drills, integrity checks, and legal gating for cross-border promotion of replicas. Monitor replication lag, key access patterns, and use alerting for policy violations.

I’d implement and maintain these as a Systems Administrator—configuring region KMS, network controls, replication jobs, runbooks, and audit pipelines.

HardTechnical

25 practiced

You discover a suspected ransomware infection that has encrypted parts of production and your backup repository. Walk through a complete incident response and recovery plan that includes containment, forensic preservation (chain-of-custody), backup validation, selective restore strategy, stakeholder communication, and legal/regulatory reporting. Prioritize actions and explain your rationale.

Sample Answer

**Situation & priorities (first 0–2 hours)**- Preserve safety: isolate infected hosts from network (unplug NIC, disable Wi‑Fi, or apply host-based ACLs) to prevent lateral movement and stop further encryption.- Convene incident response lead and record timeline; assign roles (containment, forensics, backup, comms).Rationale: stopping spread first reduces scope; clear ownership prevents duplicated actions.

**Containment**- Quarantine affected VMs/servers; block C2 IPs and malicious SMB/RDP activity on firewalls.- Put backup appliances into read-only or disconnect from network to prevent further infection.Rationale: contain both production and backup surfaces.

**Forensic preservation & chain-of-custody**- Image volatile memory and disk from compromised hosts using trusted tools (e.g., FTK Imager, dd) and document hashes, time, operator, and tool versions.- Preserve logs: SIEM, AD, firewall, backup system logs; export write-once copies.- Maintain chain-of-custody form for each artifact.Rationale: ensures evidence integrity for legal/regulatory and root-cause.

**Investigation**- Identify ransomware strain, initial vector (phishing, RDP), scope of encryption, and presence of exfiltration.- Determine timeline using file timestamps, EDR, and logs.

**Backup validation & selective restore strategy**- Verify integrity of backups: test immutable snapshots, check last clean backup before compromise, validate hashes.- If backups partially encrypted, rely on air-gapped or offline copies; if only some backup sets compromised, prioritize restoring critical systems.- Plan staged restores: domain controllers, authentication services, DNS, then application and database servers, then user data.- Before restore, rebuild or patch base OS images, apply hardened configurations, rotate credentials, and deploy endpoint protection.Rationale: staged restores reestablish trust and reduce re-infection risk.

**Communications & stakeholder management**- Notify executives, IT, security, legal, and impacted business owners with clear status, scope, and expected timelines.- Provide scripted internal messages for users (what to do, e.g., change passwords, avoid suspicious emails).- Escalate to customers/regulators as required by policy.Rationale: timely, accurate communications preserve trust and coordinate response.

**Legal/regulatory reporting**- Engage legal and compliance immediately; assess obligations (breach notification laws, data protection).- If exfiltration suspected, prepare breach notification plan with timelines, evidence, and remediation steps.- Consider law enforcement engagement (FBI/ local) for intelligence and potential recovery support.

**Recovery & hardening**- Restore from validated backups into isolated network segments; confirm integrity and functionality.- Conduct post-restore forensic review and monitoring; perform full vulnerability scans and password resets for privileged accounts.- Apply lessons learned: patching, MFA, backup immutability, network segmentation, phishing training, and runbooks update.

**Closure & metrics**- Produce incident report: root cause, systems affected, downtime, data loss, cost, and remediation.- Review RTO/RPO, update DR tests, schedule tabletop exercises.

This sequence prioritizes stopping spread and evidence preservation, then validating recoverability before restoring services to prevent reinfection and meet legal obligations.

EasyTechnical

21 practiced

When planning geographic redundancy (off-site backups and replication across regions), what factors should a systems administrator evaluate? Cover considerations such as latency, network costs, legal/data residency, time zones, and recovery orchestration complexity.

MediumTechnical

21 practiced

Design a backup and recovery strategy for a three-tier e-commerce application: web servers, application servers, and a transactional SQL database. The business requires database RTO=30 minutes, RPO=5 minutes; application servers have RTO=2 hours. Describe where to place backups/replicas, backup frequency, recovery order, and any automation you would implement.

Sample Answer

**Approach (summary)** I design a layered strategy: fast DB replication + PITR for RPO=5min/RTO=30min, image/config-based rebuilds for app/web tier with automation for RTO=2hrs, and off‑site durable backups for long‑term recovery.

**Placement & replication**- Database: Primary in AZ-A, synchronous or semi-sync replica in AZ-B (same region) for 0–5min RPO; additional async cross-region replica for region-level DR.- App servers: AMI/VM snapshots and container images in same region and a second region; keep artifacts in artifact registry (e.g., Docker registry) and config in Git.- Web servers: Stateless behind LB/ALB, autoscaling groups; configs in Git/CI.

**Backup frequency**- DB: - Continuous WAL (transaction log) streaming/archiving to object storage (S3) every 1 min (enables 5min RPO). - Daily full snapshot (nightly). - Hourly incremental snapshots (or base + WAL covers this). - PITR retention: 7–14 days.- App/web: - Weekly AMI/VM image + after any major deploy. - CI builds stored continuously (every deploy).- Config/state (secrets, configs): backed up on change to encrypted storage.

**Recovery order / procedure**1. Network, DNS, LB health checks and route traffic to standby LB.2. Promote DB replica (automated if healthy) or restore latest full snapshot + apply WAL to desired point — target <=30min RTO.3. Start application servers from latest AMIs/containers + pull latest config/secret; scale to required capacity (automation).4. Bring web tier up behind LB and verify sessions/cookie handling and cache warming.5. Run smoke tests, cut traffic fully to recovered stack.

**Automation & tooling**- Automated failover: Patroni/etcd or cloud-managed DB failover (RDS Multi-AZ, Aurora) + automated promotion scripts; health checks in monitoring (Prometheus/CloudWatch).- Backup orchestration: scheduled snapshot jobs, WAL archival to S3 via lifecycle rules; integrity checks and automated restore drills (monthly).- IaC: Terraform/CloudFormation for rebuilding infra; Ansible/Chef for config + immutable images.- Runbooks + automated playbooks: one-command restore (promote/restore + DNS swap + scale).- Alerts: SLA alarms for replication lag, snapshot failures; automated ticketing.

**Security & retention**- Encrypt backups in transit/rest, use KMS for keys, enforce least privilege for backup access, retention and secure deletion policies.

This meets RPO=5min via near‑real‑time WAL streaming and replicas; RTO=30min via pre-warmed replica promotion and automation; app RTO=2hrs via AMIs/containers + IaC and deployment automation. Regular restore tests validate readiness.

Unlock Full Question Bank

Get access to hundreds of Disaster Recovery and Business Continuity interview questions and detailed answers.

Join thousands of developers preparing for their dream job.