Incident Containment and Remediation Questions

Focuses on the practical judgment, processes, and technical actions used to respond to active security incidents, contain attacker activity, eradicate threats, remediate affected systems, preserve evidentiary integrity, and restore services with minimal business impact. Coverage includes containment strategies from immediate short term isolation and network segmentation to longer term monitored observation and selective blocking of attacker infrastructure; trade offs between rapid containment that reduces blast radius and slower approaches that preserve forensic visibility to determine attacker objectives and scope; and prioritization of remediation steps such as removing attacker access, eradicating malware, applying patches, closing exploited vulnerabilities, resetting compromised credentials, rebuilding or hardening systems, and validating fixes through testing and monitoring. Also includes recovery procedures such as phased restoration, rollback to known good images, and integration with business continuity plans. Operational topics include defining decision boundaries and escalation paths for analyst actions versus management or change control approvals, assessing business impact and continuity trade offs, coordinating with system administrators, database teams, application owners, legal and business stakeholders, preserving evidence and maintaining chain of custody for forensic analysis, communicating status to stakeholders, and conducting post incident activities including root cause analysis, lessons learned, and updates to runbooks and controls.

MediumTechnical

50 practiced

A kernel-level rootkit has been found on several Linux servers. Provide a detailed remediation plan that covers confirming detection, short-term containment, decisions between in-place cleaning vs rebuild, removing persistence mechanisms, validating the kernel and system integrity, and controls to prevent recurrence. Include steps to preserve evidence for later analysis.

Sample Answer

**Confirm detection / Triage**

- Collect indicators: suspicious kernel modules (lsmod), unseen network sockets (ss -tanp), hidden processes (ps auxw --forest vs /proc scan), unusual syscalls (auditd, eBPF), modified kernel symbols.- Capture volatile data: `ss`, `netstat`, `ps`, `lsmod`, `dmesg`, `journalctl -k` and `/proc/*` listings.- Preserve timestamps and operator log (who, when).

**Short-term containment**

- Isolate affected hosts from production network (air-gap or VLAN quarantine) but keep powered for forensics.- Disable remote access (remove from load balancers, block IPs) and implement temporary access jump box for investigators.- Snapshot live memory (magneto/volatility) and make forensic disk images before changes.

bash

# example: memory capture and disk image
sudo liME -o /tmp/mem.lime
sudo dd if=/dev/sda conv=sync,noerror bs=1M | gzip > /mnt/forensics/srv1_sda.img.gz
sha256sum /mnt/forensics/srv1_sda.img.gz > /mnt/forensics/srv1_sda.img.gz.sha256

**Preserve evidence**

- Chain-of-custody: who accessed, when, purpose.- Record checksums of images and raw memory, preserve original media, readonly mounts for analysis.- Collect relevant logs (syslog, auditd, auth.log, kernel logs) and network captures.

**In-place cleaning vs rebuild decision**

- Prefer rebuild for kernel-level rootkits: rootkits modify kernel objects, hidden persistence and incomplete trust in OS integrity.- Consider in-place only if: isolated non-production lab, critical uptime impossible and rootkit well-understood with vendor-signed remediation procedure, and full revalidation planned.- Decision factors: business impact, time-to-rebuild, evidence needs, ability to reinstall from trusted image.

**Removing persistence**

- Identify persistence vectors: kernel modules, initramfs, systemd units, cron, bootloader (grub), firmware.- Steps: boot from trusted media, mount filesystems readonly, inspect and compare initramfs, replace grub and kernel from trusted packages, reinstall firmware where compromised.- Recreate host from golden image; do not reuse binaries from suspect system.

**Validate kernel and system integrity**

- Verify kernel and package signatures (dpkg --verify / rpm -V), compare binaries with known-good hashes.- Use trusted offline tools: perform AIDE/HIDS scans, run kexec/boot Trusted kernel, perform TPM/UEFI Secure Boot verification.- Integrity checks: verify module signatures and presence of unexpected symbols.

**Controls to prevent recurrence**

- Enforce Secure Boot + signed kernels, kernel module signature verification, disable module loading if possible (grsecurity or lockdown), enable SELinux/AppArmor enforcing.- Patch management, host-based EDR with kernel visibility, eBPF monitoring, immutable infrastructure (ephemeral hosts, IaC), network segmentation, enhanced logging and SIEM detection rules for unusual kernel API usage.- Incident lessons fed into IR playbooks and CI/CD hardening.

**Post-incident**

- Perform root cause analysis, document IOC, update IDS/IPS, rotate credentials and keys, notify stakeholders, and schedule audits.

EasyTechnical

41 practiced

When preparing to perform forensic imaging on a compromised host, what artifacts and data sources should you prioritize (for example: full disk image, memory capture, system logs, application logs, network captures, snapshots)? Explain dependencies between collections, the order of operations, and why some items must be collected before others.

HardTechnical

36 practiced

You are the incident lead and discover a fast-spreading malware in a production-critical service. Stopping the service would prevent spread but cause immediate operational failure and potential safety issues. Design a decision framework that balances RTO/RPO, legal obligations, safety, business impact, and security. List required telemetry inputs, stakeholders that must be consulted, pre-authorized actions, and a decision tree that SOC and management can follow under time pressure.

Sample Answer

**Decision framework (summary)** - Objective: Stop malware spread while minimizing safety risk, legal exposure, and business loss. Use a risk-tiered, time-boxed decision model: Immediate Containment (seconds–minutes) → Stabilize & Assess (minutes–hours) → Remediate/Recover (hours–days).

**Telemetry inputs (required)** - Endpoint telemetry: EDR alerts, process trees, file hashes, execution timestamps - Network: IDS/IPS, flow logs, segmentation hit counts, external C2 connections, DNS logs - Service health: RTO/RPO SLA metrics, active transactions, safety-system states, failover readiness - Logs: Auth logs, privileged actions, change management events - Inventory: Asset criticality, safety dependencies, backup currency

**Stakeholders to consult (fast-path order)** - Incident Lead (you), SOC Triage, Platform/Service Owner, Safety/Operations Lead, Legal/Compliance, CTO/Head of Ops, Communications/PR, SRE/On-call

**Pre-authorized actions (policy-backed)** - Isolate host-to-host lateral edges (apply microsegmentation rules) - Block known malicious IPs/domains at perimeter and DNS sinkholing - Quarantine affected endpoints (network-level) but keep service process read-only if safe - Enable additional logging and packet capture - Trigger cached failover if automated and safety-validated

**Decision tree (time-pressured)** 1. Confirm malware indicators? (Yes → continue; No → monitor) 2. Is malware actively spreading laterally? - Yes → Can network microsegmentation or targeted blocks stop spread without stopping service? - Yes → Apply targeted blocks; monitor 5–15 min; escalate if spread persists. - No → Does stopping service create immediate safety hazard? - Yes → Maintain service; implement containment controls (quarantine, rate-limit, read-only); notify Safety & Legal; prepare controlled shutdown plan. - No → Stop service (pre-authorized) and failover to DR; initiate forensic snapshot and recovery per RPO.3. After containment: assess RTO/RPO impact, legal reporting, and invoke communication plan.

**Rationale & metrics** - Prioritize human safety and legal obligations over uptime. Use pre-defined thresholds (e.g., >3 hosts infected in core cluster → aggressive containment). Log every action for forensics and compliance. Conduct post-incident review to update pre-authorizations and playbooks.

HardTechnical

44 practiced

Design a full-scope tabletop exercise simulating a multi-vector ransomware attack that impacts several business units and cloud services. Provide the scenario narrative, learning objectives, required participants and their roles, decision points (pay vs restore, shutdown vs isolate), injects, measurable outcomes to assess readiness, and a plan for post-exercise remediation of playbooks and SLAs.

Sample Answer

**Scenario Narrative**A coordinated multi-vector ransomware campaign targets our org during month-end close: phishing leads to stolen credentials, an exposed management API allows lateral cloud service access, and a misconfigured S3 bucket enables data exfiltration. Ransomware encrypts on-prem file shares, cloud-hosted databases, and CI/CD artifacts. Business units affected: Finance, Engineering, Customer Support, and Sales.

**Learning Objectives**- Validate incident detection, containment, and recovery workflows across hybrid environments- Test CIRT, cloud ops, and business continuity coordination and decision-making under pressure- Exercise pay-vs-restore, shutdown-vs-isolate trade-offs, and cross-team communications- Identify gaps in playbooks, SLAs, and technical controls (IAM, backups, segmentation)

**Participants & Roles**- Exercise Lead (facilitator)- CISO / Incident Commander (decider)- Cybersecurity Engineer (technical lead) — I design/execute containment/forensic steps- SOC Lead, Threat Intel, Forensics, Cloud Ops, DevOps, Backup Admin- Legal, Finance, PR, HR, Business Unit Heads (Finance, Engineering, Sales), External Counsel, Law Enforcement liaison

**Key Decision Points**- Pay ransom or accept restoration timeline- Global shutdown vs network isolation/segmented containment- Whether to rebuild CI/CD vs restore from backups- Customer disclosure timing and regulatory reporting

**Injects (timed)**- T+0: SOC alert - abnormal SMB/encryption signature on Finance file shares- T+15: User reports encrypted home directory; ransom note appears- T+45: Cloud monitoring flags mass deletion in prod DB; CI pipelines failing- T+90: Customer ticket backlog grows; regulator inquiry simulated- T+120: Backup verification fails for one region; media leak simulated

**Measurable Outcomes**- Mean time to detect (MTTD), mean time to contain (MTTC), mean time to recover (MTTR)- Time to decision for each critical choice, decision rationale logged- Number of successful cross-team communications vs missed actionables- Percentage of systems restored from immutable backups within RTOs- Playbook execution completeness score (checklist-based)

**Post-Exercise Remediation Plan**- Immediate hotwash (within 24h) to capture actions, owners, and gaps- Produce after-action report with prioritized remediation (technical: MFA, least privilege, backup integrity; process: escalation paths, SLAs)- Update playbooks and runbooks with specific steps discovered lacking; add decision criteria for pay vs restore- SLA updates: define response and recovery SLAs by system criticality; add verification cadence for backups- Schedule targeted tabletop follow-ups and technical red/blue tests to validate remediations within 90 days

I would run this exercise with clear inject timers, scoreboard for metrics, and ensure my role focuses on rapid containment options (isolate VPCs, revoke keys), forensic evidence preservation, and automated restoration scripts to meet RTOs.

EasyTechnical

52 practiced

Define 'blast radius' in the context of a security incident. Provide practical, immediate measures you could implement to reduce blast radius across both network and identity layers, and explain the trade-offs each measure introduces for availability, performance, and forensic visibility.

Sample Answer

**Definition (blast radius)** Blast radius is the scope and magnitude of impact a security incident can have — which systems, identities, data, and network segments an attacker can access or disrupt after initial compromise.

**Immediate measures — Network layer**- Network segmentation / microsegmentation: isolate workloads using VLANs, security groups, or SDN policies. - Trade-offs: may reduce availability during reconfiguration; slight latency from additional hops; improves containment and forensic clarity.- Strict egress filtering & block lists: limit outbound channels to known destinations. - Trade-offs: can break legitimate integrations; minimal perf impact; improves detection and narrows attacker exfil channels.- Apply host-based firewall rules & ACLs: deny-by-default between tiers. - Trade-offs: operational overhead; potential false positives causing outages; enhances containment and logs network flow for forensics.

**Immediate measures — Identity layer**- Enforce least privilege & short-lived credentials (rotate keys, reduce token lifetime). - Trade-offs: usability friction; token refresh overhead; reduces credential misuse scope and improves auditability.- Enable MFA for all privileged accounts and require phishing-resistant methods (hardware keys). - Trade-offs: user friction and recovery complexity; negligible perf impact; major reduction in takeover risk and clear auth logs.- Revoke/stage-compromise containment: disable suspicious accounts, rotate service principals, revoke active sessions. - Trade-offs: possible service disruption; immediate reduction of attacker access; may obscure some realtime forensic state but preserves post-revocation audit records if logged.

**Notes on visibility & trade-offs**- More segmentation and shorter credentials increase telemetry quality but can complicate incident response if changes are made without preserving logs. Always coordinate containment with preserved logs and immutable copies (SIEM, packet captures) to balance availability and forensic needs.

Unlock Full Question Bank

Get access to hundreds of Incident Containment and Remediation interview questions and detailed answers.

Join thousands of developers preparing for their dream job.