DoorDash Systems Administrator (Entry Level) - Comprehensive Interview Preparation Guide

Direct answer

The system has one authoritative trigger source (the HR platform's joiner/mover/leaver events), one authoritative identity store (Active Directory or LDAP, AD/LDAP), a lifecycle engine that translates HR events into group-membership changes, SSSD (System Security Services Daemon, the Linux client that resolves and caches AD/LDAP identity locally) on all 1,000 hosts, a Privileged Access Management (PAM) platform that brokers time-boxed break-glass access for contractors, a configuration-management reconciliation loop that catches hosts back up after disconnection, and a central, append-only audit log that every other component writes to. Disconnected hosts are handled by treating propagation as eventually consistent rather than instantaneous, and every temporary grant is enforced with an expiry the system checks itself, not one a human has to remember.

Structured elaboration

Components.

• HR system (source of truth). Emits joiner, mover, and leaver events, including a contractor's contract end date, as the single primary trigger; manual tickets remain an exception path, not the normal mechanism, so the system's behavior does not depend on someone remembering to file a request.
• Identity lifecycle engine. Consumes HR events and translates a business event ("Alice moved from Sales to Finance," "Bob's contract ends on this date") into concrete access actions (add and remove specific role groups, schedule an auto-disable). This is the single place role-to-group mapping logic lives, rather than being duplicated per downstream system.
• Central identity store (AD/LDAP). The authoritative account and group-membership database that every Linux host defers to instead of maintaining local accounts.
• SSSD on each of the 1,000 Linux hosts. Resolves AD/LDAP-defined users and groups into local Linux identity and authenticates against the central store, caching recently resolved identity data locally, which matters directly for the disconnected-host case below.
• Privileged Access Management (PAM) platform. Note the acronym collision worth flagging for clarity: this is a distinct thing from Linux's own Pluggable Authentication Modules, also abbreviated PAM, which is the local authentication framework SSSD plugs into on each host. The Privileged Access Management platform here brokers break-glass and other temporary elevated access for contractors: vaulting credentials, granting time-boxed access, recording sessions, and enforcing expiry, rather than the lifecycle engine building bespoke privileged-access logic of its own.
• Configuration-management / reconciliation layer. Applies host-local policy (sudoers scoping derived from group membership, for example) on every host and, critically, re-pulls current authoritative state on every scheduled run, acting as the retry mechanism for any host that missed a live push while offline.
• Central audit log aggregation. Every other component ships its events here: HR event ingestion, the lifecycle engine's decisions, native AD/LDAP change auditing, the PAM platform's grant/use/expiry and session-recording events, and each host's own local authentication logs.

flowchart TD
    A[HR system: joiner, mover, leaver events] --> B[Identity lifecycle engine]
    B --> C[Central identity store: AD/LDAP]
    C --> D[SSSD on each of 1000 Linux hosts]
    B --> E[PAM platform: break-glass and temporary elevation]
    E --> D
    F[Configuration management reconciliation loop] --> D
    D --> G[Central audit log aggregation]
    B --> G
    C --> G
    E --> G

Data flow per event type. Joiner: the HR event drives the lifecycle engine to create the AD/LDAP account and assign baseline and role-derived group memberships; SSSD on any host the new hire needs picks up the identity on its next lookup, and the configuration-management layer converges any host-local artifacts (home directory, derived sudoers entries) on its next run. Mover: the lifecycle engine computes the difference between the old role's groups and the new role's groups and applies both the removals and the additions in the same operation, since removing only the additions and forgetting the removals is the single most common gap in otherwise well-designed lifecycle systems. Leaver: the lifecycle engine disables (never immediately deletes) the account to preserve forensic history, revokes any PAM-platform-vaulted grants tied to that identity immediately, and schedules deletion or archival for later under a retention policy rather than instantly. Contractor break-glass: a request triggers a PAM-platform-brokered, time-boxed grant (temporary sudoers-mapped group membership or a vaulted credential checkout), which the platform expires automatically, with the full session recorded and logged.

Contractor auto-disable and re-enable on approval. Because a contractor's engagement is date-bound rather than open-ended, the lifecycle engine tracks the contract end date from the same HR/contract feed and disables the account automatically on that date without waiting for a separate leaver event to be filed. If the engagement is extended, the extension goes through an explicit approval step (the engagement owner or manager approves it), and the same account is re-enabled and its end-date attribute updated, rather than a new account being created; reusing the same identity keeps its entire prior audit trail, including any earlier break-glass activity, attached to one continuous record instead of fragmenting it across two identities for the same person.

Handling disconnected hosts. SSSD's local cache is the primary mechanism that lets a network-partitioned host keep authenticating previously seen users for a bounded offline window, but that same cache is also the risk: a host that is disconnected when a leaver event fires may keep honoring a now-terminated user's credentials until it reconnects. Three things bound that risk: the cache's offline validity window is kept deliberately short rather than indefinite, so a prolonged disconnection expires the cached credential rather than trusting it forever; the configuration-management reconciliation loop re-pulls current authoritative state and forces a cache refresh on every scheduled run, so a host that missed a live push still catches up on its next run rather than staying silently stale; and for genuinely high-risk terminations (involuntary, security-related), an explicit fast-path revocation targets disconnected hosts specifically once they become reachable again, rather than relying purely on the standard cache-expiry timeline. The reconciliation system itself also tracks and alerts on hosts that have not successfully checked in within an expected window, so a silently stale host is visible to operations instead of assumed compliant.

Temporary access expiry. Every temporary grant, contractor break-glass elevation or an emergency admin session, is created with an explicit, system-enforced expiry from the start, never a "remember to remove this" convention. The PAM platform removes the grant automatically at the scheduled time, independent of any human follow-through. Because expiry enforcement is itself a push to the affected host, it inherits the same disconnected-host problem described above: if the host is offline at the scheduled expiry moment, the same reconciliation-on-reconnect mechanism must re-check and enforce the expiry once the host is reachable again, rather than assuming the original expiry action succeeded. Both the scheduled removal and the confirmation that it actually took effect on the host are logged as two separate events, closing the loop between intending to revoke access and verifying it was revoked.

Auditable trail of changes. Every layer, the HR feed ingestion, the lifecycle engine's decision, the native AD/LDAP directory change, the PAM platform's grant/use/expiry and session recordings, and each host's own local authentication log, ships to the same central, append-only log store, correlated by a consistent event or request identifier that threads from the original HR trigger through the directory change to the host-level effect. That correlation is what makes a single audit query able to answer "why did this account have this access, from which triggering event, approved by whom, and when was it revoked," rather than requiring a manual cross-reference across four separate systems' logs by guessing at timestamps. The audit store itself is kept append-only and access-controlled separately from the systems that generate its events, since an attacker who compromised the identity system itself would otherwise be able to erase their own tracks from a log store that same system controls.

Worked example

A contractor, csmith-ext, is onboarded on 2026-01-06 with an initial contract end date of 2026-03-31, sourced from the HR/contract system. The lifecycle engine creates the AD/LDAP account and assigns baseline contractor group membership. On 2026-02-10, csmith-ext requests break-glass elevated access during a production incident; the PAM platform grants a 4-hour window, 14:00 to 18:00 UTC, auto-expiring at 18:00 regardless of whether the session is still active, with the full session recorded and logged. On 2026-03-25, the engagement is extended to 2026-06-30; the engagement owner approves the extension, and the lifecycle engine updates the same account's contract-end-date attribute rather than creating a new account, so the February break-glass event remains attached to the same continuous identity. When 2026-03-31 arrives, the original end date, the scheduled auto-disable check reads the account's current end-date attribute, which by then already reflects 2026-06-30, so the auto-disable does not fire; had the extension approval not landed in time, the account would have auto-disabled on 2026-03-31 regardless, and a late-arriving approval would then go through the explicit re-enable path rather than silently reactivating the account on its own.

Trade-offs and pitfalls

Bounding the SSSD cache's offline validity window trades some availability (a disconnected host cannot authenticate a user whose cached credential has expired, even if that user is still legitimately employed) for security (a stale cache cannot indefinitely honor a terminated user's access), and that trade-off should be made explicitly and tuned, not left as an unexamined default in either direction. Relying on the PAM platform as the only path to emergency access is itself a single point of failure hiding inside the system meant to handle emergencies; a genuinely offline, sealed break-glass credential kept as a last resort, separate from the platform's own break-glass feature, is what actually protects against the platform itself being unavailable during an incident. Auto-disabling contractor accounts strictly by date is only as reliable as the HR/contract system's own data timeliness; a verbally agreed extension that has not yet been entered into that system will still result in the account auto-disabling on schedule, a legitimate but disruptive false positive, and the correct response is a fast, clearly documented re-enable-on-approval path, not disabling the auto-disable behavior itself, which would reintroduce exactly the forgotten-account risk it exists to close. Treating HR as the sole, always-timely trigger source is itself a risk, since HR systems occasionally lag or contain errors; an independent periodic reconciliation between the directory's account state and HR's current roster, flagging mismatches for review, catches HR-side data problems that a purely event-driven design would otherwise miss entirely. Finally, at 1,000 hosts, propagation is inherently eventually consistent, not atomic; the audit trail has to capture "revoked centrally at this time" and "confirmed enforced on this specific host at this later time" as two distinct events, not one, or the audit record will silently overstate how quickly a revocation actually took effect across the fleet.

Situation & Goal
As a Systems Administrator I’d build a continuous compliance pipeline that enforces CIS for Windows/Linux across cloud providers and on‑prem, covering image build, provisioning gates, runtime drift detection + auto‑remediation, and auditor-grade evidence.

High-level components

• Image build: Packer + Ansible/PowerShell DSC + CIS hardening roles
• Registry/Artifact store: Harbor/ECR/GCR/Artifactory with signed images (cosign)
• IaC & provisioning: Terraform + cloud provider policy (Sentinel/Cloud Custodian) + Terraform Cloud/GitOps (Flux/ArgoCD)
• Runtime enforcement: Fleet agents (OSQuery + Wazuh or CrowdStrike + Microsoft Defender for Endpoint), central rules engine (Elastic SIEM or Splunk)
• Drift remediation: Automated remediation playbooks (Ansible Tower/AWX, SSM Automation, Runbooks)
• Reporting & evidence: Immutable evidence store (S3/Blob with versioning + WORM), PDF/JSON auditor reports generated by reporting service (ELK/Timescale + custom report generator)

Data flow

1. Dev commits image config → CI builds with Packer → run CIS scanners (OpenSCAP for Linux, CIS-CAT for Windows) → fail build if critical controls fail.
2. Artifact signed → IaC plan runs policy checks → deny apply on critical violations.
3. Agents push telemetry to SIEM → rules detect drift → create ticket + trigger remediation playbook → rescan and record evidence.
4. Reporting service aggregates scan results, policy decisions, remediation logs, and signed artifacts to produce auditor packages.

Tools & rationale

• OpenSCAP / CIS-CAT: canonical CIS checks
• OSQuery + Wazuh: flexible inventory + file/registry checks
• Cloud Custodian/Sentinel: declarative policy enforcement per provider
• Ansible/SSM: mature, idempotent remediation
• Cosign + artifact store: chain of custody

Failure modes & mitigations

• Agent loss/telemetry gap → heartbeat monitoring, fallback active scanning via orchestration (Ansible, remote execution)
• False positives → staged policy (monitoring mode → enforce), whitelist exceptions tracked in evidence store
• Network partition → local cached policies + eventual sync; remediation queued
• Scale bottlenecks (SIEM/store) → partition by region, use streaming (Kafka) and autoscale workers

Validating accuracy at 10k hosts

• Synthetic canaries representing each OS/profile; randomized fault injection and seeded non‑compliance
• Continuous sampling: schedule distributed full scans on 1% hosts hourly, rotate to cover all in 24 hours
• Measure detection precision/recall by comparing agent findings vs. gold‑standard offline scans; target >99% critical detection
• Load test telemetry pipeline with replayed telemetry (Kafka + producer farm) to validate throughput and retention
• Regular audit drills: generate auditor packages and run third‑party review quarterly

Operational practices

• RBAC, MFA, key rotation, and immutable logs (WORM)
• Change control: all policy changes go through Git PRs with CI tests
• Runbooks and SLOs: MTTR targets for remediation; dashboards for compliance drift and evidence completeness

This design ensures automated enforcement across lifecycle, documented evidence for auditors, and resilient scale to manage 10k hosts.

Approach (high level)
Determine whether memory is charged to the process (user-space) or to the kernel (slab/other). Correlate RSS growth over time with anonymous mappings, heap, mmapped files, file caches, or slab growth. Then apply minimally-invasive mitigations (restart service process, apply cgroups/oom tuning, drop caches) without rebooting host.

Investigation steps & commands

• Identify PID and RSS trend:

ps -eo pid,comm,etime,rss,vsz --sort=-rss | head

• Live inspect process memory:

top -p <pid>        # observe RES, SHR, %MEM over time
pmap -x <pid>       # shows anon vs file-backed and sizes
smem -P <process_name> -k   # per-process PSS/RSS, use -c for CSV

• Inspect mappings for big anonymous or mmapped regions:

cat /proc/<pid>/maps | less
cat /proc/<pid>/smaps | grep -E "AnonHugePages|Anon|Rss|Private_Clean|Private_Dirty" -A3

• If kernel leak suspected, check slab and kernel allocations:

slabtop -s c         # live slab usage
cat /proc/slabinfo | sort -k3 -n | tail

• Correlate file-cache vs process:

free -h
cat /proc/meminfo | egrep "MemTotal|MemFree|Buffers|Cached|Slab"

• Capture a heap/profile for user-space app (if supported):

# for glibc/heap: gdb --pid=<pid> and thread apply all bt
# for jemalloc/ tcmalloc/Go/Java use their profilers (jeprof/pprof/jmap/jcmd)
gcore <pid>           # produce core for offline analysis (disk heavy)

How to decide user vs kernel leak

• User-space leak signs:
  • pmap/smaps show growing anonymous/heap regions (Anon, Private_Dirty).
  • smem/pmap show increasing RSS/PSS for that PID.
  • /proc/<pid>/maps grows new anonymous mmaps or larger heap (brk).
• Kernel leak signs:
  • Overall RSS of process not fully explaining system memory use.
  • slabtop shows particular slab caches steadily increasing.
  • High Slab in /proc/meminfo while process RES not matching memory pressure.

Immediate mitigations (no host restart)

• Restart only the service process:

systemctl restart <service>    # preferred if managed by systemd
# or for less disruption:
kill -TERM <pid> && systemctl restart --no-block <service>

• If restart must be avoided, contain memory:
  • Move process into a cgroup memory limit to prevent host OOM and trigger controlled restart:

cgcreate -g memory:leaky
cgset -r memory.limit_in_bytes=1G leaky
cgclassify -g memory:leaky <pid>

• Adjust oom_score_adj for critical processes so kernel kills the leaky one first:

echo 1000 > /proc/<leaky-pid>/oom_score_adj

• Release file caches (only if leak is file-backed or cache-heavy):

echo 3 > /proc/sys/vm/drop_caches

• If leak is in language runtime, trigger GC if supported (e.g., send SIGUSR1 to some apps or use app-specific admin endpoint).
• Enable automatic restarts in systemd to reduce impact:

[Service]
Restart=on-failure
RestartSec=10

Follow-up (medium-term)

• Run heap profiler, jemalloc/tcmalloc/Java Flight Recorder in staging to find root cause.
• Add cgroup limits and monitoring/alerts (Prometheus) to avoid host OOM.

This sequence finds whether memory is user or kernel and gives immediate production-safe mitigations without rebooting the host.

Situation & objectives
RTO ≤ 4 hours, RPO ≤ 1 hour. Small app: 3 web servers, Postgres DB, 500 GB NFS user uploads. Minimize cost while meeting SLAs.

Backup strategy (high-level)

• PostgreSQL: continuous WAL shipping (streaming replication to a standby or archive WALs) + nightly base backups (pg_basebackup).
• NFS uploads: hourly incremental snapshots (filesystem-level or block-level) + daily full snapshot.
• Web servers: stateless; keep config as code (Ansible/Git). Back up configs + package lists nightly.

Schedules & RPO/RTO mapping

• WAL streaming enables point-in-time recovery to ≤ 1 hour RPO (WAL upload every 1–5 min).
• Base backup: nightly full (keep 7 days). WAL retention to cover at least 24–48 hours.
• NFS snapshots: hourly retention 24 hours, daily full retention 14 days, weekly archives 90 days (move older to colder storage).

Storage targets & cost

• Primary backups to cheap object storage (S3/Glacier or equivalent). Keep latest 48h snapshots on faster storage (S3 Standard/IA) for quick restore; archive older to Glacier/Coldline.
• Replicate critical DB WALs to a second region/cloud account if budget allows.

Recovery plan & RTO

• DB failover to standby (minutes) or restore from base + WAL replay: estimated < 2 hours.
• NFS restore: restore latest hourly snapshot to a new volume and remount; 500 GB from object-store snapshot to fast block storage (provision parallel workers) — target < 2 hours.
• Web servers redeploy from config repo → < 30 minutes.

Testing & verification

• Quarterly full DR test: simulate db loss and NFS loss; restore into isolated VPC, verify app end-to-end.
• Monthly partial tests: restore one hourly snapshot and apply WALs to validate integrity; verify checksums and file counts.
• Daily automated backup validation: verify snapshot creation, WAL archive success, and run quick checksum.

Retention & cleanup

• Hourly: keep 24
• Daily: keep 14
• Weekly: keep 12 (months)
• Monthly/Archive: keep 1 year on cold storage

Operational notes

• Automate with cron/backup tool (Borg/Restic for files + lifecycle policy to object storage); use monitoring and alerts for failed backups.
• Encrypt backups at rest and in transit; restrict access via IAM.
• Document runbooks for restore steps and maintain RTO timers.

When to deploy an RODC (summary)

• Use in remote/branch offices with limited physical security, untrusted staff, or poor WAN reliability where you need local authentication and DNS but want to minimize risk of credential exposure.
• Ideal when you need faster logon and local Kerberos services but cannot guarantee server protection or maintenance staff.

Credential caching & Password Replication Policy (PRP)

• By default an RODC caches no domain account passwords. The PRP on the RODC controls which user/computer/service account passwords are allowed or denied for caching.
• When a user authenticates and their password is allowed by PRP, the writable DC replicates (securely) that password hash to the RODC and it is cached for subsequent local logons.
• If a password is not cached the RODC forwards authentication to a writable DC over the WAN. Administrators can view cached credentials and audit replication attempts.

Pre-seeding / pre-staging

• Run adprep /rodcprep on the schema master before creating RODCs.
• Pre-create (pre-stage) the RODC computer account in the domain and delegate the right for the local installer to use that account—this avoids having to contact a writable DC during promotion and supports air-gapped installs.

RODC limitations / roles they cannot hold

• RODC is read-only: cannot be a writable domain controller.
• RODCs cannot hold FSMO (schema, domain naming, RID, PDC emulator, infrastructure) roles.
• They should generally not be used as global writable DCs; certain operations (password changes, schema updates, some replication writes) are forwarded to writable DCs.

Security benefits for untrusted physical locations

• No writable AD database on site; reduces impact of theft/compromise.
• Only allowed credentials are cached, minimizing credential exposure.
• Local admins on the RODC can be delegated without domain-wide admin rights.
• Audit and logging on RODC helps detect local attacks.

AD DS replication behavior

• RODC receives inbound replication of AD partitions from writable DCs but does not forward write updates back.
• Passwords are replicated to RODC only when permitted by PRP and initiated by writable DCs.
• Metadata and objects replicate normally (read-only), but any write operation is rejected locally and must be processed by writable DCs.

This configuration gives branch-office performance and availability while constraining risk—use PRP and pre-seeding to strike the right balance between usability and security.

Direct answer

Encryption in transit protects data while it is moving between two points, for example your laptop and a website. Encryption at rest protects data while it is sitting in storage, for example on a server's hard drive. Both work the same basic way: the data is scrambled using a digital key, and only someone with the matching key can unscramble it back to something readable. HTTPS (the lock icon in a browser) is the everyday example of encryption in transit; a company laptop or database with disk encryption turned on is the everyday example of encryption at rest.

Structured elaboration

When I explain this to a stakeholder who is not technical, I make three deliberate choices:

1. Pick an analogy that survives the obvious follow-up question. "Scrambling data" invites "how do you unscramble it back?" so I go straight to a locked box with a key: the box (the data) can sit on a shelf (at rest) or travel in a delivery truck (in transit), and either way, only someone holding the matching key can open it. This analogy already answers the natural next question ("who has the key?") instead of dodging it.
2. Decide what to omit, not just simplify. I leave out algorithm names, protocol versions, and how the encryption keys themselves are generated and stored. Those details do not change the stakeholder's decision (do we need this, is it enough for compliance). What I keep is the one thing that matters to them: even if someone steals the disk or intercepts the network traffic, they get scrambled data they cannot use without the key.
3. Check understanding without quizzing them. Instead of asking "does that make sense?" (which invites a reflexive yes), I ask them to restate it in their own words, or pose a concrete scenario: "if someone stole this laptop from a parked car, what would they actually get?" Their answer tells me whether the concept landed.

Worked example

Here is close to what I would actually say:

"Think of our data like paperwork in a locked filing cabinet. 'Encryption at rest' means the paperwork sitting in that cabinet, meaning on our servers or backups, is written in a code that only a specific key can decode. If someone breaks into the building and steals the cabinet, they walk away with pages of gibberish. 'Encryption in transit' is the same idea for paperwork that is being carried from one office to another, meaning data moving between your browser and our servers. The little padlock icon you see next to a website address means that trip is protected the same way: even if someone intercepts the envelope mid-delivery, they cannot read what is inside. In both cases the 'key' is just a digital code that locks and unlocks the data. We keep that key separate from the data itself and tightly restrict who can use it, the same way you would not tape the safe combination to the safe."

If they ask a follow-up like "so is our data safe no matter what," that is the moment to add the caveat below rather than let the analogy imply more than it should.

Trade-offs and pitfalls

• The locked-box analogy breaks down around key management: a real safe has one physical key, but a digital key can be copied, and who is allowed to use it (and how that is audited) matters as much as the encryption itself. If the stakeholder is making a security or compliance decision, that caveat has to come back in, even though it complicates the clean story.
• Oversimplifying to "it's encrypted so it's safe" can create false assurance. Encryption at rest does not protect data while an application has it decrypted in memory for processing, and encryption in transit does not protect against someone who is logged in as an authorized user misusing their access. Naming that boundary once, briefly, is worth the extra sentence.
• Dropping all technical vocabulary can cost credibility with a stakeholder who has picked up some of it (for example, someone who has heard the term TLS from a vendor). It is fine to mention the term once, defined in one plain clause ("TLS, the technology behind that browser padlock"), so the explanation still connects to language they may encounter elsewhere.

Direct answer
A SYN flood works by sending a stream of SYN segments (often with spoofed source addresses) so the server allocates per-connection state for each one and replies with a SYN-ACK, but the final ACK never arrives, exhausting the server's backlog of half-open connections. SYN cookies let the server defer allocating any state at all until the final ACK actually shows up, so a flood of SYNs that never complete costs the server almost nothing.

Structured elaboration
Without SYN cookies, a normal server implementation stores an entry in its SYN backlog queue for every SYN it receives, holding onto that entry until the handshake completes or times out. A large flood of SYNs fills the backlog with half-open connections, so genuine clients' SYNs get silently dropped once the queue is full, that's the denial of service.

With SYN cookies enabled, once the backlog nears capacity, the server stops storing per-connection state up front. Instead, when it receives a SYN, it encodes the essential information it would normally have stored (effectively a hash of the source/destination IP, ports, and a secret, folded into the initial sequence number it sends back in the SYN-ACK) directly into its own sequence number field, and discards the SYN backlog entry immediately. If the final ACK ever arrives, the server checks that its acknowledgment number is the cookie value plus one, reconstructs the connection state on the spot, and only THEN commits real resources, at the exact moment a real, completing client shows up. If the SYN was part of the flood and no valid ACK ever comes, the server never allocated anything for it in the first place.

Worked example
From the legitimate client's point of view, the handshake looks completely normal: SYN, SYN-ACK, ACK, exactly as usual, it never knows cookies were involved. The difference is entirely server-side bookkeeping. The cost of the cookie scheme is that the server must give up storing some TCP options across the handshake (since there's no room left in a single 32-bit sequence number to also encode arbitrary options like the negotiated MSS (Maximum Segment Size) or window scale precisely), which is why some cookie implementations round MSS to one of a small handful of common values instead of preserving whatever exact value the client requested.

Trade-offs & pitfalls
SYN cookies alone do not stop a volumetric flood from consuming link bandwidth or CPU cycles processing the flood of SYNs in the first place, they only protect the connection-state table from being exhausted. A senior answer distinguishes "the handshake structure survives an attack that tries to exhaust connection state" (what SYN cookies solve) from "the network survives an attack that tries to exhaust bandwidth or CPU" (a separate problem requiring rate limiting, upstream filtering, or scrubbing, outside the scope of the handshake mechanism itself).

Direct answer

A Business Impact Analysis (BIA) is the exercise that translates "this business function is down" into a number the organization can act on: how much it costs per hour, in revenue, penalties, regulatory exposure, and customer harm, and how long the business can tolerate the outage before that harm becomes unacceptable. A continuity or risk manager typically commissions it, but the answers come from the business-function owners themselves. Its output (criticality tiers, tolerance windows, and recovery targets) becomes the backbone of everything downstream: which functions get recovered first, how much continuity budget each one justifies, and what the crisis team and exercise program actually rehearse.

Structured elaboration

What it measures. For each business function (not each application or server) the BIA asks: what breaks if this stops, who is affected, and how does the harm grow over time. That last part matters: the harm from a one-hour outage of payroll processing is trivial, the harm from a five-day outage is a legal problem, and the BIA is what turns that curve into a decision.

Who is involved.

Core outputs, defined at first use.

• Maximum Tolerable Period of Disruption (MTPD), sometimes called Maximum Allowable Outage (MAO): the longest the function can be down before the damage is unrecoverable for the business, not a technical estimate.
• Recovery Time Objective (RTO), from the business side: the target time by which the function must be working again, set below the MTPD with margin for the recovery process itself.
• Recovery Point Objective (RPO), from the business side: how much data loss (measured in time, e.g. "up to the last hour of transactions") the function can absorb.
• A criticality tier per function, usually 3 to 5 levels, used to rank recovery priority when resources are limited.

The BIA states these as business requirements. How a technical team meets a given RTO or RPO (through backup cadence, replication, or standby capacity) is a separate, downstream engineering decision, not something the BIA itself prescribes.

How the output is used downstream. The tiered list drives recovery sequencing (who gets resources first when multiple functions are affected at once), justifies continuity and resilience budget to leadership, defines the scope of the exercise program (a Tier 1 function is drilled more often than a Tier 4 one), and often becomes the evidentiary artifact regulators or auditors ask for first.

Worked example

A mid-size payments company runs its BIA on the "supplier payment processing" function. The finance lead reports the function processes about $2M/day in scheduled supplier payments, and that missed payments trigger a contractual late-payment interest charge of 1.5% per day on the unpaid balance. If a full day's batch is delayed, the direct cost is:

That number alone would suggest a loose tolerance, but the finance lead also flags that two of the company's largest suppliers have a contractual right to suspend shipment after 3 consecutive missed payment days, which would stop production, a much larger and harder-to-quantify impact. Leadership sets the MTPD at 2 business days on the strength of that qualitative flag, not the dollar figure alone, and the function is tiered as Tier 1 with an RTO of 4 hours. That tiering, RTO, and the underlying reasoning are the artifact that gets handed to the technical owners; how they achieve a 4-hour RTO is out of scope for the BIA itself.

Trade-offs and pitfalls

A BIA is frequently confused with a risk assessment: a risk assessment asks what could go wrong and how likely it is, while a BIA assumes the disruption already happened and asks how much it costs. Conflating the two produces a document that's neither useful for prioritization nor for threat planning. A second common failure is treating the BIA as a one-time deliverable rather than a living input that gets revisited when the business changes (new product lines, new regulatory exposure, an M&A integration); a BIA that's three years old is usually wrong. Finally, because the audience for this answer includes engineering roles, it's worth naming the trap directly: the BIA states what the business needs (a tolerance window and a target), not how to build it. An answer that jumps straight to backup and replication design has answered a different, narrower question than the one being asked here.

Direct answer

Detecting a SYN-flood attack means recognizing a specific signature: a very high rate of SYN packets with no matching completed handshakes, often from addresses that are spoofed or do not respond to the SYN-ACK at all. The immediate priority is distinguishing this from a legitimate traffic spike and applying mitigations that block the attack without also blocking real users.

Structured elaboration

1. Confirm the signature by counting half-open connections directly: ss -tan state syn-recv | wc -l (or the older netstat -n | grep SYN_RECV | wc -l) counts sockets stuck in SYN-RECV on the target host. Note that the aggregate ss -s summary line on current iproute2 versions (TCP: N (estab X, closed X, orphaned X, timewait X)) does not break its count down by state, so it will not show a SYN-RECV figure; use the explicit state filter (ss -tan state syn-recv), not ss -s, to get a state-specific count. A number that is unusually high and growing, combined with a packet capture showing many distinct source IPs each sending a single SYN with no follow-up ACK after the server's SYN-ACK, is the core signature of a SYN flood.
2. Distinguish spoofed from real sources: legitimate clients complete the handshake (their OS replies with an ACK) even under load; a flood using spoofed source addresses will show SYN-ACKs going out with no corresponding ACK ever arriving, because the spoofed address either does not exist or never sent the original SYN. A quick way to sanity-check this is whether the apparent source IPs in the capture cluster in ways real client populations would not (sequential or clearly fabricated ranges).
3. Apply immediate, reversible mitigations: enable SYN cookies at the OS/kernel level (sysctl -w net.ipv4.tcp_syncookies=1, after confirming the current value with sysctl net.ipv4.tcp_syncookies), which lets the server avoid committing state for half-open connections until the handshake completes; apply rate-limiting on SYN packets per source at the firewall with iptables, for example iptables -A INPUT -p tcp --syn -m limit --limit 1/s --limit-burst 3 -j ACCEPT followed by a default DROP for the remainder, which caps the rate of new SYNs accepted per interval; and, only as a last resort for the worst offenders, blackhole-filter specific source ranges that are unambiguously not legitimate customers.
4. Preserve legitimate traffic: before broadly rate-limiting, sample the traffic to estimate what fraction of current SYN volume is legitimate versus attack, so the rate limit is set above real demand rather than accidentally throttling real users during the incident.

Worked example

ss -tan state syn-recv | wc -l reports 40,000 sockets in SYN-RECV, versus a normal baseline of under 100 (the aggregate ss -s line would not have surfaced this number at all, since it does not report per-state counts). A capture shows SYN packets arriving at roughly 8,000 per second from source addresses that never appear again after the initial SYN (no ACK, no further traffic), consistent with spoofed sources. Enabling SYN cookies immediately stops the server from exhausting its connection-table memory on these half-open entries, since it no longer allocates state until the three-way handshake actually completes; concurrently, an iptables SYN rate limit (iptables -A INPUT -p tcp --syn -m limit --limit 1/s --limit-burst 3 -j ACCEPT, with a trailing DROP for anything over the limit) at the upstream firewall reduces the volume reaching the server without needing to identify or block specific IPs, which would be futile against spoofed sources anyway.

Trade-offs & pitfalls

SYN cookies have a real cost: they disable some TCP options (like window scaling, in implementations that cannot preserve them without the TCP timestamp option) for the connections they protect, which can silently degrade throughput for high-bandwidth-delay-product connections during the mitigation window; know this trade-off before enabling them by default rather than only during an active attack. Blackhole-filtering by source IP is close to useless against spoofed traffic (you would be blocking addresses that were never real senders) and should be reserved for cases where you have confirmed the sources are not spoofed. Also do not rely on the ss -s summary line to detect this: on current Linux tooling it reports only aggregate estab/closed/orphaned/timewait counts, not a SYN-RECV breakdown, so use the explicit state filter or netstat instead.

Direct answer

Build the policy as three cooperating Pluggable Authentication Module (PAM) stacks, quality enforcement, account lockout, and password history, defined once in the distribution's shared password stack rather than duplicated per service, so SSH inherits the same policy as console and su logins automatically. Use pam_faillock, the current, actively maintained lockout module, rather than pam_tally2, which it has replaced on modern Red Hat-family distributions. Test with a disposable account and a second, already-open administrator session before ever applying the policy broadly.

Structured elaboration

What each module does.

• pam_pwquality enforces quality rules on a new password at set time: minimum length, character-class variety, repeated-character limits, and similarity to the previous password, configured in /etc/security/pwquality.conf.
• pam_unix is the traditional Unix module: it verifies the password hash at login, and, in the password-change stack, computes and stores the new hash. Its remember=N option also enforces password history by keeping the last N hashes in /etc/security/opasswd, so a user cannot immediately reuse a recent password.
• pam_faillock tracks consecutive failed login attempts and enforces a temporary lockout after a configured threshold. pam_tally2 did the same job on older systems but is deprecated in favor of pam_faillock; naming the current module matters here since the question names both.

Configuration, in the shared stack file most Red Hat-family systems already use (password-auth, included by service-specific files like sshd, login, and su rather than each defining its own copy):

# /etc/security/pwquality.conf
minlen = 14
minclass = 3
maxrepeat = 3
maxsequence = 3
dcredit = -1
ucredit = -1
lcredit = -1
ocredit = -1
difok = 8
retry = 3
enforce_for_root

# /etc/pam.d/password-auth (relevant lines)
auth        required                                     pam_faillock.so preauth silent deny=5 unlock_time=900
auth        [success=1 default=ignore]                   pam_unix.so try_first_pass
auth        [default=die]                                pam_faillock.so authfail deny=5 unlock_time=900
auth        sufficient                                   pam_faillock.so authsucc deny=5 unlock_time=900
auth        required                                     pam_deny.so

account     required                                     pam_faillock.so
account     required                                     pam_unix.so

password    requisite                                    pam_pwquality.so retry=3
password    sufficient                                   pam_unix.so sha512 shadow use_authtok remember=10
password    required                                     pam_deny.so

session     required                                     pam_limits.so
session     required                                     pam_unix.so

# /etc/pam.d/sshd (relevant lines)
auth       substack     password-auth
account    required     pam_nologin.so
account    include      password-auth
password   include      password-auth
session    required     pam_selinux.so close
session    required     pam_loginuid.so
session    include      password-auth
session    required     pam_selinux.so open env_params

Password expiration (the one setting neither pam_pwquality nor pam_faillock enforces). Complexity, length, and history govern what a password can be; expiration governs how long any password, however strong, stays valid before it must change, and that comes from the aging fields in /etc/shadow that pam_unix's account-management phase checks on every login, not from either module configured above. Set the fleet-wide default in /etc/login.defs so every newly created account inherits it, then apply the same values to an existing account with chage, since login.defs only sets the default for accounts created after the change:

# /etc/login.defs (relevant lines)
PASS_MAX_DAYS   90
PASS_MIN_DAYS   1
PASS_WARN_AGE   14

# Apply the same policy directly to an existing account's shadow entry
chage -M 90 -m 1 -W 14 alice

PASS_MAX_DAYS 90 forces a change at least quarterly. PASS_MIN_DAYS 1 closes a gap the history setting alone leaves open: without a minimum age, a user could change their password twice back to back to cycle straight back to the one before, defeating remember=10 from the other direction. PASS_WARN_AGE 14 gives two weeks' notice before expiration forces a change, instead of a login that suddenly demands a new password with no warning.

Recommended settings and why each one exists (complexity, length, and history): minlen=14 and minclass=3 (requiring variety across at least three of the four character classes) drive real entropy rather than complexity theater. maxrepeat=3 blocks strings like aaaa1111. difok=8 requires enough of the new password to differ from the old one that a trivial increment does not pass. deny=5 unlock_time=900 locks an account for 15 minutes after five failed attempts, balancing brute-force resistance against a legitimate user's ability to eventually get back in without a help-desk call. remember=10 blocks the "change it, then change it right back" workaround by keeping the last ten hashes.

Integration with SSH. sshd's own PAM file does not define password or account logic from scratch. It includes the shared password-auth stack (auth substack password-auth, account include password-auth, password include password-auth). A policy change made once in password-auth therefore applies automatically to SSH logins, console logins, and su, without needing to be copied into every service file separately, which is also what keeps the policy from silently drifting out of sync between services over time.

Testing without locking out administrators. By default, pam_faillock's lockout does not apply to the root account unless the even_deny_root option is explicitly added, so the worst-case scenario is already guarded against, but this should be verified rather than assumed. Before rolling out further: keep a second, already-authenticated administrative session open on a separate connection, so a mistake in the new policy does not strand you without a way back in. Use faillock --user <test-account> to inspect a specific account's current failure count without needing a real failed login, deliberately fail logins against a disposable test account (never an admin account) to confirm the lockout actually triggers and the unlock_time countdown behaves as configured, then clear it with faillock --user <test-account> --reset. Roll out to one non-critical host first and confirm both lockout and recovery work end to end before applying it fleet-wide.

Worked example

The configuration above was checked with a complete, self-contained static syntax and ordering linter (copy-pasteable and runnable as shown, given the three files above saved alongside it as password-auth, sshd, and pwquality.conf). This does not exercise a live PAM authentication, there is no real login attempt against a real account here, it parses the exact lines shown above and verifies (a) every line is syntactically legal and (b) the pam_faillock/pam_unix/pam_pwquality lines are stacked in the order that actually enforces lockout-then-quality-then-history, which is the part hand-written PAM configs most often get wrong:

VALID_TYPES = {"auth", "account", "password", "session"}
SIMPLE_CONTROLS = {"required", "requisite", "sufficient", "optional", "include", "substack"}


def parse_line(line):
    line = line.split("#", 1)[0].strip()
    if not line:
        return None
    tokens = line.split()
    if len(tokens) < 2:
        raise ValueError(f"line has too few fields: {line!r}")
    ptype = tokens[0]
    if ptype not in VALID_TYPES:
        raise ValueError(f"unknown PAM type {ptype!r} in line: {line!r}")
    rest = tokens[1:]
    if rest[0].startswith("["):
        # bracketed control, e.g. [success=1 default=ignore]
        joined = " ".join(rest)
        end = joined.find("]")
        if end == -1:
            raise ValueError(f"unclosed bracketed control in line: {line!r}")
        control = joined[: end + 1]
        remainder = joined[end + 1 :].split()
    else:
        control = rest[0]
        if control not in SIMPLE_CONTROLS:
            raise ValueError(f"unknown control {control!r} in line: {line!r}")
        remainder = rest[1:]
    if control in ("include", "substack"):
        if len(remainder) != 1:
            raise ValueError(f"{control} must name exactly one stack file: {line!r}")
        return {"type": ptype, "control": control, "target": remainder[0], "args": []}
    if not remainder:
        raise ValueError(f"line has a control but no module path: {line!r}")
    module = remainder[0]
    args = remainder[1:]
    return {"type": ptype, "control": control, "module": module, "args": args}


def parse_stack(text):
    parsed = []
    for raw in text.splitlines():
        entry = parse_line(raw)
        if entry:
            parsed.append(entry)
    return parsed


def check_password_auth_ordering(parsed):
    auth_lines = [p for p in parsed if p["type"] == "auth"]
    modules_in_order = [p.get("module") for p in auth_lines]

    def first_index(pred):
        for i, p in enumerate(auth_lines):
            if pred(p):
                return i
        return None

    preauth_i = first_index(lambda p: p.get("module") == "pam_faillock.so" and "preauth" in p["args"])
    unix_i = first_index(lambda p: p.get("module") == "pam_unix.so")
    authfail_i = first_index(lambda p: p.get("module") == "pam_faillock.so" and "authfail" in p["args"])
    authsucc_i = first_index(lambda p: p.get("module") == "pam_faillock.so" and "authsucc" in p["args"])
    deny_i = first_index(lambda p: p.get("module") == "pam_deny.so")

    assert None not in (preauth_i, unix_i, authfail_i, authsucc_i, deny_i), (
        f"missing an expected auth-stack line: {modules_in_order}"
    )
    assert preauth_i < unix_i < authfail_i < authsucc_i < deny_i, (
        "faillock/pam_unix stack is out of order: preauth must run before pam_unix, "
        "authfail/authsucc must run after it, pam_deny must be last. Got order: "
        f"{modules_in_order}"
    )

    pwq_lines = [p for p in parsed if p["type"] == "password"]
    pwq = next(p for p in pwq_lines if p.get("module") == "pam_pwquality.so")
    assert pwq["control"] == "requisite", (
        "pam_pwquality.so must use control 'requisite' (immediate stop on a weak "
        "password), not 'required' (which would still run pam_unix.so and hash a "
        "password it just rejected)"
    )

    unix_pw = next(p for p in pwq_lines if p.get("module") == "pam_unix.so")
    assert "remember=10" in unix_pw["args"], "password-history line must set remember=N"


def main():
    with open("password-auth") as f:
        parsed = parse_stack(f.read())
    with open("sshd") as f:
        sshd_parsed = parse_stack(f.read())
    with open("pwquality.conf") as f:
        pwq_conf_lines = [l.split("#", 1)[0].strip() for l in f if l.split("#", 1)[0].strip()]

    print(f"password-auth: {len(parsed)} lines parsed, all syntactically valid PAM lines")
    print(f"sshd: {len(sshd_parsed)} lines parsed, all syntactically valid PAM lines")
    print(f"pwquality.conf: {len(pwq_conf_lines)} directives parsed")

    check_password_auth_ordering(parsed)
    print("ORDERING CHECK: preauth -> pam_unix -> authfail -> authsucc -> pam_deny : PASS")
    print("PWQUALITY CONTROL CHECK: pam_pwquality.so uses 'requisite' : PASS")
    print("PASSWORD HISTORY CHECK: pam_unix.so sets remember=10 : PASS")

    sshd_targets = {p["target"] for p in sshd_parsed if p["control"] in ("include", "substack")}
    assert "password-auth" in sshd_targets, "sshd stack must reference password-auth"
    print("SSHD INTEGRATION CHECK: sshd pam stack includes/substacks password-auth : PASS")

    print("\nALL_CHECKS=PASS")


if __name__ == "__main__":
    main()

Actual output from running this against the three files shown above:

password-auth: 12 lines parsed, all syntactically valid PAM lines
sshd: 8 lines parsed, all syntactically valid PAM lines
pwquality.conf: 11 directives parsed
ORDERING CHECK: preauth -> pam_unix -> authfail -> authsucc -> pam_deny : PASS
PWQUALITY CONTROL CHECK: pam_pwquality.so uses 'requisite' : PASS
PASSWORD HISTORY CHECK: pam_unix.so sets remember=10 : PASS
SSHD INTEGRATION CHECK: sshd pam stack includes/substacks password-auth : PASS

ALL_CHECKS=PASS

Two of these checks catch real, common hand-written mistakes: the ordering check would fail if pam_unix.so were placed before the preauth pam_faillock.so line (which breaks lockout counting), and the control check would fail if pam_pwquality.so used required instead of requisite (which would let a rejected weak password still reach pam_unix.so and get hashed anyway).

Trade-offs and pitfalls

• Using required instead of requisite for pam_pwquality.so is a subtle, common mistake. required still runs the rest of the stack even after a failure is recorded, so a rejected weak password could still reach pam_unix.so in some configurations. requisite stops immediately, which is what the policy actually needs.
• A lockout policy tested only against admin accounts risks looking safe when it is not, since even_deny_root defaults to off. Test against a disposable account specifically so the test reflects what an ordinary user will experience.
• Configuring the policy directly in each service's own PAM file, instead of the shared stack, invites drift, one service gets updated and another is quietly left on the old policy. The shared-stack approach exists specifically to prevent that.
• unlock_time set too aggressively (very long) turns a security control into a denial-of-service surface, an attacker can lock a real user out for the full window with a handful of deliberately failed attempts. 900 seconds balances brute-force resistance against that risk; a much longer value should be a deliberate choice, not a default left unexamined.