Penetration Tester Interview Preparation Guide - Spotify (Junior Level)

Direct answer

A legend that actually gets used has as few visual dimensions as possible, and each one carries exactly one meaning. I standardize on a small vocabulary (shape means component type, color means one thing like trust boundary or environment, line style means one thing like sync versus async) and I put a short label next to any icon that could plausibly mean two different things, rather than trusting the icon to speak for itself.

Structured elaboration

I organize the legend around a few categories, each with one job:

• Icons and shapes for component type. Rectangle for a compute service, cylinder for a data store, cloud outline for an external managed service, diamond for a decision or manual approval point. Each icon carries a short label with the actual service name and owning team, so the shape alone never has to carry the full meaning.
• Color for exactly one dimension. I pick one axis, most often trust level or environment (for example, green for internal, blue for customer-facing, orange for third-party), and I do not let color also imply something else like risk or status. Color-only meaning also fails for colorblind readers, so every color-coded element gets a redundant label or pattern, not color alone.
• Boundaries and grouping. A solid rounded box marks a deployment or service boundary; swimlanes mark team ownership. Arrow style is reserved for data flow semantics only: solid for synchronous calls, dashed for asynchronous or event-driven calls.
• A visible version and owner on every diagram. Diagrams drift out of date silently unless the legend itself forces a last-updated date and an owner to appear on the page.

The test I apply to every symbol before it goes in the legend: could two people in the room (one from security, one from product) each read this icon and land on a different meaning? If yes, it needs an explicit label, not just a prettier icon.

Worked example

Two genuinely ambiguous elements and how the legend resolves them:

1. An envelope icon on a connecting line. Read literally, this could mean a message queue or an actual email being sent. The legend resolves it by banning the bare envelope icon: a queue is drawn as a cylinder labeled with the actual technology ("Queue: Kafka"), and an outbound email is drawn as an external cloud icon labeled with the provider ("Email: SES"). No icon is left to carry that distinction alone.
2. A blue-colored box. Under a naive scheme, blue could mean "public-facing" or just "this team's color." The legend fixes the meaning: blue is reserved for customer-facing surfaces only, and it is always paired with a solid rounded border for "public-facing service." If a service is public but sits behind a web application firewall, that gets an explicit shield icon added rather than a new color, because color is only allowed to encode the one dimension it was assigned.

Trade-offs and pitfalls

• A notation system with too many dimensions (shape, color, border weight, icon, badge) is worse than a smaller one, because nobody memorizes six conventions; they revert to guessing, which is exactly the ambiguity the legend was supposed to remove. I keep the total vocabulary small enough to fit on one printed page.
• A legend that lives in a separate document from the diagrams decays fast: people update the diagram and forget the legend exists. Embedding the legend on the diagram itself, or enforcing it through a shared template in the diagramming tool, costs more up front but is the only version that survives six months of edits.
• Documenting a convention is not the same as enforcing it. Without a lightweight check (a template default, or a reviewer checklist item on architecture PRs), individual authors will quietly invent their own shorthand, and the legend becomes aspirational rather than actual.
• The legend has to match what the team's actual tool can render. A convention built for draw.io's rich icon set will not survive a move to Mermaid or another text-based diagram tool with a much smaller icon vocabulary, so the notation should be designed around the tool people will really use day to day.

Direct answer

As a tester, the first thing to establish is whether the comparison is naive: a byte-by-byte check with an early return, whether that is a raw == or a hand-written loop, leaks how many leading bytes of the submitted token matched before it diverged from the real hash-based message authentication code (HMAC), because a longer matching prefix means more comparison steps before the function returns false. That difference is small, but over enough repeated, carefully controlled measurements it is statistically distinguishable from ordinary network and server jitter, which is what turns "the tokens are measurably different in response time" into "I can recover the correct HMAC one byte at a time."

Structured elaboration

Detecting the side-channel. Submit a candidate token that is completely wrong (zero correct leading bytes) alongside candidates engineered to share a known, increasing number of leading bytes with a token you already know is valid for a different, disposable test case (or simply vary one byte position across many random guesses and look for a response-time pattern correlated with byte position rather than with the request itself). If the underlying comparison exits early on mismatch, the all-wrong candidate should be consistently faster on average than a candidate sharing several correct leading bytes, once you aggregate enough samples to see past ordinary variance.

The experiment to measure and confirm the leak. A single request pair proves nothing: internet and server-side jitter routinely dwarfs the microsecond-scale signal an early-exit comparison produces. Design the experiment as repeated, interleaved trials rather than a one-shot measurement: for each candidate (a fixed number of correct leading bytes), send several hundred to several thousand requests, record response latency for each, and summarize with a robust statistic (median or a trimmed mean) rather than a single sample or a raw average, since outliers from retries, garbage collection pauses, or momentary load spikes will otherwise swamp the result. Interleave requests across candidates rather than running one candidate to completion and then the next, so that a slow drift in overall server load during the experiment affects every candidate equally instead of biasing whichever one happened to run during a slow period. A genuine timing side-channel shows a reproducible, monotonic shift in the summary statistic as the number of correct leading bytes increases, checked across multiple independent experiment runs; noise does not reproduce that pattern consistently run over run.

High-level approach to recover the HMAC value. This is the standard remote/local timing side-channel technique against a non-constant-time verification (the class of attack formalized in "Remote Timing Attacks are Practical," Brumley and Boneh): rather than brute-forcing the entire secret at once, recover it one byte at a time. For the first byte position, try candidate values (up to 256 possibilities, fewer if the token's character set is constrained) while the remaining bytes are held arbitrary, and take the byte value whose average response time is measurably longer as the (probabilistically) correct byte, since the comparison walked one byte further into a match before returning false. Fix that byte and repeat for the next position. This converts an attack that is exponential in the secret's length (trying every possible value at once) into one that is linear in it: for a 32-byte HMAC-SHA256 digest, that is at most $32 \times 256 = 8192$ distinct byte-value hypotheses to test across all positions, each backed by many repeated measurements for statistical confidence, versus a keyspace of $256^{32}$ that is never going to be brute-forced directly. The linear bound is what makes an otherwise-infeasible secret recoverable at all.

Secure comparison recommendation. The fix is a constant-time comparison function that always inspects the full length of both inputs regardless of where they first diverge: hmac.compare_digest in Python, hmac.Equal in Go, or crypto.timingSafeEqual in Node.js. None of these are exotic; they are standard-library functions that exist specifically because this exact vulnerability class is well known, so recommend the language's built-in constant-time primitive rather than a custom implementation.

Responsible reporting. Document the methodology (experiment design, sample sizes, the statistical comparison used) rather than a specific measured time difference, since raw timing numbers are environment-specific and will not reproduce for whoever reads the report on different hardware or network conditions; what should reproduce is the pattern (response time correlates with correct-prefix length) and the underlying code defect (a non-constant-time comparison). Report through the organization's private disclosure channel, a security contact, security.txt, or a bug bounty program, rather than public disclosure first, and include a clear remediation recommendation (swap in the constant-time primitive named above) alongside the finding. Agree on a reasonable disclosure timeline that gives the engineering team time to both patch the comparison and rotate any secret that reporting suspects may have been partially exposed during testing, regardless of how far the recovery actually progressed before you stopped and reported it.

Worked example

This same detect-measure-exploit sequence applies to any HMAC comparison protecting a secret value, not only a session token comparison; the identical methodology recovers a webhook-signing secret's HMAC, an API request-signing HMAC, or a password-reset token's HMAC equally well, since the vulnerability is in how the comparison is implemented, not in what the compared value happens to represent. Consider testing a webhook-signature verification endpoint that checks an incoming X-Signature header against a locally computed HMAC-SHA256 using ==. Following the byte-at-a-time approach above, at most $32 \times 256 = 8192$ candidate byte-value hypotheses need to be tested to fully recover the 32-byte digest, each one backed by a batch of repeated, interleaved timing measurements rather than a single request. That bound holds regardless of whether the secret behind the HMAC is a session token, a webhook secret, or anything else, and the generalization is worth stating plainly: the vulnerability class is about the comparison operation, not the specific artifact being compared.

Trade-offs and pitfalls

This attack is far more practical on a low-latency path (same data center, internal network, or a test environment you control) than blind over the public internet, where added network hops, load balancers, and content delivery network (CDN) jitter can bury the signal under noise large enough that even thousands of samples per candidate fail to resolve it cleanly. A responsible test report should be explicit about which regime the finding was demonstrated in, and should not overstate remote exploitability if the proof of concept only succeeded under low-jitter conditions.

Mitigations exist at both the code level and the infrastructure level: the infrastructure layer matters too, though it never substitutes for the code fix. Rate limiting the endpoint under test slows an attacker's ability to gather the volume of samples this technique needs and buys detection time, and a gateway or service mesh that deliberately adds randomized response jitter raises the number of samples required to resolve the signal. Neither of these removes the underlying non-constant-time comparison; they raise the cost of exploiting it. The comparison itself still needs to be fixed, because a sufficiently patient or well-positioned attacker (or one who gains a lower-jitter vantage point, for example through a compromised machine on the same network segment) can still work around infrastructure-level noise given enough time and requests.

A tester who reports "a timing difference exists" without the aggregation-and-statistics discipline above risks a false positive: a single fast/slow pair of requests is very often just ordinary variance, not evidence of a real side-channel, and shipping that as a finding without the repeated, interleaved measurement methodology undermines the report's credibility even when the underlying code genuinely does have the defect.

Overview (one line)
I would map stakeholders, identify approvers, and set a clear pre-engagement cadence to ensure legal/operations alignment and safe, effective 4-week red team testing.

Stakeholders & why / when to include

• Executive Sponsor (CISO/Head of Security) — approves scope, risk acceptance; include at kickoff and post-report briefings.
• Legal / Privacy Counsel — ensures compliance with regulations, data handling, NDAs; include before scope finalization.
• CTO / VP Engineering — technical constraints, blackout windows, emergency contacts; include during scoping and weekly syncs.
• IT / Infrastructure Ops / Cloud Ops — provide network diagrams, whitelist test infrastructure, coordinate remediation; include before tests start and daily during high-impact actions.
• App/Product Owners — define in-scope assets and business-critical systems; include during scoping and for vulnerability validation.
• Incident Response / SOC — prepare for detection/escallation, avoid false positives; include pre-kickoff and on-call during exercise.
• DevSecOps / CI-CD Owners — for rollbacks and rapid fixes; include when tests target pipelines.
• HR / Physical Security (if social-engineering or on-site) — legal and safety coordination; include before any social or physical attempts.
• Finance/Compliance (if payments/data involved) — assess business impact; include at scoping.

Decision-makers (approval)

• CISO (primary) — final go/no-go, authorizes rules of engagement.
• Legal Counsel — signs off on contracts/NDA and data handling.
• CTO / COO — operational approval for potential service impact.

Pre-engagement communication cadence

• Week -2: Initial scoping meeting with CISO, Legal, CTO, IR, Product owners — confirm objectives, exclusions, SLAs.
• Week -1: Deliver Rules of Engagement (RoE), technical inventory, blast-radius plan; Legal & Exec sign-off.
• Day -3: Operations & SOC tabletop / contact list, escalation playbook.
• Day -1: Final confirm (email + short call) with on-call contacts and blackout windows.
• Daily during exercise: 15-min standup with SOC/ops for any active impacts; ad-hoc escalation as needed.
• Post-engagement (within 72 hrs): Executive debrief and remediation roadmap meeting.

This cadence minimizes business disruption, ensures legal/compliance coverage, and gives me the operational support needed to run realistic, safe red team operations.

Approach (why attack graphs help)
Attack graphs map how individual vulnerabilities chain to reach high-value assets; they let you prioritize fixes by impact on attacker paths instead of CVSS alone. As a penetration tester I use them to show exploitability in context and recommend remediation that breaks the most dangerous paths.

Required inputs

• Asset inventory with sensitivity tags (Crown Jewels)
• Network topology and trust relationships (ACLs, segmentation)
• Vulnerability list with exploitability data (CVSS, Metasploit modules, PoC availability)
• Authentication/privilege flows (local creds, service accounts)
• Existing mitigations (WAF, EDR, MFA)
• Likelihood priors (external vs internal access)

Example scoring method

• For each attack path p containing vulnerabilities v1..vn compute path score as:

PathScore_p = ReachabilityMultiplier_p * (1 - ∏_{i=1..n} (1 - ExploitProb_vi))

• Where ExploitProb_vi = normalized exploitability (0..1) from evidence (PoC=0.9, public exploit=0.7, theoretical=0.2) and ReachabilityMultiplier_p = 1.0 for crown-jewel target, 0.6 for sensitive, 0.3 for low-value.
• Translate PathScore into per-vuln priority by summing contributions of each vuln across all paths:

VulnPriority_v = Σ_{p contains v} PathScore_p * (ImpactWeight_p / n_p)

(ImpactWeight higher for paths requiring fewer preconditions; n_p = number of vulns in path)

Why this works

• Accounts for chaining (compound exploit probability) and target criticality.
• Penalizes long low-likelihood chains, elevates single-step high-impact exploits.

Presentation to engineering

• One-page heatmap: vulnerabilities ranked with color bands, top-10 “break-the-path” items that when remediated reduce overall PathScore by X% (show before/after totals).
• For each top item include: remediation action (patch/segmentation/MFA), required effort, rollback risk, and expected reduction in attack surface (% of high-risk paths broken).
• Use an accompanying attack-graph diagram showing representative paths (annotated with exploit types and preconditions) so engineers see concrete attacker sequences to disrupt.

Actionable recommendation style

• Immediate: quick wins (MFA, EDR tuning, network ACLs) with low effort and high PathScore reduction.
• Short-term: patch/upgrade high-priority CVEs.
• Mid-term: segmentation, credential hygiene, and compensating controls to prevent future chaining.

This method gives engineering prioritized, measurable, and explainable remediation steps focused on reducing actual attacker avenues.

Direct answer
The TCP three-way handshake establishes a reliable connection before any data flows: the client sends a SYN, the server replies with a combined SYN-ACK, and the client finishes with an ACK. Its job is to let both sides agree on starting sequence numbers and confirm that both directions of the path actually work before committing application data to the wire.

Structured elaboration

1. SYN: the client picks an initial sequence number (ISN, essentially a large pseudo-random 32-bit number) and sends a segment with the SYN flag set and that sequence number. The client moves to the SYN-SENT state.
2. SYN-ACK: the server, if it's listening on that port, picks its OWN initial sequence number, and replies with a segment that has both the SYN flag set (announcing the server's own sequence number) AND the ACK flag set (acknowledging the client's sequence number + 1). The server moves to the SYN-RECEIVED state.
3. ACK: the client acknowledges the server's sequence number + 1 with a plain ACK segment. Both sides now move to ESTABLISHED, and either side may now send data.

Why three steps rather than two: TCP needs BOTH sides' sequence numbers acknowledged, since TCP is full-duplex (both directions need independent sequence tracking). A two-way handshake could confirm only one direction; the third message is what confirms the client's original SYN actually arrived, closing the loop for the client's own sequence space.

Worked example
Suppose a client opens a TCP connection to a web server on port 443. The client sends SYN, seq=1000. The server responds SYN, ACK, seq=5000, ack=1001 (acknowledging the client's SYN by number+1). The client responds ACK, seq=1001, ack=5001. From this point, the client's next data byte will carry sequence number 1001, and the server's next data byte will carry sequence number 5001; each side tracks the OTHER side's sequence space independently via the ACK field of every following segment.

Trade-offs & pitfalls
A frequent mistake is describing the handshake as three round trips; it's actually one and a half round trips of latency, because the SYN-ACK piggybacks the server's SYN onto its ACK of the client's SYN. This is also exactly why TCP always incurs at least one round trip of setup latency before any data can flow, which is the whole motivation behind newer mechanisms like TCP Fast Open that try to send data alongside the very first SYN.

Direct answer

Before I trust my own application of a technique nobody in-house has used, I deliberately design a check that would catch me being wrong, usually by running it against a case where I already know the right answer, and I only report a result to people relying on it alongside an honest statement of what that validation did and did not cover. Trust here comes from actively trying to break my own understanding and failing, not from the technique simply producing an answer that looks reasonable.

Structured elaboration

• Before applying the new technique to the real problem, find or construct a case with a known answer, synthetic data with a known effect, a smaller version of the problem you can verify by hand, or a case where an established method already gives a trusted answer, and confirm the new technique recovers it.
• Sanity-check the assumptions the method actually requires, not just whether it runs; many techniques silently produce an output even when their assumptions are violated.
• Design the validation to specifically target where you are least confident, not the part you already understand well; a check that only confirms what you already believed is not doing much work.
• Communicate confidence and limitations in terms the audience can actually evaluate, what was tested, what was not, and what would change your confidence, rather than a bare number.

Worked example

Needed to estimate the causal effect of a new onboarding flow on retention using a technique nobody on the team had used before, synthetic control (a method that builds an artificial comparison group from a weighted blend of untreated units to estimate what would have happened without the change), for a business case going to leadership. Before touching the real question, I built a known-answer test: I took a metric with an already-established, trusted causal estimate from a past well-instrumented randomized test, reconstructed it using the new technique on the same historical data, and confirmed the synthetic-control estimate landed close to the randomized-test answer. That gave a concrete reason to trust the method beyond it having run and produced a chart. Before applying it to the real question, I explicitly checked the assumption the method depends on, that the synthetic control's pre-period trend actually tracked the treated group closely, rather than assuming it did because the model converged. When presenting to leadership, I stated plainly what had been validated, the method recovering a known answer on a comparable case, and the pre-trend assumption holding reasonably well here, and what had not, a small sample size for the treated group that widens the honest uncertainty, rather than presenting one confident number.

Trade-offs and pitfalls

• Treating "the code ran and produced an output" as proof of correctness is the most common way a newly learned technique gets misapplied; a known-answer check is what actually earns trust.
• Skipping the assumption check because the output looks reasonable is dangerous specifically because a technique can produce a plausible-looking wrong answer when its assumptions are violated.
• Presenting a confident single number to an audience that cannot independently evaluate the method, without naming what was and was not validated, sets up a false sense of certainty that is hard to walk back later.

Scope-Governance Model (Penetration Test Program)

1. Objectives & Baseline Scope

• Define per-business-unit (BU) scope: assets, IP ranges, test types (external/internal/web/mobile), success criteria, blackout windows.
• Produce a signed Scope Statement and Rules of Engagement (RoE) before work starts.

2. Approval Gates

• Gate A — Kickoff Approval: BU Sec Owner + Program PM + Testing Lead sign off baseline RoE.
• Gate B — Pre-Execution: Risk/Change board verifies no conflicting projects, approves schedule.
• Gate C — Post-Change: If change impacts risk or timeline, Program Steering approves revised scope.

3. Change-Request Process

• Submit CR form: requester, rationale, affected assets, urgency, proposed activities.
• Triage within 48 hours by Testing Lead: classify as Minor (<=4 hours, patch window friendly) or Major (>4 hrs / new assets / higher privilege).
• For Minor: Testing Lead + BU Sec Owner approve.
• For Major: Program PM + BU Sec Owner + Risk Board review, then Gate C approval.

4. Time & Cost Impact Assessment

• Estimation template: effort (hours), resource type (senior/junior), tooling/licensing, scheduling shifts, detection/retest iterations.
• Calculate delta: additional hours × hourly rates + tooling + potential retest (fixed matrix).
• Include risk buffer (15%) for unknowns.

5. Communication

• CR acknowledgement within 24h, decision within SLA (48h minor / 5 business days major).
• Updated scope, cost, and timeline communicated via change memo to stakeholders and reflected in project tracker and invoice forecasts.
• Weekly program dashboard: scope status, open CRs, approved changes, impact metrics (hours, cost, delay days).

Example: adding an internal AD engagement mid-program is Major — triage, estimate 40 hrs (2 testers), cost delta, retest cycle; present to Risk Board for Gate C approval before execution.

Brief framing (tester perspective)
As a penetration tester I must design engagements that comply with relevant laws/regulations — they shape scope, evidence retention, consent, reporting, and contractual clauses.

Key laws/regulations and impact

• GDPR (EU): Limits processing of personal data — avoid unnecessary PII collection, use data minimization, pseudonymize/test on synthetic data, and include Data Protection Impact Assessment (DPIA) clauses in contracts.
• HIPAA (US, healthcare): Protects PHI — requires Business Associate Agreements (BAA), strict logging, encrypted evidence storage, and limited scope to systems holding PHI.
• CFAA / Computer Misuse Act (US/UK): Criminalizes unauthorized access — mandates explicit written authorization (timebound, IPs, accounts), approved test windows, and escalation procedures to avoid legal exposure.
• PCI-DSS: Protects cardholder data — requires approved testing methodologies, controls around handling PANs, and evidence retention policies compliant with PCI requirements.
• NIS2 (EU): Critical infrastructure security — may require notification of incidents discovered during testing, higher assurance levels for critical sectors, and formal reporting to competent authorities.
• Local Computer Crime laws (e.g., UK Computer Misuse Act): May impose extra constraints on exploit development or persistence tests; include indemnity and clear rollback/restore plans.

Practical actions

• Include legal authorization, scoping, allowed techniques, data handling, and reporting timelines in the Statement of Work.
• Coordinate with legal/clients for BAAs, DPIAs, and breach-notification planning.

Scope & safety first

• I clarify scope, get written authorization, list systems allowed, and require an incident contact and kill-switch. I state the goal: prove the vulnerability without touching production or real PII.

Provision isolated test environment

1. Reproduce topology with Infrastructure-as-Code (Terraform/CloudFormation) using separate account/VPC, subnet ACLs, and no connectivity to prod.
2. Use ephemeral hosts (Terraform + Ansible) and immutable images (Packer) so the environment is rebuildable.
3. Apply network controls: egress deny-by-default, jumpbox for controlled access, and monitoring-only span ports to capture traffic.

Generate synthetic production-like data

• Extract schema and field formats from prod metadata (NOT data). Create synthetic dataset with realistic distributions (e.g., faker, Faker-Providers, or custom Python scripts) preserving field lengths, value entropy, relationships, and edge cases (long strings, nulls, unicode). Mask any sample values; never import real PII.

Produce replayable artifacts

• Version IaC and configuration in Git. Record exploit steps as scripts and automated playbooks. Capture network traffic (pcap), system logs, application request/response traces, and full container images or VM snapshots. Store artifacts with timestamps and checksums and a README to replay locally.

Verification & remediation proof

• Define test cases: pre-exploit baseline, exploit execution (with expected indicators), and post-remediation rerun. For each, capture artifacts and measurable assertions (e.g., unauthenticated access returned 200 with payload X; after patch, same request returns 401/blocked). Include automated tests (pytest or custom scripts) that assert vulnerability presence/absence.

Controls & cleanup

• Time-box tests, use monitoring alerts to detect anomalies, revoke credentials after use, and destroy environment automatically via IaC. Provide a replay guide and remediation recommendations (config fixes, patches, compensating controls) and include risk impact and CVSS-like scoring.

This plan proves the issue reproducibly, protects production, and gives engineers clear artifacts and pass/fail verification for remediation.

Executive summary (from a penetration tester’s perspective)
I recommend funding: annual external penetration tests, biannual/red-team (adversary simulation) exercises, and continuous secure development training. Together these reduce breach likelihood, improve detection/response, and harden code — yielding tangible cost avoidance and faster remediation.

Expected benefits

• Reduce probability of critical breaches through proactive remediation.
• Faster detection and containment (improved MTTD/MTTR).
• Fewer production incidents, lower regulatory fines, and preserved customer trust.
• Stronger security culture across engineering teams.

High-level estimated costs (annual)

• External application + infra penetration tests: $40k – $80k
• Red team (targeted exercises + purple-team follow-up): $120k – $250k
• Secure development training + labs for devs (per 100 devs): $25k – $50k
  Total: $185k – $380k

Key assumptions

• Company handles sensitive customer data and faces compliance obligations.
• Baseline security posture has medium risk (some known gaps).
• Engineering capacity to remediate prioritized findings within SLAs.

Metrics to measure success

• of critical/high findings per engagement (trend down)

• Mean time to remediate critical findings (target <30 days)
• MTTD / MTTR improvements vs baseline
• % of devs completing training and post-training secure code assessment scores
• Number of findings validated closed by follow-up tests

ROI & timeline

• Initial hardening and training: 3–6 months.
• Measurable reduction in critical findings and MTTR: 6–12 months.
• Break-even via avoided breach costs and reduced incident losses: typically 12–18 months (depending on breach-cost assumptions). Even a single avoided major incident (~$500k–$3M avoided) justifies annual spend.

I can provide a tailored cost/benefit model using our current incident history and threat profile to refine ROI.