Penetration Tester Interview Preparation Guide - Spotify (Junior Level)

Direct answer

Treat a cryptographic library-misuse audit as two separate lanes that feed each other: static analysis to find suspicious call sites at scale, and dynamic, test-vector-driven analysis to prove which of those sites are actually exploitable. Then triage findings by exploitability and blast radius, not by file order or how alarming a finding looks in isolation.

What "misuse" looks like

• Wrong mode selection: ECB mode used for anything beyond a single block. ECB encrypts identical plaintext blocks to identical ciphertext blocks, so repeated structure in the plaintext (a bitmap, a repeated field) leaks straight through the ciphertext.
• Improper padding: a custom padding-validation routine whose error path (message, timing, or response code) differs depending on whether the padding was well-formed, the classic padding-oracle shape that lets an attacker decrypt ciphertext byte by byte through repeated queries.
• Insecure defaults: a library or wrapper that ships a default IV of all zeros, a default weak cipher, or a key size that's fine for a demo but not for production, and a caller who never overrides the default inherits the weakness silently.

Static analysis

Pattern- and AST-based scanners (for example Semgrep with a crypto-focused ruleset, or a language's own security linter such as Bandit for Python) can flag known-bad call shapes directly: Cipher(..., modes.ECB()), a broken digest like MD5 used outside a non-security checksum context, or a byte-array key or IV that's a literal constant in source. Secret-scanning tools (gitleaks, truffleHog) catch hardcoded keys specifically. General static analysis (CodeQL and similar) adds language-wide security query packs beyond crypto-specific ones.

Dynamic and test-vector analysis

Run the code path that actually performs encryption/decryption against known-answer test vectors, Google's Project Wycheproof is built exactly for this: it contains adversarial test vectors (weak IVs, invalid curve points, edge-case moduli) specifically designed to catch the misuse classes above rather than just confirming a library works on "normal" input. Pair that with differential testing against a reference implementation on the same inputs, and fuzz the parsing and validation code paths that handle ciphertext or padding, since those are exactly where a padding-oracle-shaped bug tends to live.

Prioritizing remediation

Score every finding on two axes: exploitability (is the vulnerable path reachable by an unauthenticated network attacker, or only by someone who already has local code execution) and impact (does it break confidentiality of data at rest, or fail loudly and safely). Fix network-reachable, confidentiality-breaking findings first regardless of how small or old the code looks; track lower-exploitability findings on a remediation backlog with a deadline weighted by that same exploitability score, rather than by severity label alone.

Worked example

The ECB pattern leak, demonstrated directly:

from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
import os

key = os.urandom(32)
block = b"AAAAAAAAAAAAAAAA"      # one AES block (16 bytes), repeated
plaintext = block * 4

encryptor = Cipher(algorithms.AES(key), modes.ECB()).encryptor()
ciphertext = encryptor.update(plaintext) + encryptor.finalize()

blocks = [ciphertext[i:i + 16] for i in range(0, len(ciphertext), 16)]
print("all four ciphertext blocks identical:", blocks[0] == blocks[1] == blocks[2] == blocks[3])

Output:

all four ciphertext blocks identical: True

Four identical plaintext blocks produce four identical ciphertext blocks under ECB, exactly the structural leak that static-analysis rules are built to flag before the code ever ships.

Trade-offs and pitfalls

Static analysis alone produces false positives (a "bad" call inside a well-reviewed wrapper that's actually fine) and false negatives (a config-gated insecure default that's only reached through a rarely-used flag), so treat its hits as leads, not verdicts. Test-vector and fuzzing analysis alone will miss anything the vectors and fuzz corpus don't exercise. And both together still miss pure logic-level misuse, the right function called with the wrong parameters, such as a correct AEAD (Authenticated Encryption with Associated Data) call with a caller-supplied fixed nonce, which usually needs a manual review pass on the highest-risk call sites even after tooling runs clean.

Direct answer

A legend that actually gets used has as few visual dimensions as possible, and each one carries exactly one meaning. I standardize on a small vocabulary (shape means component type, color means one thing like trust boundary or environment, line style means one thing like sync versus async) and I put a short label next to any icon that could plausibly mean two different things, rather than trusting the icon to speak for itself.

Structured elaboration

I organize the legend around a few categories, each with one job:

• Icons and shapes for component type. Rectangle for a compute service, cylinder for a data store, cloud outline for an external managed service, diamond for a decision or manual approval point. Each icon carries a short label with the actual service name and owning team, so the shape alone never has to carry the full meaning.
• Color for exactly one dimension. I pick one axis, most often trust level or environment (for example, green for internal, blue for customer-facing, orange for third-party), and I do not let color also imply something else like risk or status. Color-only meaning also fails for colorblind readers, so every color-coded element gets a redundant label or pattern, not color alone.
• Boundaries and grouping. A solid rounded box marks a deployment or service boundary; swimlanes mark team ownership. Arrow style is reserved for data flow semantics only: solid for synchronous calls, dashed for asynchronous or event-driven calls.
• A visible version and owner on every diagram. Diagrams drift out of date silently unless the legend itself forces a last-updated date and an owner to appear on the page.

The test I apply to every symbol before it goes in the legend: could two people in the room (one from security, one from product) each read this icon and land on a different meaning? If yes, it needs an explicit label, not just a prettier icon.

Worked example

Two genuinely ambiguous elements and how the legend resolves them:

1. An envelope icon on a connecting line. Read literally, this could mean a message queue or an actual email being sent. The legend resolves it by banning the bare envelope icon: a queue is drawn as a cylinder labeled with the actual technology ("Queue: Kafka"), and an outbound email is drawn as an external cloud icon labeled with the provider ("Email: SES"). No icon is left to carry that distinction alone.
2. A blue-colored box. Under a naive scheme, blue could mean "public-facing" or just "this team's color." The legend fixes the meaning: blue is reserved for customer-facing surfaces only, and it is always paired with a solid rounded border for "public-facing service." If a service is public but sits behind a web application firewall, that gets an explicit shield icon added rather than a new color, because color is only allowed to encode the one dimension it was assigned.

Trade-offs and pitfalls

• A notation system with too many dimensions (shape, color, border weight, icon, badge) is worse than a smaller one, because nobody memorizes six conventions; they revert to guessing, which is exactly the ambiguity the legend was supposed to remove. I keep the total vocabulary small enough to fit on one printed page.
• A legend that lives in a separate document from the diagrams decays fast: people update the diagram and forget the legend exists. Embedding the legend on the diagram itself, or enforcing it through a shared template in the diagramming tool, costs more up front but is the only version that survives six months of edits.
• Documenting a convention is not the same as enforcing it. Without a lightweight check (a template default, or a reviewer checklist item on architecture PRs), individual authors will quietly invent their own shorthand, and the legend becomes aspirational rather than actual.
• The legend has to match what the team's actual tool can render. A convention built for draw.io's rich icon set will not survive a move to Mermaid or another text-based diagram tool with a much smaller icon vocabulary, so the notation should be designed around the tool people will really use day to day.

An advanced persistent threat (APT) emulation is scoped and run differently from a generic red team: instead of "find a path to the crown jewels," the objective is faithfully reproducing a specific threat actor's known tactics, techniques, and procedures (TTPs), usually grounded in a MITRE ATT&CK-mapped profile, so the client learns whether their detections catch that adversary specifically, not red-team tradecraft in general.

Command-and-control (C2) topology

Design a primary channel plus at least one fallback on a materially different protocol or provider, so a single detection doesn't end the emulation. Run redirectors (throwaway servers that forward C2 traffic so the real team server stays hidden) in front of the team server(s) (the machine operators run the operation from) in each region being emulated, since a real multi-region APT typically operates infrastructure geographically close to its targets, and tune beaconing jitter and sleep intervals (beaconing is the implant, meaning the agent running on a compromised host, calling back to the operator on a schedule; jitter randomizes that schedule; and the sleep interval is the time between call-backs) to blend into that region's normal traffic patterns rather than optimizing for operator convenience.

flowchart LR
    OP[Operator] --> TSP[Team Server Primary]
    TSP --> RRA[Redirector Region A]
    TSP --> RRB[Redirector Region B]
    RRA --> IRA[Implants Region A]
    RRB --> IRB[Implants Region B]
    TSF[Team Server Fallback, different provider] -.fallback.-> RRA
    TSF -.fallback.-> RRB

One OPSEC (operational security) note worth building into the plan explicitly: domain fronting, disguising C2 traffic behind a trusted content delivery network (CDN) hostname, is a technique many real threat actors used through roughly 2018, but AWS and Google both blocked classic domain fronting on their platforms that same year; Azure's exposure to the same technique has historically lagged AWS's and Google's, and whether it is closed on any given target's current infrastructure should be verified at engagement time rather than assumed. An emulation plan that leans on domain fronting as a live OPSEC technique today is emulating a capability the actor itself would likely have lost; call that out explicitly in the report rather than quietly substituting a workaround, since it's a genuine finding about how the threat model has aged.

Credential handling

Mirror the actor's documented credential-access preference, ticket-based Kerberos abuse in a Windows-heavy environment, or cloud-token theft against a cloud-native target, rather than defaulting to whatever's fastest. Store every captured credential in the engagement's encrypted evidence store, never on a compromised host, with an explicit destruction date.

Lateral movement plan

Sequence movement to match the actor's known dwell-time and pacing rather than moving as fast as technically possible; realistic pacing is part of what's being tested, since a security operations center (SOC) tuned to catch a fast, noisy actor may miss a patient one, and surfacing that gap is exactly the point of an APT emulation.

Persistence lifecycle

Match the actor's documented persistence style, some real threat groups favor lightweight, easily-abandoned mechanisms, others invest in durable, hard-to-remove footholds, and regardless of style, log every mechanism the moment it's created so the compressed two-week window doesn't erode the same cleanup discipline a longer engagement would use.

Logging and evidence preservation

Keep a continuous, timestamped operator log mapped to MITRE ATT&CK tactic categories (for example initial access, persistence, privilege escalation, defense evasion, credential access, lateral movement, and command and control) so the after-action report shows the client, technique by technique, exactly which of the emulated actor's known behaviors were and weren't detected.

Coordination with SOC and incident response (IR) to avoid unintended business disruption

Given the compressed two-week timeline across multiple regions, assign one trusted agent per region rather than a single person covering all regions, each with the authority to invoke the kill switch locally without waiting on a central approval chain the timeline can't afford to lose time to.

Measuring success

Success here is not "did the team compromise the crown jewels," it's fidelity to the emulated actor's TTPs plus a clear map of which of those TTPs the SOC detected, at what stage, and with what dwell time. Report both; perfect emulation with zero detection is the most useful, if uncomfortable, outcome, not a failed one.

Safeguards

The same core set as any production engagement, scaled to the compressed timeline and multi-region footprint: named kill-switch authority per region, an evidence and credential-destruction policy, a hard blacklist of destructive actions regardless of whether the emulated actor is known to use them, and a daily cross-region sync between operator cells so a detection or incident in one region reaches the others within minutes, not at the next scheduled check-in.

Trade-offs and pitfalls

The most common mistake is running a generic red team and relabeling it "APT emulation" afterward; without grounding every technique choice in the actor's actual documented behavior, from threat intelligence rather than assumption, the client learns nothing about readiness against that specific threat. A two-week window across multiple regions is tight enough that under-resourcing trusted-agent coverage, one person for every region, is a real risk to fast, safe kill-switch response; staff it per region even if it costs more. And citing an OPSEC technique the real actor is documented to have used, without checking whether it still works against current infrastructure, domain fronting being the clearest example, teaches the client a stale threat model.

Zero-downtime rotation for a credential consumed both by CI pipelines and long-running production services means the rotation has to be a coordinated sequence, not a single instantaneous swap, because a service that's still holding the old credential in memory will fail the instant it's invalidated if the new one isn't already available first.

The rotation sequence

1. Issue the new credential alongside the old one, both valid simultaneously, rather than replacing the old value in place. Most managed database and secrets-manager rotation flows support exactly this dual-validity window.
2. Update the secrets store (Vault, or the cloud secrets manager) with the new credential, so any NEW process that starts picks it up automatically.
3. Roll dependent services so each one picks up the new credential, either via a rolling restart (for services that only read the credential at startup) or via a hot-reload mechanism (for services designed to periodically re-fetch it without restarting). This step has to reach every consumer, including any CI pipeline that has the old credential cached in a build-time environment variable, not just the running production services.
4. Verify that all consumers are confirmed on the new credential (via a health check, or a log signal confirming the new credential was used successfully) before proceeding.
5. Only then invalidate the old credential. Invalidating before every consumer has confirmed the switch is what causes an avoidable outage.

Verification and rollback

Verification means checking, not assuming: query each service's health endpoint or logs to confirm a successful authentication with the new credential specifically, rather than just confirming the rotation script completed without error. If a service fails to pick up the new credential (a bug in its hot-reload path, or a stale cached connection pool), the rollback is to NOT invalidate the old credential yet, extend the overlap window, and fix the specific consumer before completing the cutover; the overlap window is what makes rollback possible at all, since a design with no overlap has no way back once the old credential is gone.

Auditability

Every rotation event needs its own durable audit record, independent of whatever the secrets manager's default logging captures: which credential was rotated, who or what triggered it (a scheduled job versus a manual emergency trigger), the timestamp of each step in the sequence above (issue, store update, per-consumer rollout, verification, old-credential invalidation), and the outcome of the verification step for each consumer. This record needs to answer, after the fact, two questions a compliance or incident review will always ask: "was this credential rotated on the schedule the policy requires" and "did every consumer actually confirm the switch, or did the rotation invalidate the old credential based on an assumption." Logging only that the rotation script exited successfully is not sufficient auditability, since a script can exit cleanly while a specific consumer silently failed to pick up the new credential.

Emergency compromised-key response

When the rotation is because a key was compromised, not routine, the ordinary dual-validity sequence above changes: you can't safely leave the compromised key valid for a rollout window, so the priority inverts to "invalidate immediately, accept the availability risk" rather than "wait for every consumer to confirm before invalidating." This means an emergency rotation needs its own faster procedure, pre-tested and separate from the routine rotation runbook, since improvising a faster path during an active compromise is worse than having practiced it beforehand. The audit record for an emergency rotation additionally needs to capture why it was classified as an emergency (the specific compromise indicator), since that classification is itself something a later review will scrutinize.

Trade-offs

The dual-validity window is itself a small window of extra risk (the old, soon-to-be-retired credential is still usable by anyone who might have it during that window), which is acceptable for a routine, scheduled rotation but is exactly the risk an emergency compromised-key rotation cannot accept, which is why the two scenarios need genuinely different procedures rather than one procedure with a shortcut.

Direct answer
The TCP three-way handshake establishes a reliable connection before any data flows: the client sends a SYN, the server replies with a combined SYN-ACK, and the client finishes with an ACK. Its job is to let both sides agree on starting sequence numbers and confirm that both directions of the path actually work before committing application data to the wire.

Structured elaboration

1. SYN: the client picks an initial sequence number (ISN, essentially a large pseudo-random 32-bit number) and sends a segment with the SYN flag set and that sequence number. The client moves to the SYN-SENT state.
2. SYN-ACK: the server, if it's listening on that port, picks its OWN initial sequence number, and replies with a segment that has both the SYN flag set (announcing the server's own sequence number) AND the ACK flag set (acknowledging the client's sequence number + 1). The server moves to the SYN-RECEIVED state.
3. ACK: the client acknowledges the server's sequence number + 1 with a plain ACK segment. Both sides now move to ESTABLISHED, and either side may now send data.

Why three steps rather than two: TCP needs BOTH sides' sequence numbers acknowledged, since TCP is full-duplex (both directions need independent sequence tracking). A two-way handshake could confirm only one direction; the third message is what confirms the client's original SYN actually arrived, closing the loop for the client's own sequence space.

Worked example
Suppose a client opens a TCP connection to a web server on port 443. The client sends SYN, seq=1000. The server responds SYN, ACK, seq=5000, ack=1001 (acknowledging the client's SYN by number+1). The client responds ACK, seq=1001, ack=5001. From this point, the client's next data byte will carry sequence number 1001, and the server's next data byte will carry sequence number 5001; each side tracks the OTHER side's sequence space independently via the ACK field of every following segment.

Trade-offs & pitfalls
A frequent mistake is describing the handshake as three round trips; it's actually one and a half round trips of latency, because the SYN-ACK piggybacks the server's SYN onto its ACK of the client's SYN. This is also exactly why TCP always incurs at least one round trip of setup latency before any data can flow, which is the whole motivation behind newer mechanisms like TCP Fast Open that try to send data alongside the very first SYN.

Direct answer

Before I trust my own application of a technique nobody in-house has used, I deliberately design a check that would catch me being wrong, usually by running it against a case where I already know the right answer, and I only report a result to people relying on it alongside an honest statement of what that validation did and did not cover. Trust here comes from actively trying to break my own understanding and failing, not from the technique simply producing an answer that looks reasonable.

Structured elaboration

• Before applying the new technique to the real problem, find or construct a case with a known answer, synthetic data with a known effect, a smaller version of the problem you can verify by hand, or a case where an established method already gives a trusted answer, and confirm the new technique recovers it.
• Sanity-check the assumptions the method actually requires, not just whether it runs; many techniques silently produce an output even when their assumptions are violated.
• Design the validation to specifically target where you are least confident, not the part you already understand well; a check that only confirms what you already believed is not doing much work.
• Communicate confidence and limitations in terms the audience can actually evaluate, what was tested, what was not, and what would change your confidence, rather than a bare number.

Worked example

Needed to estimate the causal effect of a new onboarding flow on retention using a technique nobody on the team had used before, synthetic control (a method that builds an artificial comparison group from a weighted blend of untreated units to estimate what would have happened without the change), for a business case going to leadership. Before touching the real question, I built a known-answer test: I took a metric with an already-established, trusted causal estimate from a past well-instrumented randomized test, reconstructed it using the new technique on the same historical data, and confirmed the synthetic-control estimate landed close to the randomized-test answer. That gave a concrete reason to trust the method beyond it having run and produced a chart. Before applying it to the real question, I explicitly checked the assumption the method depends on, that the synthetic control's pre-period trend actually tracked the treated group closely, rather than assuming it did because the model converged. When presenting to leadership, I stated plainly what had been validated, the method recovering a known answer on a comparable case, and the pre-trend assumption holding reasonably well here, and what had not, a small sample size for the treated group that widens the honest uncertainty, rather than presenting one confident number.

Trade-offs and pitfalls

• Treating "the code ran and produced an output" as proof of correctness is the most common way a newly learned technique gets misapplied; a known-answer check is what actually earns trust.
• Skipping the assumption check because the output looks reasonable is dangerous specifically because a technique can produce a plausible-looking wrong answer when its assumptions are violated.
• Presenting a confident single number to an audience that cannot independently evaluate the method, without naming what was and was not validated, sets up a false sense of certainty that is hard to walk back later.

Scope-Governance Model (Penetration Test Program)

1. Objectives & Baseline Scope

• Define per-business-unit (BU) scope: assets, IP ranges, test types (external/internal/web/mobile), success criteria, blackout windows.
• Produce a signed Scope Statement and Rules of Engagement (RoE) before work starts.

2. Approval Gates

• Gate A — Kickoff Approval: BU Sec Owner + Program PM + Testing Lead sign off baseline RoE.
• Gate B — Pre-Execution: Risk/Change board verifies no conflicting projects, approves schedule.
• Gate C — Post-Change: If change impacts risk or timeline, Program Steering approves revised scope.

3. Change-Request Process

• Submit CR form: requester, rationale, affected assets, urgency, proposed activities.
• Triage within 48 hours by Testing Lead: classify as Minor (<=4 hours, patch window friendly) or Major (>4 hrs / new assets / higher privilege).
• For Minor: Testing Lead + BU Sec Owner approve.
• For Major: Program PM + BU Sec Owner + Risk Board review, then Gate C approval.

4. Time & Cost Impact Assessment

• Estimation template: effort (hours), resource type (senior/junior), tooling/licensing, scheduling shifts, detection/retest iterations.
• Calculate delta: additional hours × hourly rates + tooling + potential retest (fixed matrix).
• Include risk buffer (15%) for unknowns.

5. Communication

• CR acknowledgement within 24h, decision within SLA (48h minor / 5 business days major).
• Updated scope, cost, and timeline communicated via change memo to stakeholders and reflected in project tracker and invoice forecasts.
• Weekly program dashboard: scope status, open CRs, approved changes, impact metrics (hours, cost, delay days).

Example: adding an internal AD engagement mid-program is Major — triage, estimate 40 hrs (2 testers), cost delta, retest cycle; present to Risk Board for Gate C approval before execution.

Situation summary (role):
As a penetration tester I discover a high-severity vulnerability in a subsidiary. The client asks me to withhold reporting because of stock-price concerns.

Ethical and legal considerations

• Professional ethics: Suppressing a valid finding violates core duties of accuracy, integrity, and non-malfeasance (professional codes such as (ISC)2/OSCP ethics).
• Legal/regulatory risk: Deliberately hiding security defects can expose the company and individuals to securities law violations, regulator penalties, breach-notification obligations (e.g., SEC, GDPR, sector-specific rules), and civil/criminal liability if exploit leads to harm.
• Fiduciary duty & duty of care: Security teams/contractors have an obligation to protect customers, users, and investors by ensuring known risks are remediated or disclosed appropriately.

My obligations and actions

1. Review contract and scope to confirm reporting obligations and nondisclosure clauses.
2. Document the finding rigorously (technical details, evidence, exploitability, impact, timeline).
3. Escalate immediately to the engagement owner/CISO and legal/compliance teams; recommend an expedited remediation plan and mitigations (temporary controls, monitoring, patch timeline).
4. Refuse any instruction to falsify or omit the report; request written authorization if client insists — I will not accept oral pressure to conceal.
5. If internal escalation fails and the vulnerability creates imminent risk or legal reporting obligations exist, advise legal counsel on required external disclosures; follow mandatory reporting laws.
6. Record all communications to protect myself; if pressured unlawfully, consider terminating the engagement and seeking guidance from professional body or counsel.

Why this approach

• Protects users and stakeholders, preserves professional ethics and legal compliance, and minimizes long-term reputational and legal risk for both me and the client.

Direct answer

Two findings are "the same issue" when one fix would resolve both. So identity is a rule over asset, weakness type and normalised location, applied in tiers: exact matches on that key are merged automatically; partial matches (same library, different call sites) are grouped under one parent with the occurrences as children; fuzzy matches on titles or text only suggest a merge for a human. Accept extra duplicates before wrongly merged findings, because a duplicate wastes time while a false merge can bury a real vulnerability.

How to decide

Core for interviews: steps 1, 2, 4 and the error trade-off. Steps 3 and 5 are extras that are rarely asked in detail.

1. Canonicalise (also called normalise: rewrite into one standard form) before comparing. Lowercase, strip query strings and trailing slashes, and replace identifiers in paths with a placeholder, so /orders/123?sort=asc and /orders/9/ both become /orders/{id}.
2. Exact key: asset, CWE (the weakness category), and the canonical location. Same key from different scanners means one finding with several sightings (the sources are kept as evidence).
3. Shape hashing for web findings (less common, mention briefly): hash (turn into a fixed fingerprint) things that describe how the request and response are structured (HTTP method, sorted parameter names, response status and content type), not their values, so two scanners' different payloads still collide when they hit the same parameter.
4. Partial matches: for a vulnerable library, the fix is upgrading the package, not editing each call site. Group by asset, vulnerability ID and package as a parent ticket, keep each call site as a child occurrence, and close the parent when none remain.
5. Fuzzy titles (for instance, token similarity: scoring how many words two titles share) are the weakest signal: use them only to populate a human review queue, never to auto-merge.

Worked example (executed)

Six raw findings arrive. The three SQL injection reports use different path spellings; two library findings sit in different files.

import re
from collections import defaultdict

def norm_path(p):
    p = p.lower().split("?")[0].rstrip("/")
    p = re.sub(r"/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}(?=/|$)", "/{id}", p)
    return re.sub(r"/\d+(?=/|$)", "/{id}", p)

def keys(f):
    exact = (f["asset"], f["cwe"], norm_path(f["path"]) if f.get("path") else None)
    parent_id = f["cve"] if f.get("cve") else f["cwe"]
    parent = (f["asset"], parent_id, f.get("package"))
    return exact, parent

def shape_key(f):
    return (f["method"], tuple(sorted(f["params"])), f["status"], f["ctype"])

findings = [
    {"src": "dast",  "asset": "orders-api", "cwe": "CWE-89", "path": "/orders/123?sort=asc"},
    {"src": "sast",  "asset": "orders-api", "cwe": "CWE-89", "path": "/orders/{id}"},
    {"src": "pentest", "asset": "orders-api", "cwe": "CWE-89", "path": "/orders/9/"},
    {"src": "sca",   "asset": "orders-api", "cwe": "CWE-79", "cve": "CVE-2099-0001", "package": "libfoo", "path": "checkout.js:40"},
    {"src": "sca",   "asset": "orders-api", "cwe": "CWE-79", "cve": "CVE-2099-0001", "package": "libfoo", "path": "cart.js:12"},
    {"src": "dast",  "asset": "orders-api", "cwe": "CWE-352", "path": "/orders/5"},
]

exact = defaultdict(list)
for f in findings:
    exact[keys(f)[0]].append(f["src"])
print("exact groups:", len(exact), "from", len(findings), "raw findings")
for k, srcs in exact.items():
    print(" ", k, "<-", srcs)

parents = defaultdict(set)
for f in findings:
    parents[keys(f)[1]].add(keys(f)[0])
for k, children in parents.items():
    print(k, "->", len(children), "occurrence(s)")

web = [
    {"src": "scanner A", "method": "GET", "params": ["sort", "q"], "status": 500, "ctype": "text/html", "payload": "' OR 1=1--"},
    {"src": "scanner B", "method": "GET", "params": ["q", "sort"], "status": 500, "ctype": "text/html", "payload": "\" OR \"a\"=\"a"},
]
print("same shape:", shape_key(web[0]) == shape_key(web[1]))

Output:

exact groups: 4 from 6 raw findings
  ('orders-api', 'CWE-89', '/orders/{id}') <- ['dast', 'sast', 'pentest']
  ('orders-api', 'CWE-79', 'checkout.js:40') <- ['sca']
  ('orders-api', 'CWE-79', 'cart.js:12') <- ['sca']
  ('orders-api', 'CWE-352', '/orders/{id}') <- ['dast']
('orders-api', 'CWE-89', None) -> 1 occurrence(s)
('orders-api', 'CVE-2099-0001', 'libfoo') -> 2 occurrence(s)
('orders-api', 'CWE-352', None) -> 1 occurrence(s)
same shape: True

Six raw findings become four exact groups, and the libfoo finding is one parent with two occurrences. The identifier CVE-2099-0001 is a made-up placeholder for the demo.

How to read the code

• norm_path turns each path into its standard form: it lowercases, drops the query string and trailing slash, and swaps whole numeric or UUID segments for {id} (UUIDs first, so a UUID that starts with a digit is not chopped up). /orders/123?sort=asc, /orders/{id} and /orders/9/ all become /orders/{id}.
• keys builds two dictionary keys per finding. The exact key is (asset, CWE, normalised path). The parent key is (asset, vulnerability ID, package), where the vulnerability ID is the CVE if the finding has one, otherwise the CWE. That is the line parent_id = f["cve"] if f.get("cve") else f["cwe"].
• The first dictionary maps each exact key to the list of scanners that reported it. The second maps each parent key to the set of distinct exact keys (occurrences) under it.
• The three SQL injection reports (dast, sast, pentest) share one exact key, so they collapse to one group with three sightings. The CSRF finding (CWE-352, cross-site request forgery) stays separate because its CWE differs even though the path normalises the same.
• The two libfoo findings are two different exact groups (different files), but they share one parent key, so they become one parent ticket ("upgrade libfoo") with two child occurrences. libfoo is a made-up front-end JavaScript library with a cross-site scripting flaw, which is why its CWE is 79.
• shape_key is the shape-hashing idea in code: two scanners that fire different payloads at the same method and parameters, getting the same status and content type, produce the same shape, so they can be treated as one candidate.

Which errors to accept

At scale

Comparing every pair of findings grows with the square of the count (1,000 findings need about 500,000 pair checks, 2 million need about 2 trillion). Use hashing of the exact key (one pass, constant time per record), and block, meaning only compare fuzzy candidates that already share the same asset and CWE group, so most pairs are never compared. Recompute fingerprints incrementally on each ingest instead of rescanning the backlog.

Pitfalls

• Deleting merged records instead of linking them removes the audit trail.
• Over-normalising a path can merge distinct endpoints (/users/{id} and /users/me).
• Substitution order matters in norm_path: replacing digits before UUIDs turns the UUID 3f2a9c1e-1111-4222-8333-444455556666 into {id}f2a9c1e-... and /api/2fa/setup into /api/{id}fa/setup, so two different endpoints could be mangled or merged wrongly. The UUID pattern therefore runs first, and the digit pattern is anchored to a whole path segment.
• Merging on CVE (a published vulnerability identifier) alone across assets hides which systems are affected.

Direct answer, one paragraph for the CEO

"Risk appetite is how much uncertainty and possible loss the company is willing to take on to reach its goals, set by leadership in advance, the way you set a budget. We want to sell online and move fast, so we accept some risk, but we have said we will not accept any real chance of losing customer payment data, and we want unplanned losses from the web portal to stay under $100,000 a year. When security finds a weakness, we compare it with that line and decide how fast to act."

Worked example (illustrative)

A flaw in the customer portal could let attackers view order histories. The estimate: 40% yearly chance and $400,000 cost, so an expected loss of $160,000 a year (0.4 times 400,000 = 160,000). That is above the $100,000 appetite and also above the $120,000 tolerance defined under Pitfalls, so it jumps the queue and gets fixed now. A similar flaw in an internal reporting page that comes to $20,000 a year sits inside appetite and can wait for the next release.

What the appetite statement does for prioritization

• It gives engineering a rule instead of an argument for each ticket.
• It makes exceptions visible: going above the line needs an executive's name.
• It is reviewed yearly, because the business changes.

Pitfalls

Appetite is not a promise of zero risk. Risk tolerance is the allowed wiggle room around the appetite for a given area, stated in the same unit. For the portal: appetite is $100,000 a year, tolerance is up to $120,000 a year. Do not use the terms interchangeably in front of an audit committee (the board subcommittee that oversees controls and audits).

Where tolerance changes the answer

A flaw that comes to 30% x $370,000 = $111,000 a year is above the $100,000 appetite but inside the $120,000 tolerance. It does not jump the queue: an executive acknowledges it, it gets a dated fix plan and a place in the next release. The $160,000 flaw is above even the tolerance, so it is fixed now and reported upward.