Symmetric Encryption and Block Ciphers Questions

High-level issues

• API misuse / ambiguous input: concatenating ciphertext + tag is fragile — callers can pass wrong lengths; library expects ciphertext||tag but caller responsibility should be explicit.
• Error handling: function lets AESGCM raise InvalidTag implicitly and may propagate exceptions or leak info via different error messages/timings.
• Side-channels & secrecy: plaintext handled/processed by this function; if processing occurs before a clear authenticated check (or on exceptions) you risk leaking data. No zeroing of sensitive material, no input length checks, no IV uniqueness checks.
• Protocol-level: no checks on key length, IV uniqueness/reuse, AAD validation.

What could go wrong

• Silent misuse (wrong tag length) -> decrypt raises or returns garbage if misused elsewhere.
• Exception messages or timing differences could be used by an attacker to learn about validity.
• IV reuse breaks confidentiality/integrity.
• Logging or process_plaintext might leak secrets.

Corrected implementation and safer API contract

• Validate inputs, explicitly expect combined ciphertext_with_tag or separate ct and tag with lengths documented.
• Catch InvalidTag and return an auth-failure without revealing details; don't differentiate error types in responses.
• Keep decryption pure: return plaintext to caller; let caller decide how/where to process it (separation of concerns).
• Zero sensitive buffers where possible.

Example corrected code:

from cryptography.hazmat.primitives.ciphers.aead import AESGCM
from cryptography.exceptions import InvalidTag

def decrypt_aead(key: bytes, iv: bytes, aad: bytes, ciphertext: bytes, tag: bytes) -> bytes:
    # Validate lengths (AES-GCM tag is 16 bytes)
    if len(tag) != 16:
        raise ValueError("invalid tag length")
    if len(iv) != 12:
        raise ValueError("expect 12-byte IV for AES-GCM")

    aesgcm = AESGCM(key)
    combined = ciphertext + tag  # explicit and documented

    try:
        plaintext = aesgcm.decrypt(iv, combined, aad)
    except InvalidTag:
        # Authentication failed: do not leak details, normalise timing where needed
        raise PermissionError("decryption failed") from None

    # Do not process or log plaintext here; return to caller to handle securely.
    return plaintext

Safer API contract (recommendations)

• Accept either (ciphertext_with_tag) OR (ciphertext, tag) but document expected tag length and IV length.
• Do not perform application-specific processing inside the crypto primitive.
• On auth failure always return a uniform error/response; avoid detailed errors or differing status codes that leak info.
• Enforce IV uniqueness at protocol level; rotate keys appropriately.
• Zero sensitive buffers after use and avoid logging secrets.

Overview
Five common AEAD implementation pitfalls, their practical security consequences, and mitigations — from a cryptographer’s perspective.

1. Nonce reuse (e.g., AES-GCM)

• Consequence: Tag forgery, plaintext recovery, and complete loss of confidentiality/integrity due to keystream/reused GHASH collisions.
• Mitigation: Use deterministic nonce construction (counter or HKDF-based per-key nonce), enforce unique nonces per key, use algorithms with built-in nonce misuse resistance (e.g., AES-SIV) for risky environments.

2. Verifying tag after parsing / acting on plaintext

• Consequence: Early application of unauthenticated plaintext enables injection attacks, command injection or state corruption before authenticity check.
• Mitigation: Always authenticate before use — verify the tag fully, then parse/act. Keep decryption and parsing separate; parse only after successful verification.

3. Truncated tags

• Consequence: Lowered brute-force cost for forgery; with short tags (e.g., 32 bits) attacker success probability rises, enabling practical forgeries.
• Mitigation: Require sufficiently long tags (>= 96 bits recommended; 128 bits ideal). If truncation is necessary, compensate with rate-limiting and rekeying.

4. Improper AAD handling

• Consequence: Mismatched AAD between endpoints causes accepted forged messages or undetected manipulation of associated context (e.g., headers), breaking binding.
• Mitigation: Clearly define and canonicalize AAD format; include protocol identifiers, sequence numbers, and versioning in AAD; test cross-language serialization.

5. Non-constant-time tag comparisons / side-channels

• Consequence: Timing or memory side-channels allow attackers to recover tag bytes incrementally, enabling forgery with fewer guesses.
• Mitigation: Use constant-time compare routines provided by vetted libraries; avoid byte-at-a-time verification loops; ensure compiler/runtime won’t optimize away constant-time behavior.

End with secure defaults: prefer well-reviewed libraries, explicit key lifecycle (rekeying/limits), and continuous fuzzing and unit tests for AEAD misuse.

Situation & consequence (high level)
Nonce/IV reuse in AEADs like AES‑GCM breaks the nonce-uniqueness assumption and quickly destroys security: it can leak plaintext and enable forgery.

Concrete attacks

• Loss of confidentiality: AES‑GCM uses a GHASH-based synthetic IV and a counter-mode keystream for encryption. Reusing (K, IV) causes keystream reuse; XORing two ciphertexts C1 ⊕ C2 cancels keystream and yields P1 ⊕ P2, enabling plaintext recovery with known structure or chosen plaintext.
• Forgery/Tag recovery: Reused IVs let an attacker solve for GHASH-state differences and recover authentication subkey information (H = E_K(0)), enabling tag forgery with far fewer attempts than brute force. Practically, repeated IVs have led to universal forgery and key recovery attacks against GCM variants.

Relation to KDF & nonce construction

• Key-derivation should separate encryption keys and any per-session IV seeds. If a KDF derives a single key but the protocol reuses IVs across rekeys, security breaks. Always derive fresh keys or fresh IV-salts per handshake.
• Nonce construction: build nonces from a per-session random salt (from KDF) XOR a monotonically increasing counter or sequence number, or include explicit direction/role bits to avoid collision across endpoints. Example: TLS 1.3 derives a static per-connection IV and XORs it with record sequence numbers.
• Misuse-resistant choices: use AEADs that tolerate nonce misuse (e.g., AES‑GCM‑SIV) or ensure protocol-level guarantees: unique nonces per key, monotonic counters, and robust KDF domain separation for keys/IVs.

Recommendations

• KDF: derive separate keys for encryption, for H (if needed), and an IV salt; include context strings and role bits.
• Nonces: prefer sequence-number-derived nonces XORed with per-session IV; avoid random nonce generation without collision checks.
• Consider misuse-resistant AEADs where implementation risk of reuse is non-negligible.

High-level goal
Provide continuous confidentiality/integrity for multi-hour streaming while limiting key exposure and preserving forward secrecy where feasible.

KDF & primitives

• Use HKDF-SHA256 (HKDF-Expand for per-message keys) with a long-lived root secret (RK) and per-epoch salt.
• AEAD: AES-256-GCM or ChaCha20-Poly1305 for payloads.
• For forward secrecy, periodically derive RK' via an ephemeral Diffie-Hellman (X25519) exchange and mix into HKDF-Extract (RK := HKDF-Extract(dh_shared, RK || context)).

Rekeying triggers (combined policy)

• Time: max epoch = 1 hour (configurable).
• Data volume: max_bytes_per_key = e.g., 1 GB.
• Message count: max_msgs_per_key = e.g., 1e6.
• Entropy event: explicit client/server request or network event (suspicion of compromise).
• Use the earliest trigger to start rekey.

Rolling-key overlap & transition

• Overlap window W (e.g., 30s): new epoch key K_v+1 generated and advertised; both K_v and K_v+1 accepted for decryption during W. Senders switch to K_v+1 immediately after advertising plus sequence threshold to avoid interleaving. After W + max_rtt + slack, retire K_v and securely zeroize.

Message tagging & versioning

• Each record header includes: { key_version (32-bit), epoch_start_ts, sequence_number, nonce } and MAC from AEAD.
• Receivers validate key_version → pick corresponding key, reject unknown/future versions. Sequence numbers + per-epoch window prevent replay.

Forward secrecy & compromise containment

• Regular ephemeral DH for RK updates (e.g., every N epochs or on trigger). Use PFS to ensure past ciphertext cannot be decrypted if long-term RK is compromised.
• Keep ephemeral private keys ephemeral and zeroized. Store only minimal state: current RK, next RK, and mapping key_version→derived_key with expiry.

Operational notes & trade-offs

• Shorter epochs = better compromise containment but more signaling. Overlap window must cover client clock skew and retransmissions.
• HKDF allows stateless derivation; include protocol context (stream ID, endpoint IDs) in info.
• Audit logs for rekey events; hardware-backed key material recommended.

Clarify goal & constraints

• Move from AES-CBC || HMAC-SHA256 (MAC-then-Encrypt) to AEAD (AES-GCM or ChaCha20-Poly1305) with zero/low downtime, maintain interop with old clients, and prevent downgrade attacks.

High-level phased plan

1. Preparation (no-visible-user impact)

   • Define protocol versioning and ciphersuite IDs: e.g., v1 = CBC+HMAC, v2 = AEAD-AESGCM, v3 = AEAD-ChaCha20-Poly1305.
   • Add explicit negotiation fields to handshake (client_supported = [v1,v2,...], server_selected, server_sig over negotiation).
   • Update server and library stacks to understand both modes.

2. Key-material migration

   • Use HKDF to derive AEAD keys from existing master secret to avoid rekeying downtime:
     • Derive ae_key = HKDF-Expand(master_secret, "AEAD key", length)
     • Derive legacy keys unchanged for v1 paths.
   • Store key version metadata and a monotonic key-version counter in KMS; allow dual-usage of master_secret-derived keys.
   • Plan full rotation: generate new master_secret and provision to clients gradually.

3. Transitional compatibility modes (runtime)

   • Dual-mode servers: accept both v1 and v2 frames. For minimal change, support "wrapped" transitional record:
     • Prefer AEAD by default; if client advertises only v1, fall back to CBC+HMAC.
     • Server attaches a negotiation proof: server_signed( client_hello | server_hello | selected_version ). This prevents undetected server-initiated downgrade.
   • Optionally send both records during transition for critical messages: primary AEAD record + legacy-encrypted blob for old clients (size trade-off).

4. Preventing downgrade attacks

   • Require server signature (private key known to server) over full handshake including selected ciphersuite and an anti-downgrade sentinel value when downgrading from client-supported AEAD to legacy.
   • Clients verify server_sig and the presence/absence of known sentinel strings; reject silent downgrades where server selects lower version but signature indicates higher was supported.
   • Use monotonic counters or timestamps in negotiation to detect replayed/downgraded handshakes.

5. Rollout and deprecation

   • Stage A: Release server supporting both modes; release client update that prefers AEAD, validates server_sig, and logs fallbacks.
   • Stage B: Telemetry phase — measure percent of sessions using AEAD, error rates, lambdas for legacy-only clients.
   • Stage C: Enforce AEAD-only for new devices by version flag; allow legacy-only clients through a compatibility gateway with strict logging and time-limited exceptions.
   • Stage D: After threshold (e.g., 99.9% migrated), revoke legacy-only capability — rotate keys to remove legacy-derived keys from KMS.
   • Provide kill-switch and rollback scripts in case of large-scale failures.

Operational considerations

• Test vectors and interoperability matrix for AES-GCM and ChaCha20-Poly1305.
• Performance: benchmark AES-GCM vs ChaCha20-Poly1305 across platforms; prefer ChaCha20 on low-end devices.
• Backwards secrecy & forward secrecy maintained by continuing DH/ECDHE; ensure AEAD keys derived per-session.
• Audit logs, metrics (cipher distribution, handshake failures), and phased client update cadence.

Security trade-offs

• Hybrid “double-send” increases bandwidth but eases compatibility.
• Deriving AEAD keys from existing master_secret reduces immediate operational overhead but plan for full rekey to independent randomness for long-term security.

This plan minimizes downtime, preserves backward compatibility, prevents downgrades via signed negotiation, and provides clear checkpoints to retire legacy crypto safely.