Requirements Gathering and Scoping Questions

First I’d elicit precise constraints so the team and stakeholders share a clear target:

• Ask concrete questions: What does “real-time” mean here (sub-second, seconds, minutes)? Which metrics require that freshness vs. which can tolerate delay? What user journeys or dashboards depend on this? What SLOs/SLAs, query volumes, retention, and peak concurrency should we design for? What cost ceiling or budget constraints exist? Any compliance/consistency requirements?
• Capture answers in a short spec: metric → freshness requirement → consumers → query patterns → acceptable error bounds.

Trade-offs between real-time and batch:

• Real-time (streaming): low latency, high hardware/engineering cost, operational complexity, harder at scale; enables immediate alerts and user-facing features.
• Batch: simpler, cheaper, easier to test and backfill; higher latency but sufficient for trend analysis & many KPIs.
• Consider hybrid: near-real-time for critical metrics, batch for all others; approximate methods (sampling, sketches) to reduce throughput.

Phased recommendation:

1. Phase 0 — Discovery (1–2 sprints): finalize spec, measure event volumes, run cost estimates, prototype query load using sampled production data.
2. Phase 1 — MVP near-real-time (2–4 sprints): implement an event pipeline that provides 1–5 minute freshness using existing infra (e.g., Kafka → micro-batch jobs or lightweight stream processors). Expose dashboards and alerts for priority metrics.
3. Phase 2 — Incremental improvements (ongoing): optimize hot paths to sub-second using streaming processors (Flink/ksqlDB) only for metrics with strong business value; add SLOs, monitoring, and cost controls.
4. Phase 3 — Harden & scale: autoscaling, cost optimization, retention, and documentation.

Deliverables and success metrics:

• Clear spec document, prototype cost estimate, MVP dashboards, latency and cost dashboards, and a rollout plan.
  This balances product value (fast wins for key metrics) with bounded engineering effort and measurable ROI before investing in full streaming.

Approach: Treat the 4-week, three-person constraint as a strict timebox. Decide "must-have" vs "nice-to-have" by asking: does this feature deliver the core user value (create → share → view) safely and reliably within time? If not, defer. Prioritize by risk (technical unknowns first), user impact, and engineering effort.

Must-have criteria:

• Delivers the core user story end-to-end (user records/selects a clip, attaches minimal metadata, shares a link/message, recipient can play the clip).
• Secure and stable: basic input validation, permission checks, and handling for network failures.
• Analytics for success measurement (share attempted, play started).
• CI, basic tests, and an internal rollout path.

Minimal scope (4 weeks, 3 people):

• Clip creation: record or select up to 30s, basic trimming UI
• Upload: background upload to existing media CDN with resumable support
• Share: generate shareable URL + native OS share sheet integration
• Playback: lightweight player for shared link (web/mobile deep link)
• Permissions: public-by-link sharing model (no complex ACLs)
• Monitoring: simple logging + events for share/play
• Tests & CI: unit tests for upload/playback logic, smoke test

Nice-to-have (deferred):

• Social feed, comments, advanced editing, reactions, expiration controls, granular ACLs, rich previews.

Execution plan:

• Week 0: design + spike (upload API + deep link)
• Weeks 1–3: parallel implementation (one on client UI/recording, one on upload/playback + tests, one on sharing/deep links + QA)
• Week 4: integration, bugfixes, instrumentation, limited rollout.

This keeps focus on core value while minimizing scope and technical risk.

Requirements & constraints

• Functional: authorize, capture, refund, void, store payment methods (PCI scope), retry logic, idempotency, audit trail, reconciliation, KPIs (latency, throughput).
• Non-functional: PCI-DSS compliance, throughput (~orders/sec), SLOs (p99 latency), high availability, observability, deployability.
• Integration: support current payment providers, backwards-compatible API for order service, retry/backoff, secure secrets, rate limits.

Technical constraints & design decisions

• Keep payments data separated; service owns payment state; orders reference payment_id.
• Use synchronous call for authorization (low latency path) and async events for reconciliation and refunds.
• Persistence: strong consistency for payment state (ACID DB) and append-only ledger for audit (immutable events).
• Network: circuit breaker, bulkhead, rate-limiting.

Migration risks

• Downtime or partial failures causing lost payments or double-charges.
• Inconsistent semantics between monolith and new service (idempotency mismatch).
• Increased latency on happy path.
• PCI scope mistakes exposing sensitive data.
• Operational complexity: monitoring, runbooks.

Data consistency considerations

• Use event-driven pattern: monolith writes an "initiate_payment" event while new service becomes the source of truth for payments.
• Two-phase pattern for safe cutover: orchestrated syncs + reconciliation job to compare ledgers.
• Idempotency tokens and monotonic transaction IDs to prevent double processing.
• Compensating transactions for failures (refunds/voids).
• Reconciliation: daily/real-time diffs between order and payment stores, alerting, manual correction flows.

Incremental rollout strategy

1. Build service behind feature flag; run in shadow mode: mirror requests to new service and compare responses without affecting live flows.
2. Select low-risk traffic: route subset of customers/orders to new service (canary) with full monitoring.
3. Switch read-after-write: new payments created in service; orders still read legacy data for verification until parity confirmed.
4. Dual-write for transitional period: monolith writes to both payment DB and service via API; reconciliation validates parity.
5. Full cutover: route all writes to service; keep legacy payment code read-only for historical queries.
6. Cleanup & hard delete after reconciliation window; maintain runbook for rollback, manual refunds, and incident responses.

Metrics & monitoring

• Track success/failure rates, latency, duplicate charges, reconciliation mismatches, PCI alerts.
• Automated tests: contract tests, chaos tests, end-to-end payment flows.

This plan minimizes customer impact, preserves consistency via idempotency and reconciliation, and limits PCI scope while allowing safe, observable migration.

Acceptance criteria (must be testable & unambiguous)

• Functional: When a valid "right to be forgotten" request is approved, all personal identifiers (PII) tied to user-id X are deleted or rendered irreversibly anonymized across systems within 24 hours, except where legal hold applies. Confirm via API: GET /users/X returns 404 or anonymized record with no PII.
• Scoped: Deletion respects scope (full account vs. limited data categories) per request.
• Referential integrity: Business-critical non-PII records remain (e.g., order totals) but any direct identifiers are removed or replaced with pseudonyms.
• Auditability: For every request, an immutable audit entry exists with request-id, requester, timestamp, scope, outcome, and proof-of-deletion (hash or snapshot).
• Authorization: Only authorized roles/systems may approve requests; unauthorized attempts are rejected.
• Compliance exceptions: Legal holds block deletion and are logged with justification.

Automated test strategy (mapping)

• Unit tests
  • Validate input parsing, scope validation, authorization checks, legal-hold flag logic.
  • Example: function isDeletionAllowed(userId, scope) -> boolean.
• Integration tests
  • Test deletion flows across services using test doubles or ephemeral environments: user service, profile store, analytics pipeline; verify data removal or pseudonymization.
  • Verify messaging (events published to downstream systems) and compensating rollbacks.
• End-to-end (E2E) tests
  • Simulate full request: submit request via UI/API, approve, wait for processing, then assert via public APIs, database snapshots, search indices, and third-party integrations that PII is gone.
  • Include scenarios with legal hold, partial scope, and failure/retry.
• Regression tests
  • Periodic suite that replays a sample of prior deletion requests and validates audit evidence and downstream scrub.

Production controls & audit logs (must be immutable and queryable)

• Audit log content: request-id, user-id, requester-id, approver-id, timestamps (requested/approved/completed), scope, legal-hold flag, outcome, deletion-proof (content hash or list of affected tables/keys), retention policy.
• Storage: Write-once append-only store (WORM), e.g., secure S3 with Object Lock or dedicated immutable DB. Logs must be cryptographically signed.
• Monitoring & alerts: SLA breach alerts if deletion not completed within allowed window; anomaly detection for suspicious repeated requests.
• Proof-of-deletion: Store a signed manifest listing locations and record hashes before and after anonymization; for irreversible deletions, store attestation that data was purged and index entries removed.
• Access control & retention: Strict RBAC for viewing logs; retain audit logs per legal/regulatory timeline; provide export capability to compliance teams.

Edge cases and verification

• Third-party processors: Ensure contracts + automated API/webhook confirm deletion at downstream vendors; tests mock vendor confirmations.
• Backups: Define backup retention and deletion lifecycle — automated tests verify eventual purging from backups per policy.
• Idempotency & retries: Tests ensure repeated requests do not resurrect data or create inconsistent state.

This strategy provides clear acceptance criteria, layered automated tests (unit → integration → E2E), and production-grade immutable audit evidence to demonstrate GDPR compliance.

Functional requirements define what the system must do — specific features and behaviors visible to users or other systems. Non-functional requirements (NFRs) define how well the system performs those functions — quality attributes like performance, reliability, security, and maintainability.

Three functional requirements for a ride‑sharing app:

1. Rider-driver matching: When a rider requests a ride, the system must find and notify the nearest available driver within N km and present estimated arrival time. (Acceptance: driver assigned within 5s).
2. Payment processing: The app must allow riders to add payment methods, calculate fares, and charge the selected method at trip completion. (Acceptance: successful charge or clear error message).
3. Trip tracking and receipts: The app must record trip route, distance, fare breakdown, and provide a downloadable receipt to rider and driver after each trip.

Three non-functional requirements:

1. Latency / Performance: The end-to-end time from ride request to receiving driver match must be <2 seconds under 95th percentile load. Ensures good user experience.
2. Availability / Reliability: Matching and payment services must have 99.95% uptime monthly and graceful degradation (e.g., read-only ride history) during partial outages.
3. Security / Privacy: All payment data must be PCI-compliant; user PII must be encrypted at rest and in transit, with role-based access controls and audit logging.

These examples show functional behaviors and measurable NFRs that guide design, implementation, testing, and SLOs.