Requirements Traceability and User Acceptance Testing Questions

Approach:

• Record assumptions explicitly in a centralized Assumptions Log with ID, owner, rationale, date, confidence, and impact if false.
• Maintain a Traceability Matrix (TM) that links Assumption IDs to Requirements, Design Decisions, and Acceptance Criteria so auditors can follow rationale and update artifacts when assumptions change.
• Use tooling (Confluence/Jira/Excel/ReqIF) and enforce reviews: each requirement/design must reference related assumptions; change-control updates all linked items.

Example Traceability Matrix format (CSV/CSV-like):

AssumptionID | Assumption | Owner | Confidence | LinkedReqIDs | LinkedDesignIDs | LinkedAcceptanceCriteria | LastReviewed
A-001 | Customer will provide SSO metadata within 2 weeks | AE | Medium | R-010,R-012 | D-03 | AC-05: IdP metadata loaded & validated | 2025-05-10

One example entry explained:

• A-001: The solution’s SAML SSO design (D-03) depends on timely IdP metadata. Requirements R-010 (Authentication) and R-012 (Onboarding timeline) inherit this assumption. Acceptance Criteria AC-05 explicitly tests that metadata is loaded and validated; if A-001 fails, timeline and alternative design D-04 (temporary OAuth) are triggered.

Best practices:

• Assign owners and review cadence, mark assumptions by confidence and risk, and link via unique IDs so automated reports can highlight requirement/design items affected by assumption changes.

1. Functional behavior — Complete and correct migration of production workloads

• Criterion: 100% of targeted applications and databases successfully migrated with data integrity (checksum match) and no functional regressions.
• Measured by: Automated post-migration validation scripts (end-to-end tests + DB checksum) run by QA/Release Engineering. Acceptance: zero critical defects; <=2 medium defects blocked by mitigation plan.

2. Performance — Equivalent or better runtime performance

• Criterion: 95th-percentile API latency ≤ pre-migration baseline +10% and throughput ≥ baseline under 95% of expected peak load.
• Measured by: Load/perf testing team using JMeter/k6 against production-like traffic; compare to baseline reports.

3. Security — No critical vulnerabilities and compliance baseline met

• Criterion: Zero critical/high CVEs in scanned images; IAM roles follow least-privilege; data at-rest and in-transit encrypted. SOC/Pentest pass with no high findings.
• Measured by: Security team using SCA/DAST/SAST scans, container/image scanning, and an external pentest; results documented in security sign-off.

4. Operations readiness — Runbooks, monitoring, and rollback proven

• Criterion: Completed runbooks for deploy/rollback/incident, monitoring/alerts instrumented with alert-to-ack SLA ≤ 15 minutes, and one successful failover test.
• Measured by: SRE/Ops through runbook review, monitoring dashboards (Prometheus/CloudWatch), and live failover drill witnessed by Ops manager.

5. User acceptance — Primary user task success rate

• Criterion: In UAT, ≥ 90% of sampled end-users complete the top 10 critical tasks without assistance and mean SUS (usability) score ≥ 75.
• Measured by: Product Owner/UAT coordinator collecting task completion metrics, session recordings, and SUS survey results; sign-off from business stakeholders.

Acceptance criteria

• Functional: 100% of customer records present in target; no data corruption; referential integrity preserved.
• RPO/RTO: RPO ≤ 5 minutes of transactions; RTO ≤ 30 minutes for full cutover.
• Performance: Target DB supports production QPS with 95th percentile latency ≤ current baseline.
• Security/compliance: Data encrypted in-transit and at-rest; access controls mirror source; audit trails retained ≥ 7 years.
• Operational: Rollback plan validated; cutover window ≤ 2 hours; zero lost transactions.
• Stakeholder sign-off: Ops, Legal, Product approve pre-cutover checklist.

Cutover plan (Pre-copy → Delta sync → Cutover)

1. Preparation (weeks prior)

• Schema & mapping validation, create target schema, test migrations on subset.
• Baseline counts, constraints, index plans, capacity & perf tests.
• Create runbooks, rollback scripts, network/firewall access, and backups (source snapshot + target init).

2. Pre-copy (bulk migration)

• Export + import using parallelized ETL (chunked by customer_id ranges or time windows) with per-chunk checksums; run in read-only mode where possible.
• Produce per-chunk manifests: row_count, checksum (e.g., SHA256 of stable canonical serialization), min/max PK.

3. Delta sync (ongoing changes)

• Enable change data capture (CDC) from source (binlog/CDC stream) to stream into target continuously; apply in-order with idempotency.
• Maintain lag monitoring; replay queues for retries.

4. Cutover (final sync & switch)

• Schedule short write-fence: stop writes to source or route writes to a queue; apply remaining CDC until Zero Lag.
• Run final reconciliation: global row counts, per-table checksums, referential integrity checks.
• Promote target to primary; redirect application traffic using feature flags/load balancer DNS TTL < 60s.
• Post-cutover: Monitor for errors 2x baseline alerting; keep source read-only for N hours before decommission.

Verification checks

• Automated: Per-table row counts, per-chunk SHA256 checksums, FK constraint asserts, index validation, schema diff.
• Transactional integrity: Reconcile last N minutes of transactions via CDC logs vs. application event logs.
• Functional: Run application smoke tests, user journeys, and performance load tests against target.
• Sampling: Random record-level field-by-field compare for 0.1% of records covering edge cases (large blobs, nulls, encodings).
• Security/compliance: Verify encryption, IAM roles, and that PII redaction/retention matches policy.
• Observability: Confirm metrics (QPS, latency, error rates), alerts, and trace continuity.

Capturing migration evidence for legal & ops

• Immutable artifacts:
  • Manifests and manifests hashes per chunk (timestamped) stored in WORM storage (S3 Object Lock) with access logs.
  • CDC audit log export with sequence numbers and checksums.
  • Final reconciliation report: per-table counts, checksum diffs, and signed attestations from Ops and Solutions Architect.
  • Runbook execution logs and timestamps (who performed each step), automation job logs, and screenshots of dashboards.
  • Backups and snapshots identifiers for source and target with retention metadata.
• Formal documentation:
  • Pre-cutover checklist, risk register, rollback decision tree, and cutover approval form signed by stakeholders.
  • Post-migration report: incident log, SLA verification, and stakeholder sign-offs.
• Legal-ready packaging:
  • Bundle artifacts into an immutable archive with cryptographic signature (PGP) and provide a verification procedure.
  • Retain access logs and IAM change records showing who accessed or modified data during migration.

Risk & rollback

• If verification fails: immediately revert DNS/load balancer to source read-only, replay missed CDC, or restore from source snapshot. Define kill-switch with clear owner and <15 min execution plan.

This plan balances minimal downtime, verifiable integrity, and auditable evidence suitable for legal and ops stakeholders.