Agile and Scrum Fundamentals Questions

Kanban vs Scrum for frequent interrupts

Pros/cons:

• Scrum: good for predictability and planned sprints but struggles with frequent unplanned work.
• Kanban: better for flow and handling variable work, continuous delivery; harder to guarantee longer commitments.
• Scrumban: hybrid — sprint cadence with WIP limits for flexibility.

Recommendation: Scrumban

• Rules: set WIP limits per swimlane (e.g., In Progress max 5, Review max 3).
• Swimlanes: Planned Work, Interrupts/On-call, Expedite (P0).
• Urgent handling: define an interrupt policy—if an interrupt estimated > 8 hours, create a ticket, assign on-call and decrease sprint capacity accordingly; if <2 hours allow ad-hoc handling without re-planning.

Operationalize:

• Reserve 20% capacity per sprint for interrupts or enforce strict WIP for interrupts lane.
• Use SLA targets for incident response and count interrupts in velocity reporting to improve future capacity planning.

Triage & communication for production pipeline failure

1. Immediate triage (first 15–30 minutes):

• Stopgap: identify scope (which datasets/dashboards affected).
• Run quick health checks (job logs, cluster metrics, schema drift).
• Assign incident owner (on-call + data eng lead).

2. Immediate communications:

• Notify stakeholders: incident channel + severity, ETA for next update, dashboards impacted, public-facing note if needed.
• Post an initial incident brief within 30 minutes.

3. Criteria to interrupt sprint work:

• Interrupt if: business-critical SLA breached, revenue/operational risk, or >X% of dashboards incorrect.
• Otherwise route to on-call rotation and continue sprint tasks with reduced capacity planning.

4. Ensuring fix without derailing sprint:

• Create an incident ticket with priority and link to sprint board; estimate remaining work and re-schedule noncritical stories.
• Implement fix with safety: blue/green or feature-flagged rollforward; prepare rollback plan (snapshot restore or replay strategy).
• Postmortem: owner drafts within 48 hours; conduct blameless postmortem within one week, identify root cause, action items, and add remediation stories to backlog.

This balances urgent remediation, transparent stakeholder comms, and preserving sprint predictability.

Managing cross-team dependencies during sprint planning

Requirements to clarify:

• Identify feature interfaces, SLAs, and delivery windows.

Tools & artifacts:

• Dependency board (Kanban swimlane) linked to JIRA tickets with explicit dependency tags.
• Pre-planning syncs (15–30m) one week before planning with reps from Data Eng, Analytics, Product Eng.
• Explicit dependency tickets with acceptance criteria, owner, and due date.
• Shared contract docs (schema registry, API contracts) in Confluence + automated contract tests.

Process:

• During backlog refinement mark dependencies and estimate lead time; surface high-risk dependencies in planning.
• Use RACI on tickets: who delivers what and who verifies.

Escalation path:

• Triage in pre-planning -> if unresolved, escalate to Engineering Managers within 48 hours -> if still blocked before sprint start, decline or re-scope story and capture as blocked with mitigation plan.

This keeps dependencies visible, testable, and actionable before sprint commitments.

Requirements & constraints

• Five data-engineering teams, many consumer squads, central platform, shared dependencies, need cross-team cadence.

Framework comparison (brief)

• Nexus: lightweight Scrum-of-Scrums for <10 teams; good for code-centric coordination.
• LeSS: lean, single product backlog; fits if product boundaries are unified.
• SAFe: prescriptive, good for large orgs needing release trains and governance.

Recommendation: LeSS + Release Train elements

• Rationale: fewer process layers; encourages single backlog for platform features and dataset products, but add a quarterly Release Train for coordinated releases.

Concrete architecture

• Backlogs: Single product backlog for data platform features; separate area backlogs per team for implementation. Use tags for dataset-products and consumers.
• Cross-team roles: Product Owner (platform PO), Area PO (team-level), Scrum Masters, Release Train Engineer (RTE) for cadence, Data Architect for standards, Governance Board for contracts.
• Release coordination: 8–12 week release trains with synchronization sprint boundaries; weekly sync (dependency board) and bi-weekly PI planning for next train.
• Dependency resolution: visible dependency board (kanban), dependency owners commit to pre-negotiated APIs/contracts; dedicated dependency-resolution slot in sprint planning.
• Governance: lightweight committee (Data Steward, Security, Architect, PO) for approvals; automated gates in CI for contracts and compliance.

Trade-offs: Keeps agility and avoids SAFe overhead while ensuring cross-team release safety via release-train mechanics.

Purpose of spikes: reduce uncertainty and produce decision-making artifacts.

Scoping a spike

• Objective: single question (e.g., can we parse format X at scale?).
• Timebox: 1–3 days (max one sprint for deep unknowns).
• Deliverables: prototype code, benchmark numbers, list of risks, recommended next steps.

Success criteria

• Clear pass/fail thresholds (e.g., parse throughput >= Y records/sec, memory < Z GB, or feasibility decision).

Workflow

1. Create spike ticket with scope and acceptance criteria.
2. Timebox and log outcomes; stop when time expires.
3. Convert outcome: if successful, create estimated backlog stories (implementation, tests, infra). If not, create alternative approach stories or abandon.

Example: unknown compressed log format → 3-day spike: implement parser, benchmark on 1M rows, produce regression tests, record failure modes. If parser fails, spike recommends vendor or different approach.

This ensures spikes produce actionable outcomes without open-ended research.