Curiosity and Investigative Problem Solving Questions

Assess intellectual curiosity and investigative habits: asking why, digging into root causes, exploring alternatives, and demonstrating enthusiasm for learning how systems work. Candidates should show evidence of proactively researching problems, asking good follow up questions, and a mindset of continuous improvement. This topic captures behavioral indicators that predict thoroughness and long term growth potential.

EasyBehavioral

42 practiced

Tell me about a time when asking a seemingly simple or naive question uncovered a larger issue or improved system reliability. What was the question, how did you investigate further, and what lasting change resulted?

MediumTechnical

49 practiced

Design a runbook template for recurring intermittent 503 errors affecting a microservice. Specify the sections, which diagnostic commands and telemetry to include (kubectl, curl, SQL), how to correlate traces across services, and how to capture an incident timeline that helps future investigations.

Sample Answer

Runbook: Recurring Intermittent 503s — Microservice X

Purpose & Scope- When Microservice X returns intermittent 503s for >1% of requests or SLA breached.- Impact, owner, on-call rota, escalation contacts.

Pre-checks (quick triage)- Is alert still firing? Link to alert & threshold.- Check recent deploys / config changes (last 24h).- Confirm reproducibility (time window, paths, clients).

Immediate Info to Capture (Incident stub)- Start time, reporter, symptoms, affected clients/regions, request examples (IDs/paths), % errors, SLO impact.

Key Telemetry to Review- HTTP metrics: success/error rates, 5xx by code, latency percentiles.- Pod/container metrics: CPU, memory, restarts.- Env metrics: node pressure, kube events.- Upstream/downstream errors: DB, cache, auth.- Traces: request span, root cause service, errors per trace id.

Diagnostic Commands (ordered, with intent)- Kubernetes: - kubectl get pods -n ns -l app=x -o wide - kubectl describe pod <pod> -n ns # recent events, OOM, liveness probe - kubectl logs <pod> -c app -n ns --since=15m | grep -i "503" -A5 -B5 - kubectl exec -it <pod> -n ns -- sh -c 'netstat -tnp | grep :<port>' - kubectl get hpa,deploy -n ns- Networking / HTTP: - curl -v -H "X-Request-ID: test-<ts>" https://svc/path # reproduce, capture headers - curl --resolve svc:443:<podIP> ... # test direct pod- Database: - psql -h db -U user -c "SELECT count(*) FROM pg_stat_activity WHERE state='idle in transaction';" - Check slow queries: SELECT * FROM pg_stat_statements ORDER BY total_time DESC LIMIT 10;- Infrastructure: - kubectl top pod -n ns - kubectl get nodes -o wide; kubectl describe node <node>- Observability queries: - PromQL: sum(rate(http_requests_total{service="x",status="503"}[5m])) by (pod) - Logs: search for "upstream timed out" / "connection refused" / "503"- Tracing: - Find example trace by request-id: Jaeger/Tempo query for traceId - Follow spans to see which component returned 503 and whether it's an internal error or gateway timeout.

How to Correlate Traces Across Services- Ensure propagation of tracing headers (traceparent / x-request-id / b3).- Given a failing request-id or trace id from logs/curl, open in tracing UI and: - Identify first error span and service that raised 503. - Map latency spikes to specific spans (DB, external API, queue). - Correlate span attributes (pod, node, instance-id, upstream-host).- Use logs->traces links: from a log line with trace id jump to trace, and from trace to logs for involved spans.

Investigation Steps1. Reproduce with curl; capture request-id and response headers.2. Locate traces and logs for that request-id across services.3. Check pod health: restarts, OOM, liveness failures.4. Inspect upstream dependencies for timeouts or resource exhaustion.5. Confirm load patterns: burst traffic, autoscaler limits, queue backlog.6. If config/deploy suspected, rollback last change to test.

Mitigation Playbook (ordered, low-risk first)- Scale replicas / increase HPA minimum.- Temporarily increase timeouts between services / API gateway.- Re-route traffic (kubectl scale/adjust k8s service) or disable problematic instance via weighted routing.- Roll back recent deploy if correlation strong.- Restart affected pods/nodes as last resort with controlled rollout.

Post-Incident Capture (timeline for future investigations)- Timeline table: timestamp, actor, action, observation, evidence (links: logs, traces, dashboards, commits).- Root cause hypothesis + evidence mapping.- Actions taken and outcome (metrics pre/post).- Follow-ups: code fix, alert tuning, runbook updates, SLO impact, RCA owner and deadline.

Automation & Preventive- Ensure tracing headers enforced and logged.- Add alert: spike in 503s + trace id sampling.- Add synthetic checks that assert successful end-to-end requests and validate upstream dependencies.

Example quick reproducible command blockkubectl logs $(kubectl get pods -l app=x -o jsonpath='{.items[0].metadata.name}') -n ns --since=10m | grep "request-id=123"curl -v -H "x-request-id: test-123" https://gateway/svc/path

Notes & Best Practices- Always capture request/trace ids in the incident stub.- Prefer evidence-first actions (observe before change).- Preserve logs/traces snapshot for postmortem.

EasyTechnical

40 practiced

Explain the value of a blameless postmortem and list the essential elements that should appear in a post-incident report for an SRE team. Describe how you ensure action items are specific, owned, and tracked to completion.

EasyTechnical

52 practiced

You see a sudden CPU spike on a critical service. List the top telemetry sources and quick checks you would perform in the first 10 minutes to gather evidence for a root cause. Include infra, app, and external dependency checks and why each matters.

HardTechnical

40 practiced

Create a rubric to evaluate a senior candidate's investigative skills during a hiring interview. List observable behaviors, artifacts, and practical exercises you would use, and explain how you would score and weight curiosity, hypothesis generation, data analysis, communication, and system knowledge.

Sample Answer

Rubric summary (total 100 pts). Weighting: Curiosity 20, Hypothesis generation 20, Data analysis 25, Communication 20, System knowledge 15.

Observable behaviors (what interviewer watches):- Curiosity: Asks clarifying Qs, probes beyond prompt, explores edge cases, requests missing context or metrics.- Hypothesis generation: Proposes multiple, prioritized hypotheses, justifies likelihood, identifies dependencies and failure modes.- Data analysis: Selects appropriate metrics/logs, designs queries/visual checks, interprets noisy signals, distinguishes correlation vs causation.- Communication: Explains reasoning aloud, summarizes next steps, adapts language for technical/non-technical listeners, documents assumptions.- System knowledge: References relevant architecture patterns, tools, capacity constraints, common failure modes, and mitigation strategies.

Artifacts to request:- Sample runbook/incident postmortem they’ve written- Snippets of monitoring queries/dashboards- Short architecture sketch or sequence diagram

Practical exercises (30–45 min):1) Live incident postmortem: present a degraded-service scenario (logs, alerts, latency charts). Candidate must triage, list hypotheses, run a couple of targeted queries (or pseudo-queries), recommend mitigation and follow-up.2) Debug whiteboard: ask to draw data flow and identify single points of failure and observability gaps.3) Data problem: provide time-series with anomaly; ask to compute SLO impact and propose remediation.

Scoring guidance (per competency, 0–4):0 — Missing/incorrect1 — Limited, shallow2 — Adequate, some gaps3 — Strong, minor gaps4 — Exemplary, complete and pragmatic

How to combine:- Score each competency 0–4, multiply by weight factor (e.g., Curiosity weight 20 → max 20 points = score*5).- Total out of 100; thresholds: 85+ hire, 70–84 strong considered, <70 needs follow-up.

Calibration notes:- Prefer demonstrated applied experience over hypotheticals.- Penalize hidden assumptions not disclosed; reward validation plans.- Use artifacts to verify claims (e.g., review their runbook for quality).

Unlock Full Question Bank

Get access to hundreds of Curiosity and Investigative Problem Solving interview questions and detailed answers.

Join thousands of developers preparing for their dream job.