Observability and Monitoring Architecture Questions

Building visibility into infrastructure and services: metrics, logs, and traces, dashboards and alerting, SLIs/SLOs, and the design of an observability stack. Covers instrumenting systems for actionable signal, reducing alert noise, and diagnosing production issues from telemetry. Infrastructure-wide observability, distinct from network-specific monitoring.

MediumTechnical
29 practiced

Compare Elasticsearch, ClickHouse, and an object-storage-plus-index approach as the backend for log storage at scale. Discuss search performance for ad-hoc queries, ingestion throughput, cost at long retention, schema flexibility, and operational overhead for each.

HardTechnical
36 practiced

Define SLOs for the observability pipeline itself: ingestion availability, storage durability, query freshness, and end-to-end latency for a dashboard query. For each one, propose a concrete target, how you'd measure it, and what action fires when it's breached.

HardSystem Design
26 practiced

Design a self-healing telemetry ingestion pipeline: it should detect a failed collector or processor, reroute telemetry to a healthy instance, replay buffered data after a failure, and auto-scale under load, all while exposing its own health so platform engineers can tell when the observability system itself is degraded. What state would you need to track to do this safely, and what stops the remediation logic itself from causing a cascading failure?

HardSystem Design
51 practiced

Design a DaemonSet-based collection agent for a shared Kubernetes cluster: it needs to gather logs, metrics, and traces from every node, handle backpressure gracefully, support dynamic configuration (for example via CRDs), and remain safe to upgrade without dropping telemetry. What would you build in for multi-tenant isolation and failure handling?

HardTechnical
51 practiced

Time-series databases lean on a handful of compression techniques: block-chunking, delta-of-delta timestamp encoding, XOR-based float compression (as in Facebook's Gorilla), and dictionary encoding for labels. Explain how each works and how it affects write throughput and query performance, and contrast a dense, monotonically-increasing counter against a sparse gauge: which techniques help most for each, and why?

Unlock Full Question Bank

Get access to all 47 Observability and Monitoring Architecture interview questions and detailed answers.

Sign in to Continue

Join thousands of developers preparing for their dream job.