InterviewStack.io LogoInterviewStack.io

Algorithm Design & Real-Time System Optimization Questions

Algorithm design techniques and real-time optimization strategies applicable to distributed systems and latency-sensitive architectures. Covers scheduling, resource management, concurrency, distributed algorithms, load balancing, and performance optimization under strict latency requirements.

MediumSystem Design
53 practiced
Design a deadline-aware scheduler for an inference cluster where each request contains a soft or hard deadline and a priority. Describe algorithms and data structures you would use (e.g., earliest-deadline-first, multiple priority queues), how you would support preemption or cancellation, and approaches to avoid starvation and ensure fairness.
EasySystem Design
67 practiced
Design a single-node, low-latency serving process to execute an ensemble of 5 small models per request. Target: P95 end-to-end latency <150ms at 2000 RPS. Describe process architecture (threads/processes), batching strategy, CPU/GPU allocation decisions, I/O optimizations, memory layout, and how you'd instrument per-model latencies.
MediumTechnical
95 practiced
Explain hedged requests (sending duplicates to multiple backends) and design how to implement hedging at an inference gateway to reduce tail latency. Include criteria for issuing hedges, deduplication and cancellation techniques, and the cost/throughput trade-offs involved.
HardTechnical
91 practiced
For a recommender system with embedding tables larger than any single node's memory, design algorithms for sharding embedding tables across machines to minimize tail read latency under heavy read load. Discuss placement heuristics, hot-shard caching, replication vs partitioning trade-offs, routing decisions, and strategies to handle skewed access patterns.
MediumTechnical
48 practiced
Implement (in Java or Python) a concurrent prioritized queue with aging for inference requests. The queue should normally return the highest priority item but gradually increase the effective priority of older items to prevent starvation. Provide interface definitions, key methods, and explain your approach to ensure thread-safety and reasonable performance.

Unlock Full Question Bank

Get access to hundreds of Algorithm Design & Real-Time System Optimization interview questions and detailed answers.

Sign in to Continue

Join thousands of developers preparing for their dream job.