Infrastructure Domain Expertise Areas Questions

Clearly articulate the specific infrastructure domains where you have depth: Linux systems, Windows servers, virtualization, networking, storage, monitoring, backup/disaster recovery, cloud platforms, or security. Quantify your experience (e.g., 'managed 150 Linux servers', 'designed backup strategy for 50+ applications'). Be honest about gaps.

MediumTechnical

31 practiced

A public-facing web application is intermittently slow; latency spikes correlate with increased database wait times and occasional packet retransmits. Describe a step-by-step troubleshooting plan: what high-level dashboards, host-level metrics, database diagnostics, and network captures you would collect; which tools you'd use (tcpdump, iostat, perf, slow query logs); and the order of checks to isolate network, compute, or storage issues.

Sample Answer

**High-level plan (steps in order)**

1. Verify symptoms on dashboards — confirm time windows and scope (which nodes, regions, endpoints).2. Quick host-level triage to distinguish compute/network/storage.3. Targeted captures & DB diagnostics during next spike.4. Root-cause analysis and fix/mitigation.

**Dashboards & metrics to collect**- APM / frontend: request latency, error rate, user geography, traces (Datadog/New Relic/Jaeger).- Infra: host CPU, load, memory, disk I/O latency, network tx/rx, interface errors.- DB: active connections, lock/wait times, query counts, buffer pool hit ratio.

**Host-level checks & tools**- vmstat / sar / top / htop — CPU, runqueue, context switches.- iostat -x 1 10 and iotop — disk throughput and latency; look for high await and %util.- free / slabtop — memory pressure and swapping.- ss / netstat / ifconfig / ethtool — socket states, interface errors, NIC counters.- perf top/record if suspect CPU kernel/user hotspots.

**DB diagnostics**- Enable/collect slow query log; use EXPLAIN on slow statements.- Check performance_schema / pg_stat_activity (Postgres) for waits, locks.- Monitor buffer/cache misses, checkpoint activity, replication lag.

**Network captures**- tcpdump on app and DB hosts during spike (rotate, e.g., -C -W) capturing SYN/ACK, retransmits, RTT:

bash

tcpdump -i eth0 tcp and host <db> and port <3306> -w spike.pcap -C 100

- Analyze in Wireshark or tshark: retransmits, dup acks, RTO, MTU issues. Run mtr/traceroute to check path.

**Order of checks to isolate domain**1. Dashboards → confirm pattern.2. Host CPU/memory/io (vmstat, iostat) — if high I/O await → storage.3. Network counters and tcpdump — if retransmits/RTOs → network.4. DB internals (slow logs, locks) — if heavy waits/long queries → database.5. Profiling (perf) if CPU-bound.

**Notes & mitigations**- Correlate timestamps across logs (NTP).- Capture minimal pcap + metrics to limit overhead.- Temporary mitigations: scale read replicas, increase DB connection pool, enable connection pooling, route around problematic host, throttle heavy queries.

EasyTechnical

29 practiced

Describe the difference between block storage and object storage. For each, explain performance characteristics, consistency models, common use cases (e.g., VM disks, databases, backups, media archival), and examples of technologies/services you have administered (e.g., SAN/iSCSI, AWS EBS, S3, Ceph).

EasyBehavioral

34 practiced

Describe your hands-on experience with cloud platforms (AWS, Azure, GCP). For each platform you've used, quantify resources managed (number of instances, VPCs, storage TB), list key services you administered, and describe one migration or optimization you led. If you lack experience with a platform, state that clearly.

MediumTechnical

26 practiced

You're planning storage expansions for a SAN currently at 200 TB used and projected 30% annual growth. Describe how you would model growth, plan procurement cycles, choose RAID/erasure-coding and tiering strategies, account for rebuild times and performance impact, and execute expansion/migration with minimal downtime.

Sample Answer

**Growth modelling & capacity targets**- Start with current 200 TB used and 30% YoY growth. Project year-by-year: - Year1 = 200 * 1.3 = 260 TB - Year2 = 260 * 1.3 ≈ 338 TB - Year3 ≈ 439 TB- Add safety buffer: plan for 20% headroom above projection for unexpected spikes and degraded-mode capacity. So procurement targets: Year1 target ≈ 312 TB usable.

**Procurement cadence**- Purchase on a 12–18 month cadence tied to lead times and budget cycles.- Maintain 3–6 months of spare procurement runway (contracted quotes, reserved SKUs).- Stagger expansions (purchase smaller, incremental shelves) to avoid forklift upgrades and spread risk.

**RAID / erasure coding & tiering**- For high IOPS/low latency (VMs, DB): use RAID10 or mirrored pools.- For capacity-efficient bulk (backups, archive): use RAID6 or array erasure coding (if vendor supports EC) for better space efficiency with acceptable rebuild overhead.- Implement tiering: hot tier (NVMe/SSD) for active workloads, warm tier (SAS SSDs) for less active, cold tier (spinning disks or object store) for infrequently accessed data. Use automated policy-based tiering.

**Rebuild times & performance impact**- Estimate rebuild time: depends on drive size and array bandwidth. Example: 10 TB drive, rebuild throughput 300 MB/s: - Rebuild time ≈ (10 TB) / (300 MB/s) ≈ 9.6 hours (single drive). With large arrays and multiple failures, rebuilds can be days.- Account for rebuild impact: degraded state increases rebuild I/O and latency. Mitigations: - Limit rebuild priority during business hours. - Use spare drives pre-staged and proactively replace predictive-fail drives. - Prefer erasure coding for very large pools to reduce rebuild scope but test rebuild performance.

**Expansion & migration with minimal downtime**- Use non-disruptive online expansion features: add shelves to existing pool or create new pool and migrate.- For VMs: use storage vMotion or live migration to move volumes to new pools without downtime.- For block LUNs: use array-level replication or mirror relationships (split/replicate) to migrate, then cutover during maintenance window.- Steps: 1. Plan capacity mapping and map workloads to tiers. 2. Pre-stage hardware and racks, network zoning, and multipath config. 3. Expand or create new pool; apply replication/async mirror. 4. Migrate data during low-usage windows; validate integrity and performance. 5. Decommission old volumes after verification.- Test the entire procedure in a lab, monitor rebuild and latency metrics, and have rollback plans.

**Monitoring & metrics**- Track usable capacity, growth rate, average IOPS, latency p95/p99, rebuild queue lengths, and SMART alerts.- Automate alerts for threshold crossings (e.g., 70% usage) to trigger procurement.

This approach balances capacity efficiency, performance, risk mitigation for rebuilds, and minimal downtime through staged, tested migrations.

HardTechnical

26 practiced

You must migrate 500 VMs from VMware vSphere to KVM/QEMU with minimal downtime and zero data loss. Outline an end-to-end migration plan: discovery and mapping, conversion tooling (virt-v2v, OVF/OVA, storage export), network mapping, validation/testing strategy, scheduling/batching approach, and a rollback plan for failed conversions.

Sample Answer

**Situation & Goals**Migrate 500 VMs from vSphere to KVM/QEMU with minimal downtime and zero data loss. My plan minimizes risk via discovery, staged conversion, testing, and clear rollback.

**Discovery & Mapping**- Inventory VMs (vCenter API/PowerCLI): OS, CPU/RAM, disk types (thin/thick), snapshots, IPs, NIC types, VMware Tools, applications, dependencies, RPO/RTO.- Classify by criticality & OS (Windows/Linux), storage backend (VMFS, NFS), network VLANs, and unique hardware IDs.- Produce migration matrix mapping source VM -> target host/pool, storage class, VLAN, and required downtime window.

**Conversion Tooling**- Prefer virt-v2v for Linux/Windows VMs; use OVF/OVA export for complex appliances.- For disk export: use vddk-based tools or qemu-img convert via SSH streaming to avoid full copy: qemu-img convert -p -O qcow2 "vpx://..." "ssh://kvmhost/var/lib/libvirt/images/vm.qcow2"- Ensure VMware Tools removal and guest-agent installation post-conversion (qemu-guest-agent, cloud-init).- Validate UUIDs and drivers (virtio-net/blk) and inject virtio drivers for Windows.

**Network Mapping**- Map VLANs and IP allocations; pre-create libvirt networks or Neutron equivalents.- Preserve MAC where possible; otherwise update DNS and DHCP reservations.- Test connectivity in isolated test network before production cutover.

**Validation & Testing**- Functional: boot, service start, NIC, disk integrity, app sanity checks, checksum of critical data.- Performance: I/O and CPU baseline comparisons.- Data integrity: run rsync/robocopy with checksum between last sync and cutover.- Use snapshot/checkpoint and backup before each batch.

**Scheduling & Batching**- Batch by dependency and criticality (start with low-risk Linux dev VMs, then stateless apps, then DBs).- Use incremental syncs: initial full copy, frequent delta syncs, final quiesce and last sync during maintenance window (minutes downtime).- Parallelize per host capacity; target 10–20 concurrent conversions depending on network/storage.

**Rollback Plan**- Keep source VMs intact and powered off only after validation.- If conversion fails, power on source VM; use DNS switch or VIP rollback to revert clients.- Maintain final incremental sync data for reattempts; if storage corrupted, restore from backups (tested beforehand).- Document automated rollback playbooks (PowerCLI and libvirt commands) and execute pre-approved escalation.

**Operational Controls**- Monitoring, logging, runbooks, and stakeholder communication. Dry-run 5–10 pilot VMs capturing metrics and refine timing.

Unlock Full Question Bank

Get access to hundreds of Infrastructure Domain Expertise Areas interview questions and detailed answers.

Join thousands of developers preparing for their dream job.