Infrastructure as Code and Configuration Management Questions

Infrastructure as Code and Configuration Management covers designing, implementing, and operating infrastructure defined as code and the practices that keep that infrastructure consistent and auditable. Topics include Infrastructure as Code principles and patterns, declarative versus imperative approaches, idempotency, state management, module and template design for reusable infrastructure, version control integration, testing and validation of infrastructure code, drift detection and remediation, safe rollout and rollback strategies, policy as code and Git operations for infrastructure, and how to scan and remediate infrastructure misconfigurations. This topic also encompasses integration points with system configuration tools and considerations for managing secrets and secure defaults within infrastructure definitions.

EasyTechnical

32 practiced

Explain the relationship between Infrastructure as Code tools (Terraform, CloudFormation) and configuration management tools (Ansible, Chef, Puppet). For a typical service that needs both provisioning and in-guest configuration, explain a recommended separation of responsibilities and lifecycle integration.

HardSystem Design

26 practiced

Design a CI/CD validation platform for Terraform that can process thousands of changes per day with low latency. The platform must run quick pre-merge checks, policy evaluations, and schedule heavier integration tests, while preventing resource sprawl, controlling cost, and providing actionable developer feedback.

Sample Answer

**Clarify requirements & constraints**- Pre-merge fast checks (lint, fmt, plan-diff) with < few minutes latency.- Policy-as-code (OPA/Conftest) enforced pre-merge; heavier integration tests scheduled post-merge.- Handle thousands of changes/day, limit resource sprawl, control cost, actionable dev feedback.

**High-level architecture**- Git-triggered pipeline orchestrator (lightweight webhook service) → CI runners pool (autoscaling) → Policy evaluation service → Job scheduler for integration tests → Resource manager + cleanup service → Observability & feedback UI.

**Core components**- Webhook/orchestrator: validate event, dedupe branches, gate flood protection.- Fast CI runners: ephemeral, containerized (Kubernetes with nodepools), run terraform fmt/validate/plan with remote backend (Terraform Cloud/Enterprise or S3+Dynamo locking).- Policy service: OPA/Conftest as a service; evaluate plan JSON; return fail/warn with pointers to offending resources.- Integration scheduler: queue heavier tests (infra apply in ephemeral isolated accounts/projects) with cost/time budgets and prioritization.- Resource manager: tag, TTL, and automated destroy; ephemeral accounts via org-level provisioning, enforce quotas.- Cost control: per-PR budget tokens, pre-approve expensive changes, spot instances, size-limited ephemeral environments.- Feedback loop: PR comments with plan diff, failed policy snippets, remediation hints, links to repro logs and cost estimate.

**Data flow**1. Push → orchestrator dedupes → start fast runner.2. Runner creates plan (remote state), posts plan JSON to policy service.3. If pass, annotate PR; if risky or needs integration, schedule test job with permissions.4. Integration runs in ephemeral env; resource manager enforces TTL and destroys resources after test; cost metrics logged.

**Scalability & reliability**- Autoscale runner pools per queue length; use prioritized queues for critical branches.- Cache Terraform modules/artifacts; share remote state to avoid repeated downloads.- Use rate limits and leader-election to avoid burst provisioning.

**Trade-offs**- Ephemeral full applies cost vs confidence: mitigate via sampling, approval gates.- Using Terraform Cloud simplifies locking/plan runs but adds vendor dependency.

**Metrics & monitoring**- Latency (PR -> plan), throughput, cost per test, orphaned resource count, policy failure rates.- Alert when orphaned resources > threshold or cost spike.

**Why this works**- Separates fast, low-cost checks from expensive integration tests; enforces policy early; automates cleanup and budgeting to prevent sprawl; provides clear, actionable feedback tied to plan and policy failures.

EasyTechnical

24 practiced

Describe common safe rollout strategies for infrastructure changes (canary, blue-green, progressive expansion, feature flags for infra). Explain the trade-offs between speed, complexity, and safety and give examples of when each should be used in cloud infra rollouts.

MediumTechnical

31 practiced

Your CI needs to run Terraform for PRs, but you must avoid exposing credentials or allowing untrusted forked PRs to access organization secrets. Propose a secure CI pattern for running Terraform plan and apply stages, including how to safely provide secrets, limit access, and prevent secrets leakage in logs.

Sample Answer

**High-level pattern**- Use short-lived, least-privilege credentials minted at CI runtime (OIDC / STS) and separate roles for plan vs apply.- Disallow secrets in untrusted fork builds; run plan in a safe, read-only context and only allow apply from protected branches with human/maintainer approval.

**Concrete implementation (GitHub Actions example)**- PR workflow (runs for forks): - Use GitHub OIDC to assume an IAM role that has only read/list/plan permissions (no write, no state modifications). - Configure Terraform with remote backend (e.g., S3 + DynamoDB) but use a read-only backend role or a plan-only backend configuration (or read a snapshot) so state cannot be mutated. - Run terraform init/plan, output a saved plan file artifact (encrypted) and post a sanitized plan summary as a comment. - Do NOT inject organization secrets from repository/organization secrets—CI must refuse to pass secrets to workflows that originate from forks (use `if: github.event.pull_request.head.repo.fork == false` for steps that need secrets).

- Apply workflow (protected): - Trigger apply only on merges to protected branches or via a workflow_dispatch that requires reviewers/approvals. - Use OIDC to assume an apply-role with write/state permissions; require MFA or enforced approvals in repo settings. - Use the saved plan artifact (signed/validated) to run terraform apply.

**Secrets & leak prevention**- Never echo secrets; enable CI masking of secrets and disable TF debug logging (TF_LOG unset).- Mark sensitive outputs in Terraform and redact provider responses before posting to PR.- Restrict logs: run terraform with `-no-color` and wrap sensitive commands; scan logs for secrets and fail if detected.- Use ephemeral credentials (short TTL) and rotate roles/policies regularly.

**Least-privilege policies**- Plan-role: read-only on state, describe/list permissions on resources.- Apply-role: limited to specific resource actions and the backend state write.- Audit with CloudTrail; alert on abnormal usage.

This pattern prevents exposing org secrets to forked PRs, minimizes blast radius with least privilege, and ensures apply actions are controlled and auditable.

MediumTechnical

27 practiced

You run Terraform and discover Terraform plan shows 200 resources drifted after a manual maintenance window. How would you assess which drifts to accept (adopt), which to revert, and the safest automated remediation approach to apply at scale without causing downtime? Describe your step-by-step operational plan.

Sample Answer

**Situation & goal**

I find 200 resources drifted after manual maintenance. My goal: determine which changes are safe to adopt vs revert, and implement automated, low-risk remediation at scale without downtime.

**1) Rapid triage (10–30 min)**- Run `terraform refresh && terraform plan -out=tfplan` to capture current vs desired state.- Group drifts by resource type, tag, environment, and change type (config drift vs metadata vs lifecycle change).- Identify critical paths (load balancers, DBs, networking) and all resources with no-recreate diffs.

**2) Risk assessment matrix**- High risk: resources whose replacement causes downtime (RDS, ASG launch config, subnet changes) — treat as “review.”- Medium: non-disruptive config changes (tags, security group rules with additive changes) — candidate to adopt.- Low: immutable metadata or drift from external monitoring — auto-adopt with validation.

**3) Validate intent**- Check change origin: review maintenance logs, change tickets, operator notes, cloud audit logs.- If change intentional and documented → mark Adopt. If accidental or unknown → mark Revert or Hold.

**4) Safe remediation plan**- For “Adopt” non-disruptive: run `terraform apply -refresh-only` or update state with `terraform import`/`state mv` where appropriate; use `-target` for small batches.- For “Revert” or destructive: create scoped plans (`terraform plan -target=... -out=...`), run in maintenance windows, use blue/green or rolling updates (e.g., update ASG launch template then roll one instance at a time).- For ambiguous: run non-destructive tests in staging by replaying changes.

**5) Automation at scale**- Build pipeline: generate per-resource-type plans, require approvals for high-risk plans, run canary on small subset (e.g., one AZ or instance), monitor metrics and rollback on errors.- Use Terraform Cloud/Enterprise or CI to store plan artifacts, and require manual approval for destructive changes.

**6) Safeguards & monitoring**- Back up state and snapshots (DB/volume) before apply.- Use health checks, slow rollout, and automated rollback scripts.- Post-apply: run acceptance tests and continuous monitoring for 30–60 minutes.

**7) Runbook & communication**- Document steps, owners, rollback steps for each group.- Communicate schedule and expected impact to stakeholders.

This approach minimizes downtime by adopting safe, documented changes, isolating destructive work, using staged automated rollouts, and enforcing approvals and monitoring.

Unlock Full Question Bank

Get access to hundreds of Infrastructure as Code and Configuration Management interview questions and detailed answers.

Join thousands of developers preparing for their dream job.