Continuous Integration and Delivery Pipelines Questions
Design and implement continuous integration and continuous delivery pipelines that reliably build, test, validate, and deploy applications and infrastructure. Topics include pipeline as code practices, defining stages and triggers for builds and tests, automated testing strategies across unit, integration, smoke, and end to end tests, gating and environment promotion, branching and release strategies, artifact management and versioning, and deployment patterns such as rolling updates, blue green deployments, and canary releases. Candidates should be able to design rollback and recovery procedures, integrate infrastructure provisioning into pipelines, select and configure pipeline tooling such as Jenkins, GitHub Actions, GitLab CI, Azure Pipelines, or cloud vendor pipeline services, and reason about observability and reporting for pipeline health and test execution. Practical considerations include environment parity, pipeline security, secrets handling, pipeline as code best practices, and trade offs between speed and safety.
MediumTechnical
32 practiced
Describe how you would secure CI/CD runners and agents in a cloud environment. Include isolation, resource permissioning, ephemeral credentials (OIDC), artifact signing, log redaction, and guarding against supply-chain attacks.
Sample Answer
**Approach summary** I’d treat CI/CD runners/agents as short-lived, untrusted compute that must be isolated, least-privileged, and observable. Apply ephemeral identity, strong artifact provenance, and pipeline hardening to reduce supply‑chain risk.**Isolation & runtime controls**- Run runners in a dedicated account/tenant or project and isolated VPC/subnet with strict egress controls and NAT/proxy for allowed destinations.- Use ephemeral containers or autoscaled short-lived VMs (scale-to-zero) so no long-lived host state.- Enforce runtime hardening: immutable images, read-only rootfs, seccomp/AppArmor, and limited host capabilities.**Least-privilege resource permissioning**- Assign no static credentials. Use per-job roles via cloud native workload identity (AWS STS AssumeRole with OIDC, GCP Workload Identity, Azure AD workload identity).- Narrow IAM policies: role-per-repository or role-per-pipeline, resource scoping (S3 buckets, ECR repos), and time-bound sessions.- Use resource quotas, network policies, and separate build artifacts/storage with ACLs.**Ephemeral credentials (OIDC)**- Configure CI provider OIDC trust to issue tokens to runners for a single job. Validate audience and repository/job claims.- Require short token lifetime and token exchange to temporary cloud credentials; log and monitor token usage.**Artifact signing & provenance**- Sign build artifacts with tools like cosign/Sigstore; store signatures alongside artifacts in registry.- Produce SBOMs and attestations (in-toto, SLSA) and publish to an attestation store; enforce verification in deployment pipelines.- Verify signatures and SBOMs before promoting to production or deploying.**Supply-chain defenses**- Enforce authenticated registries, image scanning (vuln/CVE), and deny-listing of known-bad base images.- Require reproducible builds where possible and build in hermetic environments.- Implement policy-as-code (e.g., OPA/Gatekeeper, e.g., “reject unsigned artifacts,” “enforce SBOM”) and CI linting for third-party dependencies.- Use provenance checks and continuous monitoring for new vulnerabilities.**Logging, monitoring, and redaction**- Centralize logs to SIEM/Cloud Logging through forwarders; redact PII/secrets at source using log filters and structured logging libraries.- Prevent secrets from being emitted: scan logs for secrets, enforce secret masking in runners, and use secret-injection from managed secret stores (no direct env vars).- Alert on anomalous token use, unusual egress, or artifact mismatch.**Operational practices**- Rotate and audit OIDC trust, IAM roles, and secrets; run periodic pentests and dependency audits.- Capture metrics: build duration, network egress, failed signature verifications, and policy violations.- Educate teams on supply-chain hygiene and require attestation checks in CD.This combination of isolation, ephemeral OIDC-based creds, least-privilege IAM, artifact signing/provenance, logging with redaction, and policy enforcement materially reduces attacker surface and supply-chain risk for cloud CI/CD runners.
EasyTechnical
40 practiced
Write a GitHub Actions workflow YAML that builds a Node.js application (npm) for Node 14 and 16, runs unit tests, caches node_modules, and uploads test reports as artifacts. Trigger on push and pull_request targeting main. Keep steps idempotent and concise.
Sample Answer
**Brief approach**- Use a matrix for Node 14 and 16- Cache dependencies keyed by package-lock.json and Node version- Use npm ci for deterministic installs- Run tests and upload test-results directory as artifact- Trigger on push and pull_request to main
yaml
name: CI
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
build-and-test:
runs-on: ubuntu-latest
strategy:
matrix:
node: [14, 16]
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}
cache: 'npm' # built-in caching for node modules
- name: Install dependencies
run: npm ci
- name: Run tests (generate reports)
run: |
mkdir -p test-results
npm test -- --outputFile=test-results/results.xml || true
- name: Upload test reports
uses: actions/upload-artifact@v4
with:
name: test-results-node-${{ matrix.node }}
path: test-results
Notes:- actions/setup-node cache: 'npm' plus npm ci provides idempotent installs; alternative: actions/cache with key: npm-${{ runner.os }}-${{ matrix.node }}-${{ hashFiles('**/package-lock.json') }} if finer control is required.- Ensure your test runner is configured to write reports into test-results (e.g., jest-junit, mocha reporters).
EasyTechnical
36 practiced
Explain the difference between tagging an artifact (for example docker:latest or v1.2.3) and using an immutable digest (sha256). Why is using image digests recommended for deployments, and how would you incorporate digest-based deployment in a CI/CD pipeline?
Sample Answer
**Difference — tag vs digest**- Tag (docker:latest or v1.2.3): mutable human-readable label that can be moved to point to different image manifests over time. Convenient for workflows and versioning, but not guaranteed stable.- Digest (sha256:...): immutable content-addressed identifier produced by the registry; always references the exact image bytes and metadata.**Why use digests for deployments**- Reproducibility: guarantees the exact image will run (no surprise changes).- Safety: prevents accidental upgrades from moving tags (e.g., latest).- Security & auditing: ties deployed artifacts to scanned digests; enables verified provenance.- Reliable rollbacks: you re-deploy the exact previous digest.- Better caching behavior in registries and runtimes.**How to incorporate digest-based deployment in CI/CD (practical steps)**1. Build and push - CI builds image and pushes to registry (CI user).2. Capture digest - Read push output or call registry API (or `docker manifest inspect` / `crane digest`) to get sha256.3. Persist artifact - Store digest in CI artifacts, release metadata, or an artifact registry entry (e.g., Git tag, release notes, S3, DB).4. Render deploy manifests - Use image reference format image:tag@sha256:sha... or image@sha256:... - For Kubernetes/Helm, template the image field with the digest (avoid relying on mutable tags).5. CD uses digest - Deployment pipeline reads the stored digest and applies manifests (kubectl/terraform/helm) ensuring immutable image.6. Integrations and best practices - Run vulnerability scanning on image before publishing digest. - Sign images (cosign) and verify signature in CD. - Keep tag->digest mapping for human traceability but let CD use digest.Using this flow as a Cloud Engineer ensures reliable, auditable, and secure deployments across AWS/GCP/Azure workloads.
EasyTechnical
31 practiced
What key metrics and logs would you collect to monitor CI/CD pipeline health and test execution? Describe dashboards, alerting thresholds, and automated reporting you would implement to ensure fast feedback and reliability for engineering teams.
Sample Answer
**Situation and goal**As a Cloud Engineer I focus on CI/CD observability that gives fast feedback and reliable deployment pipelines across AWS/GCP/Azure.**Key metrics to collect**- Pipeline-level: build success rate, failure rate, median and p95 build time, queue wait time, deployment frequency, change lead time.- Job-level: test pass/fail counts, flaky test rate, test execution time distribution, code coverage trend.- Resource-level: agent CPU/memory, container restart rate, disk I/O, network egress during builds.- Cost/throughput: cost per build, concurrency used.**Logs**- Build logs, test logs (with structured test IDs), deployment logs, orchestration/agent logs, authentication/permission errors, infra provisioning events.**Dashboards**- Overview: success rate, average build time, open/failed runs, deployment frequency (last 24h, 7d).- Drilldowns: slowest jobs, flaky-tests leaderboard, agent health and autoscaling metrics.- Business view: mean time to recover, releases per week.**Alerting thresholds**- High-severity: pipeline failure rate > 10% over 30m; master branch build failures > 5 consecutive.- Medium: p95 build time > baseline * 2 for 15m; flaky test rate > 3% for 24h.- Low: agent CPU > 80% sustained 10m, queue wait time > 5m.Use multi-channel alerts (pager, Slack, email) with runbook links and suppressions for deployments.**Automated reporting**- Daily summary email/Slack with failures, flaky tests, top offenders, cost delta.- Weekly actionable report: trends, suggested agent scaling, test-suite optimization.- Auto-ticketing for repeated failing jobs and test retest triggers for suspected flakes.Rationale: combine pipeline, test, and infra signals to isolate causes quickly and drive reliability and fast feedback for engineers.
MediumTechnical
37 practiced
You observe that a subset of tests are flaky and cause pipeline instability. Propose a strategy to triage and mitigate flaky tests while maintaining fast feedback. Cover tactics like quarantining tests, reruns, prioritization, test parallelization, and investment decisions.
Sample Answer
**Situation & Goal**I’d stabilize CI by triaging flaky tests to restore fast, reliable feedback while minimizing developer interruption and cloud cost.**Immediate triage**- Run analytics: tag flaky tests via CI metadata (pipeline id, job, runtime, failure messages) and surface top offenders with failure rate and MTTR.- Quarantine high-flake tests: automatically mark tests with failure rate > threshold (e.g., 5% over 7 days) and remove them from the main blocking pipeline.**Mitigation tactics**- Reruns with limits: for transient failures, implement configurable auto-rerun (1 retry) in CI (e.g., CodeBuild/CodePipeline or Jenkins) and surface retried runs separately.- Prioritization: keep fast, high-value smoke tests in the blocking path; move slow or low-value tests to gated nightly or pre-merge jobs.- Test parallelization: shard suites across ephemeral cloud runners (AWS Fargate/EKS spot nodes) to keep wall-clock time low while quarantined tests run elsewhere.- Instrumentation: add logging, distributed traces, and CloudWatch metrics for flaky tests to identify root causes (network timeouts, rate limits, resource exhaustion).**Investment decisions**- Quick wins: flakiness due to infra (timeouts, ephemeral resources) — increase timeouts, stabilize test fixtures, or use stable mock services.- Medium: invest in dedicated, isolated test environments (VPC endpoints, warmed databases) to remove cross-test interference.- Long-term: rewrite brittle tests (replace end-to-end with contract/unit tests), add more deterministic infrastructure (localstack alternatives, test harnesses).- Cost vs confidence: quantify cost of flaky runs (compute + dev time) vs remediation; prioritize fixes with highest ROI.**Measurement & Process**- Track SLOs: pipeline success rate, mean time to green, and feedback latency.- Policy: quarantined tests require an owner, triage ticket, and SLA for either fix or permanent relocation.- Continuous review: weekly dashboard and monthly investment backlog reviews.This approach balances fast feedback, operational cost, and targeted engineering investment aligned with cloud infrastructure stability.
Unlock Full Question Bank
Get access to hundreds of Continuous Integration and Delivery Pipelines interview questions and detailed answers.