Identity, Authentication, and Access Management Questions
Designing and operating identity and access control systems. Covers authentication protocols and standards (OAuth, SAML, OIDC, MFA), authorization models (RBAC, ABAC), identity lifecycle and privilege management, IAM architecture and automation, and access control across cloud and on-premises environments. The 'who can do what' control plane, distinct from cryptographic key management.
Design a scalable Single Sign-On (SSO) system for a multi-tenant SaaS product using OpenID Connect. Requirements: support tenant-specific IdPs, SSO and single-logout, MFA, session propagation across subdomains, and handle 50k authorization requests/second. Provide a component diagram, token flows (ID and access tokens), refresh token handling, tenant isolation strategy, HA and scalability considerations, and how you would test SSO failover.
Sample Answer
Direct answer
Route every tenant's login through a central single sign-on (SSO) gateway that resolves which identity provider (IdP) to federate to based on the tenant, using OpenID Connect (OIDC) as the protocol, issuing short-lived ID and access tokens plus a longer-lived refresh token, and propagating one session across subdomains via a shared, domain-scoped session cookie or a centrally issued token validated by every subdomain's own service. At 50,000 authorization requests per second, the gateway and token-issuance path must be horizontally scaled and stateless (session state lives in a shared store, not in any one instance), with tenant-specific IdP configuration and per-tenant session/token isolation enforced structurally, not by convention, so a bug in one tenant's IdP integration cannot leak into another tenant's session validation.
Structured elaboration
flowchart TB
User[User Browser]
App[Tenant App]
Gateway[SSO Gateway]
IdPRouter[Tenant to IdP Router]
IdP1[Tenant A IdP]
IdP2[Tenant B IdP]
TokenSvc[Token Issuance Service]
SessionStore[(Session Store)]
User --> App
App --> Gateway
Gateway --> IdPRouter
IdPRouter --> IdP1
IdPRouter --> IdP2
IdP1 --> TokenSvc
IdP2 --> TokenSvc
TokenSvc --> SessionStore
TokenSvc --> App
Tenant-specific IdPs
- Each tenant registers its own IdP configuration, issuer URL, client credentials or public key, supported flows, in a tenant-to-IdP router keyed by tenant id, or resolved from the user's email domain at the login page before the OIDC redirect happens (enter your email, look up the domain, resolve the tenant, redirect to that tenant's IdP).
- The gateway's OIDC client configuration loads per-request from this registry rather than being hardcoded, so onboarding a new tenant's IdP is a configuration change, not a code deploy.
- Many tenant IdP integrations pair OIDC authentication with SCIM (System for Cross-domain Identity Management) provisioning: the IdP pushes user create, update, and deactivate events to the platform automatically, so a user deactivated in the tenant's own directory loses SSO-based access immediately rather than only at their next token expiry, closing the exact gap a purely authentication-only integration leaves open (a deactivated employee's still-valid refresh token would otherwise keep working until it expires on its own).
SSO and single sign-out (SLO)
- Login: the standard OIDC authorization code flow, with PKCE (Proof Key for Code Exchange) for public and single-page-application clients, against the resolved tenant IdP; the gateway exchanges the authorization code for tokens and establishes the cross-subdomain session below.
- Logout: front-channel logout, a same-browser redirect telling each subdomain's session to end, for apps the user's current browser can still reach, plus back-channel logout, a server-to-server call from the IdP or gateway to each subdomain's logout endpoint, for apps or background sessions the browser round trip won't reliably reach. A front-channel-only implementation silently leaves a session live in a background tab, or an app the browser navigated away from before the redirect chain completed.
Multi-factor authentication (MFA)
- MFA is enforced at the tenant IdP, when it supports MFA itself, or at the gateway as a secondary step-up after the primary OIDC authentication completes. Either way, the resulting ID token's claims should carry an authentication-methods-reference (
amr) claim indicating which methods were actually used, so downstream services can enforce "this specific action requires MFA to have occurred" without reimplementing MFA themselves.
Session propagation across subdomains
- Two workable patterns: a shared, domain-scoped cookie (
Domain=.example.com) carrying an opaque session identifier every subdomain validates against a shared session store, or no shared cookie at all, with each subdomain independently validating a short-lived access token on its own requests and refreshing it via the shared refresh-token flow when near expiry. - Recommendation: the token approach for API/service-to-service calls, stateless, scales without a shared session-store round trip on every request, and the cookie or a lightweight session-existence check for the browser-facing session itself, so a user isn't forced to re-authenticate per subdomain. These solve genuinely different problems, is there a live browser session, versus, is this specific API call authorized.
Handling 50,000 authorization requests per second
- The authorization-code exchange and MFA steps happen once per login, not once per request, so the 50k/sec figure is really about validating already-issued access tokens on ongoing requests, not re-running the full OIDC flow 50,000 times a second. Validating a signed JSON Web Token (JWT) access token is a local signature check against a cached public key, not a network call, which is exactly what makes this throughput achievable without every validating service round-tripping to the IdP.
- Horizontally scale the gateway and token-issuance tier statelessly behind a load balancer; the only genuinely shared state is the session and refresh-token store, which needs its own horizontally scaled, low-latency backing store, since it becomes the real bottleneck candidate once the stateless tiers are scaled out.
Token flows for ID and access tokens, differentiated by client type
- Server-side (confidential) client: standard OIDC authorization code flow with a client secret. The ID token, a signed JWT identifying who the user is (subject, tenant,
amr), is consumed once by the server to establish its own session, and the access token used to call downstream APIs is held server-side, never exposed to the browser. - Single-page application (public) client: authorization code flow WITH PKCE, no client secret, since a public client cannot keep one, because the older implicit flow (tokens returned directly in a redirect fragment) is now legacy specifically because it exposes tokens in browser history and logs and has no code-exchange step to bind the token request to a specific, verified request. PKCE closes that gap without needing a secret the application can't safely hold. The application receives its own short-lived access token directly and must handle refresh, below, without ever holding a client secret.
Refresh token handling
- Refresh tokens are long-lived relative to access tokens, so they need proportionally stronger protection: store them server-side only whenever the client architecture allows it (trivial for server-side clients; for single-page applications, this is the actual argument for a backend-for-frontend pattern that holds the refresh token on the application's behalf, rather than the browser code holding it directly).
- Implement refresh-token rotation with reuse detection: each refresh issues a NEW refresh token and invalidates the old one. If an already-invalidated refresh token is ever presented again, treat it as a signal of token theft, someone replaying a stolen, stale refresh token, and revoke the entire token family, not just that one token, forcing re-authentication.
Tenant isolation strategy
- Every token, ID and access, carries a
tenant_idclaim set by the gateway at issuance from the tenant resolved during login, never from client-supplied input, and every downstream service validates that claim against the tenant/resource context of the request being made, so a valid token for tenant A can never be replayed against tenant B's resources even if the signing key is shared platform-wide. - Session-store and cache keys are namespaced by
tenant_idas a first-class part of the key, not an afterthought filter applied after a broader lookup, so a bug in one tenant's session-store query cannot accidentally return or invalidate another tenant's session.
High availability (HA) and scalability considerations
- No single point of failure in the gateway/token-issuance tier, stateless, horizontally scaled, deployed across multiple availability zones. The shared session/refresh-token store needs multi-node replication with a defined consistency model, eventual is usually acceptable for session reads, given a session's own short practical lifetime already bounds the cost of a brief staleness window.
- A specific tenant's own IdP being unavailable should degrade gracefully for THAT tenant only, new logins for that tenant fail clearly, without affecting any other tenant's ability to log in, which the tenant-scoped IdP registry above structurally guarantees as long as IdP calls are made per-tenant, not through a shared blocking call.
Testing SSO failover
- Chaos-test the token-issuance tier by killing individual gateway instances under load and confirming the load balancer routes around them, with no failed logins beyond the in-flight requests to the killed instance, measured, not assumed.
- Simulate a single tenant's IdP outage (point a test tenant's IdP config at an unreachable endpoint) and confirm that tenant's logins fail fast with a clear error while no other tenant's login success rate is affected, directly testing the tenant-isolation claim above under an actual failure, not just normal operation.
- Test the refresh-token reuse-detection path explicitly: replay an already-rotated refresh token in a test environment and confirm the entire token family is revoked, not just silently rejected once.
Worked example
Capacity arithmetic for validating 50,000 authorization requests per second via local JWT signature verification. The per-instance verification rate is a stated assumption, labeled as such, not a measurement; only the resulting instance count is the derived claim:
import math
target_requests_per_sec = 50_000
per_instance_verifications_per_sec = 20_000 # stated assumption, not a benchmark
instances_no_headroom = math.ceil(target_requests_per_sec / per_instance_verifications_per_sec)
instances_with_n_plus_1 = instances_no_headroom + 1
print(f"target: {target_requests_per_sec:,} authorization requests/sec")
print(f"instances at {per_instance_verifications_per_sec:,}/sec/instance: {target_requests_per_sec/per_instance_verifications_per_sec:.1f} -> {instances_no_headroom} (rounded up)")
print(f"with N+1 redundancy: {instances_with_n_plus_1} instances")
Output (actually run):
target: 50,000 authorization requests/sec
instances at 20,000/sec/instance: 2.5 -> 3 (rounded up)
with N+1 redundancy: 4 instances
Three validating instances cover the raw target at this assumed per-instance rate; four gives standard N+1 redundancy, tolerating any single instance failure without dropping below the target throughput.
Trade-offs and pitfalls
- Choosing stateless per-request access-token validation for the API path scales well but pushes complexity onto every downstream service, each must correctly validate signature, expiry, tenant_id, and audience claims. A single service that gets this wrong is a tenant-isolation bug waiting to happen, so this logic belongs in a shared library or sidecar, not reimplemented per service.
- A shared domain-scoped cookie for session propagation only works within a single registrable domain; a platform that lets tenants use their own custom domains (white-labeling) breaks this pattern entirely and needs a token-based, not cookie-based, session-propagation strategy for those tenants specifically.
- Refresh-token rotation with reuse detection adds real complexity, tracking token families and distinguishing a legitimate near-simultaneous refresh race (a flaky mobile network retrying) from actual theft. A naive implementation can revoke a legitimate user's session on a benign race; test this path deliberately, including the benign case, not just the theft case.
- Testing SSO failover only against your own gateway's failure modes misses the most common real failure in a multi-tenant SSO system: one specific tenant's IdP being slow or down, not the whole platform. Failover testing needs a per-tenant IdP-outage scenario as its own first-class test, not just a generic "kill an instance" chaos test.
Define permission boundaries and explain how they differ from IAM role policies and resource policies in major cloud providers. Provide an example scenario where permission boundaries can prevent privilege escalation for delegated IAM administrators who otherwise could grant too-broad permissions.
Sample Answer
Direct answer
A permission boundary is an Identity and Access Management (IAM) feature that sets the maximum permissions an identity-based policy is ever allowed to grant a specific user or role. It doesn't grant anything by itself; it caps what any policy attached to that principal can grant, which makes it fundamentally different from a role policy, which actively grants permissions, and from a resource policy, which grants access from the resource's own side.
Structured elaboration
How it differs from role policies. A role's identity-based policy is additive: it grants whatever actions and resources it lists. A permission boundary is a ceiling evaluated alongside it: for an action to actually succeed, it must be allowed by both the identity-based policy and the permission boundary at the same time. Attaching a broad identity policy to a role does nothing beyond what the boundary also independently allows, which is exactly the point of using one.
How it differs from resource policies. A resource policy attaches to and is evaluated on behalf of the resource, deciding who may touch it, potentially reaching across account boundaries. A permission boundary attaches to and caps a specific principal, regardless of which resource that principal touches. One is resource-centric, the other principal-centric, and a boundary does not grant cross-account access the way a resource policy can; it only ever narrows what the principal it's attached to can do.
Scenario preventing privilege escalation. A common real situation: a team lead is delegated the ability to create IAM roles and policies so their team can self-serve provisioning, without that delegated administrator being able to grant themselves, or any role they create, permissions the organization never intended, including full administrative access. Without a permission boundary, a principal who can create and attach IAM policies can trivially self-escalate: create a new role, attach an administrator-access policy to it, and use it. Attaching a permission boundary to every role that delegated administrator is allowed to create, enforced by a condition on their own role-creation permission requiring any new role to carry a specific boundary policy, caps whatever identity policy they attach at that boundary's ceiling; even an administrator-access policy on the new role can't exceed it, closing the self-escalation path.
Worked example
The delegated administrator's role is granted iam:CreateRole, iam:PutRolePolicy, and iam:AttachRolePolicy, but only on the condition that any role created carries iam:PermissionsBoundary equal to a specific policy, DevTeamBoundary, that allows only a defined set of development-related actions and explicitly excludes IAM and other administrative actions:
{
"Effect": "Allow",
"Action": ["iam:CreateRole", "iam:PutRolePolicy", "iam:AttachRolePolicy"],
"Resource": "*",
"Condition": {
"StringEquals": {
"iam:PermissionsBoundary": "arn:aws:iam::123456789012:policy/DevTeamBoundary"
}
}
}
If the delegated administrator creates a role with an administrator-access policy attached, and the required boundary, the role is created, but any administrator-granted action is blocked at evaluation time by the intersection with the boundary, since the boundary's own policy never included those actions in the first place. If they instead try to create a role without the required boundary attached, the iam:CreateRole call itself is denied by the condition, closing that escape route too.
Trade-offs and pitfalls
The most common early confusion is treating a permission boundary as a grant: attaching only a boundary with no identity policy at all gives a principal literally nothing, since the boundary only ever narrows, never grants. A more serious pitfall is leaving the boundary policy itself, or the condition requiring its use, editable by the delegated administrator: if they can also modify or detach the boundary, or edit the condition enforcing it, the entire control is void, so the boundary policy and the enforcing condition must live outside the delegated administrator's own permission set entirely. A boundary defined too permissively "just to be safe" can end up functionally equivalent to no ceiling at all, defeating its purpose without anyone noticing until it's tested. Finally, permission boundaries must be applied identity by identity, which is a real operational trade-off against an organization-wide guardrail like a service control policy that applies automatically to every principal in scope, including ones created tomorrow: boundaries give more per-team customization, but a missed role when delegating a new administrator is a genuine coverage gap unless attaching one is itself automated or enforced.
Design a CI/CD pipeline access model where build agents and deployment jobs have just enough privileges for each pipeline stage. Explain how to provision ephemeral credentials per job, inject secrets securely at runtime (without storing them in plain text in logs), sign and verify build artifacts, and prevent credential leakage. Describe integration with secret managers, workload identity federation, and artifact attestation.
Sample Answer
Direct answer
Give each pipeline stage its own narrowly scoped, short-lived credential, minted just before that stage runs and expiring shortly after, obtained through workload identity federation rather than a stored static secret; inject any further secrets into the job's runtime environment, never its source or logs, through a dedicated secrets manager or key-management service; and treat the build artifact itself as something that must be cryptographically signed and later verified before deployment, not merely produced.
Structured elaboration
Ephemeral credentials per job. Rather than storing a long-lived cloud credential inside the CI/CD (continuous integration and continuous delivery) system's own secret store, have the build agent present its pipeline's own OpenID Connect (OIDC) identity token, which most modern CI/CD platforms can mint fresh per job, to a workload identity federation endpoint, which exchanges it for a short-lived, narrowly scoped cloud credential valid only for that specific stage, typically for minutes. Scope each credential to exactly what its stage needs: a build stage gets read access to source and write access to an artifact repository, a deploy stage gets write access to the target environment and nothing else, so a compromised build stage can never also deploy to production. This is also where developer single sign-on (SSO) and cross-account role assumption fit together: a human developer triggering a pipeline authenticates through the organization's own SSO, while the pipeline's execution identity separately assumes a distinct, narrowly scoped role per target account, for example a separate role for a build account, a staging account, and a production account, so a developer's own broad SSO identity is never what actually executes a cloud action; the pipeline's own per-stage assumed role is.
Injecting secrets without leaking them. Pull any additional secrets, a database password, a third-party API key, from a dedicated secrets manager or a key-management-integrated vault at job start, inject them as environment variables or mounted files scoped to that job's own process, and never let them reach build logs. Most CI/CD platforms automatically mask values sourced through their own secret-reference mechanism, but that masking only covers the original value; it does nothing for a script that prints a transformed or derived copy of the same secret, which is the common, unglamorous way a properly masked secret leaks anyway. Where the organization already runs a secrets manager such as HashiCorp Vault, or a cloud key-management service (KMS) for its encryption keys, integrate the pipeline against that same system rather than standing up a second, parallel secret store, so there is exactly one place secrets are issued, rotated, and audited from.
Signing and verifying build artifacts. Sign every build artifact, a container image or a package, with a key the build stage alone can access, ideally itself short-lived or tied to that specific pipeline run's identity rather than a long-lived signing key shared across every build, and attach the resulting signature plus provenance metadata, which source commit, which pipeline run, which build environment produced it, as an attestation. The deploy stage verifies both the signature and the attestation before deploying anything, refusing an artifact that wasn't produced by this exact pipeline or whose signature doesn't match, which closes the gap where a malicious or simply mismatched artifact could otherwise be substituted between the build and deploy stages.
Preventing credential leakage. The ephemeral-credential design above is itself the primary defense, since a credential scoped to one stage and a few minutes has almost nothing left to leak once it expires; scrubbing build logs for anything resembling a token or key pattern is a reasonable second layer; and treating the build environment itself as untrusted between runs, tearing it down and recreating it fresh for each job rather than reusing a long-lived build machine, prevents a credential from a previous run lingering on disk into the next one.
Approval gates and commit-signature verification. Require any deployment to a production-target stage to pass through an explicit approval gate that checks a policy condition, for example requiring the triggering commit to carry a signature from a verified, authorized developer key, confirming the change actually originated from someone authorized rather than an unsigned or spoofed commit, before the production-scoped ephemeral credential is ever issued. The approval gate and the credential-issuance step should be the same control point, not two independent checks that could disagree: a failed approval should mean the credential is never minted at all, not merely that a warning gets logged somewhere alongside it.
flowchart LR
Dev["Developer (SSO login)"] --> Trigger["Pipeline trigger (signed commit)"]
Trigger --> Build["Build stage: ephemeral cred, source read + artifact write"]
Build --> Sign["Sign artifact + attestation"]
Sign --> Gate["Approval gate: verify commit signature"]
Gate --> Deploy["Deploy stage: ephemeral cred, target-account write only"]
Vault["Secrets manager / KMS"] -. injects secrets .-> Build
Vault -. injects secrets .-> Deploy
Worked example
A container-image pipeline: the build stage's job token is exchanged, via workload identity federation, for a credential scoped to read the source repository and push to the artifact registry, valid 15 minutes. Once the image builds, the pipeline signs it with a key scoped to that specific run and attaches provenance attesting the source commit and build environment. Before deployment, the approval gate checks that the triggering commit carries a verified signature from an authorized developer; if it does not, the pipeline halts and no production credential is ever requested. On approval, the deploy stage's own job token is exchanged for a separate, narrower credential scoped only to the production account's deployment role, which first verifies the image's signature and attestation match this exact pipeline run before pulling and deploying it.
Trade-offs and pitfalls
The most damaging pitfall is a build script that correctly fetches a secret through the platform's masking mechanism, then prints a transformed or derived version of it, a base64-encoded copy, a substring, that the masking doesn't recognize and leaks into logs anyway; masking helps, but it is not a substitute for disciplined script behavior. A second is granting the deploy stage a broad, standing credential "to keep things simple," which defeats the entire point of stage-scoped ephemeral credentials the moment any single stage is compromised. A third is treating artifact signing as a checkbox without actually verifying the signature and attestation at deploy time, which leaves signing providing an audit trail after an incident rather than any actual protection before one. The core trade-off: this design carries real setup cost, workload identity federation configuration per environment, a signing and attestation pipeline, an approval gate wired to commit-signature verification, which is proportionate for a production deployment pipeline but likely excessive for a low-stakes internal tool, where a simpler secrets-manager-only approach may be the right, deliberately less rigorous choice.
Design role-based access control (RBAC) for a corporate admin dashboard that manages account policies, billing, and reporting across multiple subsidiaries and cost centers. Describe role hierarchy, permission scoping, delegation patterns, audit logging, and how you'd represent inherited permissions to administrators.
Sample Answer
Direct answer
Model roles as a shallow hierarchy where each level inherits everything below it plus its own additions, keep the resource scope (which subsidiary or cost center a grant applies to) as a separate attribute from the role itself rather than baked into the role's name, treat delegation as a distinct, time-bounded, audited action rather than a quiet permanent reassignment, and log every permission-affecting change with enough context to reconstruct "who could do what, and why" long after the fact.
Structured elaboration
Role hierarchy. Define a small number of base roles, for example Organization Admin above Subsidiary Admin above Cost-Center Manager above Viewer, where each level inherits the capabilities of the level below it plus whatever it adds. Represent this as an explicit graph or tree, not as a flat table of independently-maintained role-permission pairs, so a new capability only needs to be added once at the correct level rather than copied into every affected role. Keep the hierarchy shallow, roughly four to five levels at most: every additional level makes "why does this person have this permission" harder for an administrator to reason about, which is a real operational cost even when the model itself stays technically correct.
Permission scoping. Every grant is really a triple: role, resource scope, and the assignment itself, not just a role name. A "Billing Admin" role has to be granted per subsidiary, so that a corporate administrator can hold Billing Admin for Subsidiary A without automatically holding it for Subsidiary B. Implement scope as an explicit attribute on the assignment record (user, role, scope) rather than encoding it into the role's name (a role called "Subsidiary-A-Billing-Admin" duplicates the entire role catalog once per subsidiary and stops scaling the moment a new subsidiary is added).
Delegation patterns. Keep delegation structurally distinct from a permanent role assignment. When a Subsidiary Admin goes on leave and hands a subset of their scope to a peer for two weeks, that should create its own record (delegator, delegate, the specific scope being delegated, and an expiry date) that lapses automatically, rather than a manual role reassignment someone has to remember to undo. A delegation can never exceed what the delegator themselves currently holds, and either the delegator or an Organization Admin should be able to revoke it early.
Audit logging. Log every permission-affecting event (a role granted or revoked, a delegation started or ended, a scope changed) with the actor, the target, the before and after state, and a timestamp, retained separately from the operational database so the log survives even if the operational data is later modified. This is exactly the data a multi-subsidiary compliance review or a separation-of-duties recertification will need. Just as important: log actual use of high-risk permissions, not only the grant of them, since "user X was granted Billing Admin" cannot answer "was Billing Admin ever actually exercised, and for what."
Representing inherited permissions to administrators. The single biggest usability risk in any hierarchy is an administrator not realizing a permission is inherited rather than directly granted. The admin console should show, for any given user, their full effective permission set with each entry's origin explicitly labeled: granted directly at this scope, inherited from a role level above, or delegated by a named person until a specific date, rather than a flat checklist that hides where a permission actually came from. A "trace" view that walks the inheritance chain for a single permission back to its source is the single highest-leverage feature for both troubleshooting an access complaint and answering an auditor's question.
flowchart TD
OA["Organization Admin<br/>(all subsidiaries)"] --> SA1["Subsidiary Admin: EMEA"]
OA --> SA2["Subsidiary Admin: APAC"]
SA1 --> CC1["Cost-Center Manager: EMEA-Marketing"]
SA1 --> V1["Viewer: EMEA-Marketing"]
SA1 -. delegated, expires in 14 days .-> D1["Peer admin (temporary)"]
Worked example
User jsmith is granted Subsidiary Admin scoped to EMEA. The effective permission view for jsmith would show:
| Permission | Origin |
|---|---|
| View EMEA billing | Directly granted (Subsidiary Admin, EMEA scope) |
| Approve EMEA cost-center budgets | Inherited from Subsidiary Admin level |
| View global reporting dashboard | Inherited from Cost-Center Manager level below (rolled up) |
| Approve APAC billing | Delegated by asingh until 2026-08-11 |
Reading this table, an auditor can immediately see that jsmith's APAC access is not a mistaken scope leak but a time-bounded delegation with a named delegator and an expiry date, which is exactly the distinction a flat permission checklist would hide.
Trade-offs and pitfalls
The most common structural mistake is baking scope into role names instead of keeping it as a separate attribute: it looks fine with three subsidiaries and becomes an unmanageable role catalog by the twentieth. A close second is implementing delegation as a quiet, permanent role reassignment rather than a genuinely time-bounded record; this is the single most common way "temporary" access becomes permanent access nobody remembers granting, and it is exactly the failure a separation-of-duties audit exists to catch. Deep inheritance chains are technically elegant but operationally opaque to the administrators who actually have to reason about them day to day, so the model should trade some theoretical elegance for a hierarchy shallow enough that a person can hold the whole thing in their head. Finally, there is a real trade-off between centralizing all scoping logic in the application layer, which keeps one source of truth but means the application must never have a bug that bypasses its own check, versus also enforcing some of it at the database layer (row-level security keyed on scope), which adds defense in depth at the cost of having two places that must stay consistent with each other.
Design a Privileged Access Management (PAM) architecture that provides secure shell and console access across on-prem and cloud systems. Include vaulting of credentials, session brokering, just-in-time elevation, session recording/forensics, approval workflows, and integration with SIEM and IdP.
Sample Answer
Direct answer
The architecture centers on one mandatory broker that every administrator must pass through to reach any target, whether that target is an on-premises server over secure shell (SSH) or a cloud provider's console, so that vaulting, approval, elevation, and recording can all be enforced at one narrow chokepoint rather than dozens of direct paths. The broker authenticates the requester through the organization's existing identity provider (IdP), checks that a just-in-time elevation request was approved, retrieves or generates a credential from the vault (a short-lived certificate where the target supports it, a vaulted password otherwise), proxies the full session while recording it, and streams every event to the security information and event management (SIEM) system for correlation and alerting.
Structured elaboration
Vaulting and credential issuance, on-prem and cloud alike. For systems that support certificate-based SSH, the vault issues an ephemeral SSH certificate signed by an internal certificate authority, scoped to one user, one target host, and a short time-to-live (TTL), so no long-lived static SSH key is ever distributed to an administrator's machine at all. For legacy systems that only support password authentication, the vault stores the credential and injects it directly into the brokered session without ever displaying it to the user, rotating it automatically after each use. Cloud targets follow the same pattern using each provider's own native mechanism where one exists (a session-management service that doesn't require an inbound SSH port at all), with the PAM layer's approval, recording, and audit wrapper applied on top rather than administrators using that native mechanism directly and unrecorded.
Session brokering. The broker does not simply authenticate a connection and step aside; it proxies the actual SSH or console protocol traffic for the entire session's duration. This is what makes full session recording possible and is also what lets the broker enforce elevation expiration mid-session (if a just-in-time window ends while a session is still open, the broker can terminate it), rather than only checking permissions once at connection time.
Just-in-time elevation and approval workflows. An administrator authenticates to the PAM portal through the organization's IdP (single sign-on, ideally with multi-factor authentication (MFA) already enforced at that layer), then submits a request naming the specific target, the duration needed, and a justification. The request routes to an approver, resolved through the IdP's own group or ownership data (the team that owns the target system) rather than a hard-coded individual name that goes stale as people change roles. On approval, the vault issues the scoped credential and the elevation window begins; it expires automatically, and the broker enforces that expiration against any still-open session.
Session recording and forensics. Every brokered session, SSH terminal input and output, or a cloud console's screen activity, is recorded and stored in a tamper-evident, access-controlled forensic store, indexed by who connected, which target, when, which approval or ticket authorized it, and how long the session lasted. This indexing is what makes a recording actually useful during an investigation: a security analyst needs to search "every session against this host in the last 48 hours," not scroll through an undifferentiated pile of video files.
Integration with the security information and event management (SIEM) system. Every PAM event, login, elevation request, approval or denial, session start and end, and specific high-risk actions detected inside a session (a destructive command pattern, for example), streams to the SIEM in near real time. This is what turns privileged-access logging from an after-the-fact audit trail into an active detection surface: the security operations team can correlate a privileged session against other telemetry (an unusual outbound network connection immediately following a session, for instance) instead of only reviewing PAM logs once an incident is already suspected through some other means.
Bridging on-premises and cloud reachability. On-premises systems typically sit behind a network boundary the broker cannot reach directly from outside, so a lightweight relay or agent installed inside the on-premises network establishes an outbound connection to the broker, avoiding the need to open an inbound path into the internal network. Cloud targets, by contrast, are usually reachable through the cloud provider's own API surface, so the broker calls that provider's session mechanism directly. The architectural point is that both paths terminate at the same broker, vault, approval workflow, and recording pipeline, so security operations has one consistent audit trail across on-premises and cloud rather than two disconnected access models that have to be reconciled separately during an investigation.
Worked example
flowchart LR
Admin["Administrator"]
IdP["Identity provider: SSO + MFA"]
Portal["PAM portal: JIT request"]
Approver["Approver: resolved via IdP group"]
Broker["Session broker / bastion"]
Vault["Credential vault: SSH cert issuance / password injection"]
OnPrem["On-prem relay agent"]
Target["On-prem SSH host / cloud console"]
Recording["Session recording store"]
SIEM["SIEM"]
Admin -- "authenticate" --> IdP
Admin -- "request access" --> Portal
Portal -- "routes for approval" --> Approver
Approver -- "approved, bounded window" --> Vault
Admin -- "connects through" --> Broker
Broker -- "fetches ephemeral credential" --> Vault
Broker -- "proxies session via relay" --> OnPrem
OnPrem --> Target
Broker -- "records full session" --> Recording
Broker -- "streams every event" --> SIEM
Concretely: during an incident, a site reliability engineer (SRE) needs emergency root access to an on-premises database server. They authenticate through the IdP into the PAM portal, request access naming the incident ticket, and the request routes to the on-call database team lead (resolved via IdP group membership, not a hard-coded name) for approval. On approval, the vault issues a one-hour SSH certificate scoped to that one server, and the broker proxies the SSH session through the on-premises relay agent, recording the full terminal session. The same engineer separately needs console access to a cloud virtual machine in a different account for the same incident; the broker calls the cloud provider's own session mechanism, wrapped in the same approval and recording pipeline, so the resulting audit trail looks identical in shape to the on-premises session despite the underlying transport being completely different. Every step, request, approval, session start, session end, streams to the SIEM, so a security analyst reviewing the incident afterward sees both accesses correlated against the same incident ticket in one place.
Trade-offs and pitfalls
- The broker becomes a single, highly consequential point of failure and a high-value target. Concentrating all privileged access through one chokepoint is exactly what makes the other controls enforceable, but it also means a broker outage blocks legitimate access to everything behind it, and a broker compromise is catastrophic. This has to be designed with a highly available broker cluster and a deliberately separate, tightly controlled break-glass path that bypasses the broker entirely for genuine emergencies, since depending on the broker to grant emergency access to a broken broker is a contradiction.
- The on-premises relay agent is a new operational dependency layered onto every on-prem target. If it goes down, brokered access to that environment goes down with it, so the emergency path for on-premises systems needs to be tested independently of the relay's normal availability, not assumed to always be there.
- Recording every session at full fidelity has real storage cost and retention implications, and in some jurisdictions or industries, recording certain kinds of session content carries its own compliance and privacy considerations; retention duration and access to the recordings need their own deliberate policy, not a default "keep everything forever" setting inherited from the tool's defaults.
- Streaming every event to the SIEM is only valuable if detection rules are actually built on top of the stream. A common failure mode is treating SIEM integration as complete once the logs are flowing, without anyone building the correlation rules (an unusual destructive command inside a session, a session immediately followed by anomalous egress) that turn the stream into actual early detection rather than passive archival.
- Approval routing based on IdP group membership can silently go stale. If a target system's owning team changes and the IdP group is not updated to match, approvals can route to the wrong team, or to a group with no active members, which fails safe in the sense that access is blocked, but it can also block a genuine emergency at the worst possible time if nobody notices the staleness beforehand.
Unlock Full Question Bank
Get access to all Identity, Authentication, and Access Management interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.