Secure Architecture and Design Principles Questions

Direct answer

Separation of duties (SoD) means dividing a sensitive process among different people so that no single person can carry it out, or hide it, alone. It protects against fraud and honest mistakes. A conflict of duties exists when one person holds two or more capabilities that should be separate. For a database administrator (DBA), the classic conflict is power over the data combined with power over the evidence and the approvals.

What a DBA conflict looks like

Controls to mitigate it

1. Split administrative roles: operational DBA (keeps the database running), security administrator (grants access, manages audit settings) and an auditor role that can only read logs.
2. Audit logs written to a separate system the DBA cannot edit, owned by the security team, in write-once storage.
3. Changes via ticket and peer review, applied by a pipeline.
4. Individual named accounts and a privileged access management (PAM) tool that records admin sessions and issues time-limited access.
5. Separate key custody: encryption keys administered by a different team than the database. Mask sensitive columns for routine work.
6. Independent review: someone outside the DBA team samples DBA activity logs every month (a detective control).

Example: DBA Sam wants to add a column. Sam raises a ticket; a colleague reviews it; the pipeline runs it; the log goes to the security team's system. Sam cannot alter or erase that record.

When the team is too small to split

Use compensating controls: management review of privileged activity and external log storage, documented as an exception.

Direct answer

I would stop new over-permissioning at the source with organization-level guardrails and paved-road defaults, then shrink existing excess using usage data, so that least privilege is the easy default rather than a periodic cleanup. Developers keep self-service inside a safe boundary, so security stops being a ticket queue.

Mechanisms, in the order I would roll them out

1. Organization guardrails (SCPs, service control policies: a permission ceiling no account below can exceed): deny the things nobody legitimately needs, such as disabling audit logging, creating long-lived access keys for users, or using unapproved regions.
2. Permission boundaries for delegated role creation. Teams may create their own roles, but only up to a maximum set of permissions, so self-service cannot become privilege escalation. A permission boundary is a policy attached to a role that sets its ceiling without granting anything: effective access is only what both the role's own policies and the boundary allow. For example, with a boundary that allows only s3:*, dynamodb:* and sqs:*, a team role that was given iam:* and s3:* ends up with S3 access only, because IAM actions fall outside the ceiling.
3. Paved-road templates. A paved road is the platform team's pre-approved, ready-made way to build and deploy a service, so the secure choice needs no extra effort. The standard infrastructure-as-code module (a reusable template that defines cloud resources as code) for a new service creates its own role with only listed actions on named resources and short-lived credentials. The easiest path is the safe path.
4. Policy checks in the pipeline. A change that adds a wildcard action or grants new access beyond an approved baseline fails the pull request (cloud providers offer policy-validation and policy-comparison checks; AWS IAM Access Analyzer provides both).
5. Right-sizing from usage. Unused-access findings (roles, keys and permissions not used within a window you set, such as 90 days) open automatic pull requests to remove them. A typical before and after, using illustrative names:

Before: Action "s3:*"                          Resource "*"
After:  Action ["s3:GetObject","s3:PutObject"]  Resource "arn:aws:s3:::orders-uploads/*"

The access logs showed the service only ever read and wrote objects in one bucket, so everything else could go.
6. Exceptions expire. Broad access is granted with an owner and an end date, enforced by the platform, rather than reviewed by hand.

Patterns across service types

Keeping it valid over time

Policies live in version control; the pipeline blocks regressions; unused-access findings flow to owners automatically; and a small sample audit (not a full review) checks the automation is working.

Not slowing developers

Measure request-to-working-permission time. A boundary that lets a team self-serve in minutes removes the incentive to ask for admin "just in case".

Existing creep

Do not try a big-bang cleanup. Fix the most dangerous first: roles with administrator-equivalent rights and cross-account trust (a role that principals in another cloud account are allowed to assume), then the unused ones, with usage data proving removal is safe.

Direct answer

"Secure by design" means security properties are built into the structure of the system before features are written, so a mistake by one developer or one compromised component cannot automatically become a breach. It is about decisions that are cheap on day one and expensive to retrofit. For a new microservices platform (an application split into many small services that call each other over the network), I would make three decisions first.

Three day-one decisions and what each removes

1. Every service gets its own cryptographic identity, and every call is mutually authenticated and encrypted. Mutual TLS (mTLS) means both sides of a connection present certificates, so a service proves who it is, not just the server. Pair it with a default-deny network policy: a service can call only the services on its allow list. As a concrete rule, in Kubernetes that is a NetworkPolicy that first denies all inbound connections to every service in the namespace, then adds one allow rule such as "payments accepts connections only from orders"; anything not listed is refused. Do the same for outbound traffic (egress): deny by default and allow each service only the destinations it needs, so notifications can reach its email provider and nothing else. Removes: spoofed internal calls, sniffing of east-west traffic (service-to-service traffic inside the platform), and free lateral movement (an attacker hopping from one compromised service to the next).
2. Least privilege per service, with short-lived credentials. Each service has its own role that can touch only its own database and queues, and its credentials expire in minutes or hours and are issued automatically, not stored in config. Removes: the "one stolen shared secret opens everything" failure and long-lived leaked keys.
3. Authorization is checked in every service, not only at the front door. The gateway (the single entry point for outside traffic) authenticates the user and passes a signed token with their identity inward (a token is a small data package saying who the user is, signed by the identity service so it cannot be altered; each service checks the signature with the issuer's public key and also checks that the token has not expired and was meant for it). Each service then decides whether this user may do this action, instead of trusting that "the request came from inside". Removes: broken access control caused by trusting network location, and the confused-deputy problem (a legitimate service tricked into using its own privileges on an attacker's behalf).

Worked example

Suppose a vulnerable library lets an attacker run code inside the notifications service. With these decisions, the attacker holds only that service's identity: the egress policy lets it reach the email provider and nothing else, and payments accepts connections only from orders, so a direct call to payments is dropped at the network layer. Suppose a policy gap let the packet through anyway: payments would still reject it, because it verifies the token's signature and expiry and then finds no user with refund rights in it. Either way, the compromised service's role cannot read the payments database. One bug stays one bug.

Making it stick

I would ship these as a paved road (a service template and platform defaults) so teams get them without effort, and enforce them in CI (continuous integration, the automated build pipeline) so drift fails a build.

Trade-offs and pitfalls

• mTLS and per-service roles add operational work (certificate rotation, policy management). A service mesh (a layer of small proxies running beside each service that handles encryption, certificates and retries so application code does not) or other platform automation absorbs most of it.
• Do not choose "a WAF (web application firewall, a filter that blocks known attack patterns in web requests) at the edge" as a day-one answer. It is a useful layer, but it does nothing about a compromised internal service.
• What would change my choices: a very small platform with two services might defer the mesh and start with network policy plus per-service roles.

Direct answer. I would not choose between "apply as designed" and "weaken it" before measuring. First I find out what the friction actually costs, then I look for ways to keep the security property while removing the cost, and only then compensate elsewhere. The decision involves product, the engineers who own latency, and the risk owner, and security does not decide alone.

Key terms. Friction is any step or delay a legitimate user or request pays to get a security property. A compensating control is a different safeguard that covers the same risk when the preferred control is weakened. p95 latency is the time under which 95 percent of requests complete. An identity provider is the service that logs users in and vouches for who they are. Revocation is cancelling a token or session that has not yet expired, for example after a device is reported stolen; with caching, a revoked token can keep working until the cached answer expires. A risk owner is the person with authority to accept the leftover risk. A replayed credential is a captured valid credential re-sent by an attacker.

Step 1: Name the property you need. "Strong authentication on every request" is a means. The property is "a stolen or replayed credential cannot be used for long, and sensitive actions need fresh proof". Once stated, other designs can deliver it.

Step 2: Measure the cost on the actual path. Illustrative numbers: the p95 budget is 300 ms and a remote token check (a network call to the identity provider on every request) adds 30 ms, which is 30/300 = 10 percent of the budget. At 2,000 requests per second, checking with the identity provider on every request means 2,000 calls per second to it, which is also a reliability dependency.

Step 3: Look for a redesign that keeps the property.

• Validate a short-lived signed token (JSON Web Token) locally at the service, so no network call per request: the token carries a signature made with the identity provider's private key, and the service checks it using the matching public key it already holds, so it needs no call to the provider. The limit is a revocation lag as long as the token's lifetime (illustrative: a 5-minute token can be misused for up to 5 minutes after revocation), which may be longer or shorter than the 60-second cache described next, so compare the two before choosing. This describes tokens signed with a private key; a token signed with a shared secret would force every service to hold the secret that can also forge tokens.
• Cache the decision per session for 60 seconds. With 10,000 active sessions, that is at most 10,000/60, about 167 checks per second instead of 2,000, a 12 times reduction. The trade-off is that revocation can lag by up to a minute.
• Keep the expensive proof (a fresh multi-factor prompt, called step-up authentication) for sensitive actions such as changing payout details, not for every page view.

Step 4: Choose among three outcomes.

Who I involve. The product owner (user impact), the performance or SRE owner (the latency budget is theirs), the identity team (what is feasible), and the risk owner who signs if we weaken or compensate. The same logic applies to inspection controls, such as scanning every request body for malicious content or leaked sensitive data. Full inspection of every payload has a latency and cost price, so inspect fully where data is sensitive, and elsewhere inspect a sample, or inspect asynchronously (copy the traffic and analyse it after the response has been sent, so users do not wait, at the cost of catching problems later).

Pitfalls. Weakening a control with no written statement of what was lost. Treating latency as untouchable or security as untouchable; both are trade-offs to be priced.

Direct answer

Controls come in three jobs. Preventive controls stop a bad event from happening. Detective controls notice that it is happening or has happened. Corrective controls limit the damage and restore normal operation. A sound design uses all three because prevention is never perfect, so you need to see failures and be able to act on them. The dangerous gap is a mix with no working detection or no working correction: you cannot respond to what you cannot see, and seeing without a way to act only gives you better information about the loss.

Customer-facing web app, worked example: credential stuffing (attackers replaying passwords leaked elsewhere)

More examples across the stack:

• Cloud storage: a rule that blocks public buckets (preventive), an alert when a bucket policy changes (detective), an automated job that reverts the change (corrective).
• Data: least-privilege database accounts (preventive), alerts on unusually large exports (detective), restore from tested backups (corrective).
• Code: dependency checks in the build pipeline (preventive), runtime anomaly monitoring (detective), a one-step rollback (corrective).

How I balance them

Weight toward prevention where it is cheap and reliable (patching, secure defaults). Weight toward detection and correction where prevention cannot be guaranteed: an attacker holding a valid stolen password looks like a normal user, so detection and fast session revocation matter more than another input filter.

Dangerous gaps

• Preventive only: a breach can run for months unseen.
• Detective without correction: alerts fire but nobody can disable the account or restore data quickly.
• Detective without an owner: the alert goes to an unwatched mailbox at 2 a.m.
• Corrective never tested: the backup that fails on restore day.

How a tester validates each class

Preventive: try to bypass it. Detective: run the attack in a safe test and confirm the alert fired, how fast, and that a human saw it. Corrective: run a restore or revocation drill and time it.