Secure Architecture and Design Principles Questions

Direct answer

Least privilege means every person, program or service gets only the access it needs to do its current job, for only as long as it needs it, and nothing more. The point is to shrink the damage when an account is stolen or misused: a stolen account that can read one folder is a small incident, one that can administer everything is a large one.

Humans: just-in-time elevation (JIT)

Day to day, an engineer holds a low-privilege role through group membership. When they need production access, they request it for a stated reason and a short window, a second person or an automated rule approves, and the access expires on its own (for example after an hour). Everything done during the window is logged. The standing "always admin" account disappears, so a phished laptop yields little.

Machine identities: one scoped, short-lived identity per workload

A machine identity is the credential a program uses (a service account or cloud role). Give each service its own role, name the exact actions and resources, and issue short-lived credentials from the platform rather than long-lived keys pasted into config. Example for an invoice service that only reads PDFs (this is an AWS-style policy; other clouds use the same ideas under different syntax):

{
  "Effect": "Allow",
  "Action": ["s3:GetObject"],
  "Resource": ["arn:aws:s3:::acme-invoices/*"]
}

Line by line: Effect: Allow says this rule grants access; Action lists the one operation permitted, s3:GetObject, meaning read a stored file; Resource names exactly what it applies to, here every file in the one storage bucket called acme-invoices (the long arn:aws:s3:::acme-invoices/* string is AWS's unique name format for a resource, and the * means any file inside it). Anything not allowed is denied by default. It cannot list other buckets, write, or touch the database. A related capability-based pattern (holding the token is itself the permission) hands out a narrow token for one operation instead of broad credentials. For example, a pre-signed link is a web address with a signature built in that lets whoever has it download one specific file for a few minutes, with no account needed.

Where it erodes over time

• Privilege creep: Priya moves from the payments team to search but keeps her payments database access, because nobody removes access on a transfer.
• Temporary becomes permanent: an emergency wildcard policy ("Action": "*", where the asterisk means every possible action) added during an outage and never narrowed.
• Orphaned service accounts and keys from retired projects.
• Copy-pasted policies that carry more permissions than the new service needs.
• Request friction: if getting the right permission is slow, people ask for the broad one.

Counter-measures

Tie access to team membership so it moves with the person; expire grants by default; compare granted permissions to permissions actually used (cloud providers report last-used data, and AWS IAM Access Analyzer, one example tool, can generate findings for unused roles, keys and permissions; other platforms have comparable reports) and remove the difference.

Direct answer

"Secure by design" means security properties are built into the structure of the system before features are written, so a mistake by one developer or one compromised component cannot automatically become a breach. It is about decisions that are cheap on day one and expensive to retrofit. For a new microservices platform (an application split into many small services that call each other over the network), I would make three decisions first.

Three day-one decisions and what each removes

1. Every service gets its own cryptographic identity, and every call is mutually authenticated and encrypted. Mutual TLS (mTLS) means both sides of a connection present certificates, so a service proves who it is, not just the server. Pair it with a default-deny network policy: a service can call only the services on its allow list. As a concrete rule, in Kubernetes that is a NetworkPolicy that first denies all inbound connections to every service in the namespace, then adds one allow rule such as "payments accepts connections only from orders"; anything not listed is refused. Do the same for outbound traffic (egress): deny by default and allow each service only the destinations it needs, so notifications can reach its email provider and nothing else. Removes: spoofed internal calls, sniffing of east-west traffic (service-to-service traffic inside the platform), and free lateral movement (an attacker hopping from one compromised service to the next).
2. Least privilege per service, with short-lived credentials. Each service has its own role that can touch only its own database and queues, and its credentials expire in minutes or hours and are issued automatically, not stored in config. Removes: the "one stolen shared secret opens everything" failure and long-lived leaked keys.
3. Authorization is checked in every service, not only at the front door. The gateway (the single entry point for outside traffic) authenticates the user and passes a signed token with their identity inward (a token is a small data package saying who the user is, signed by the identity service so it cannot be altered; each service checks the signature with the issuer's public key and also checks that the token has not expired and was meant for it). Each service then decides whether this user may do this action, instead of trusting that "the request came from inside". Removes: broken access control caused by trusting network location, and the confused-deputy problem (a legitimate service tricked into using its own privileges on an attacker's behalf).

Worked example

Suppose a vulnerable library lets an attacker run code inside the notifications service. With these decisions, the attacker holds only that service's identity: the egress policy lets it reach the email provider and nothing else, and payments accepts connections only from orders, so a direct call to payments is dropped at the network layer. Suppose a policy gap let the packet through anyway: payments would still reject it, because it verifies the token's signature and expiry and then finds no user with refund rights in it. Either way, the compromised service's role cannot read the payments database. One bug stays one bug.

Making it stick

I would ship these as a paved road (a service template and platform defaults) so teams get them without effort, and enforce them in CI (continuous integration, the automated build pipeline) so drift fails a build.

Trade-offs and pitfalls

• mTLS and per-service roles add operational work (certificate rotation, policy management). A service mesh (a layer of small proxies running beside each service that handles encryption, certificates and retries so application code does not) or other platform automation absorbs most of it.
• Do not choose "a WAF (web application firewall, a filter that blocks known attack patterns in web requests) at the edge" as a day-one answer. It is a useful layer, but it does nothing about a compromised internal service.
• What would change my choices: a very small platform with two services might defer the mesh and start with network policy plus per-service roles.

Direct answer

I decide by tenant risk and revenue, not one model for everyone. The thousands of small customers go into pooled infrastructure, the few large regulated customers get dedicated stacks, and the same code and infrastructure template serve both, with a path to move a tenant up when needs change. The terms vary between vendors, so here are the definitions I use.

Shared and pooled are both forms of logical separation (tenants on common infrastructure, separated by policy); dedicated is physical separation.

• Shared: all tenants in one application and database, separated by a tenant identifier on every row (and database policies).
• Pooled: tenants grouped into several independent pools (also called cells: complete, separate copies of the stack, each serving a slice of tenants), each with its own database, with a limited number of tenants per pool.
• Dedicated: one tenant has its own database and compute, often its own cloud account.

Comparison

Illustrative cost arithmetic

Assume 5,000 small tenants and 3 large ones. A shared platform costing $9,000 a month is $1.80 per small tenant. Pooled at 250 tenants per pool means 20 pools; at $1,500 a month each, that is $30,000, or $6.00 per tenant. A dedicated stack at $1,800 a month (infrastructure only, excluding people) is 1,000 times the shared per-tenant cost, so it is justified only where the customer pays for it: against a $20,000 monthly contract it is 9% of revenue. A pool failure exposes 250 of 5,000 tenants, 5%, versus 100% for one shared environment. The 3 large tenants are on dedicated stacks in every scenario, adding 3 x $1,800 = $5,400 a month. Total platform bill: shared model $9,000 + $5,400 = $14,400 a month; pooled model $30,000 + $5,400 = $35,400 a month, so pooling adds $21,000 a month, or about $4.20 more per small tenant ($21,000 / 5,000), as the price of limiting a failure to 5% of tenants.

Decision rule

1. Contract or regulation requires dedicated infrastructure: dedicated.
2. Tenant is large, spiky or high-risk (a noisy neighbor, meaning a tenant whose heavy usage slows the others, or a very sensitive data class): pooled in a small pool, or dedicated if the price supports it.
3. Otherwise: pooled by default, shared only for free or trial tenants whose data is low sensitivity.

I would start the small tenants in pools rather than one shared database, because splitting a single large database later is much more painful than adding pools.

Pitfalls

Dedicated tenants drift if stacks are hand-built; use infrastructure as code from one template. Per-tenant exceptions multiply support cost. What would change my call: if the regulated customers are very few and revenue concentrated in them, I would invest in dedicated-by-default automation sooner.

Direct answer

I rely on a short list of principles and can point to a concrete place each one shows up. The examples below are from a composite cloud-native software-as-a-service (SaaS) platform with a data pipeline, of the kind I have designed and reviewed. Illustrative detail is labelled as such.

Fail-safe defaults versus secure by default. The first is about decisions and error paths: no rule means deny, and a broken check means deny. The second is about configuration: the setting you get without choosing anything is the safe one.

Data-engineering mappings. The same principles map onto a data platform as follows.

• Least privilege: column and row level access on the warehouse, so analysts see masked personal data by default.
• Defense in depth: encryption at rest plus access control plus audit logging on the lake.
• Segmentation: raw, curated and shared zones with separate roles, so a bad notebook cannot overwrite raw data.

Outside data work the same principles look like this: in a mobile app, least privilege means requesting only the device permissions a feature uses; in a back-office tool, separation of duties means the person who creates a payee cannot also approve the payment; in a web front end, attack surface reduction means removing unused routes and debug pages before release.

What separates a strong answer

Principles pull against each other. Fail-closed protects data but hurts availability, so choose per action based on the harm of a wrong yes versus a wrong no. Name one trade-off you actually made, not only the list.

Direct answer

Defense in depth means protecting an asset with several different controls in a row, so that when one fails (and eventually one will) the attacker still faces the next. Think of an airport: ID check, ticket check, security scan, boarding gate. A forged ticket gets past one step but not all of them. Layers are only worth having if they are independent: an attacker who beats layer one by some method should gain nothing toward beating layer two.

What makes layers independent rather than merely redundant

Redundant layers fail together; independent layers fail for different reasons. Check four things for any pair of layers:

• Different failure mode: a WAF (web application firewall, a filter that inspects web requests for attack patterns) is defeated by an encoding trick; parameterized database queries are not affected by that trick. For example, a WAF rule may block the text ' OR 1=1 -- (a classic SQL injection string). An attacker sends the same characters percent-encoded (%27%20OR%201%3D1--, or encoded twice) in a way the WAF does not decode but the application does, so the attack slips through. A parameterized query sends the SQL text and the user's value to the database separately (SELECT * FROM users WHERE name = ? with the value supplied on its own), so the database treats whatever arrives as plain data and never as SQL, however it was encoded.
• Different trust boundary and credentials (a trust boundary is the line where data or requests pass from a less-trusted zone into a more-trusted one, such as internet to web tier, or web tier to database): if one admin password or one management console controls both layers, compromising it removes both.
• Different technology or rule source: two firewalls from one vendor with one copied rule set share the same bug and the same misconfiguration.
• Different owner or change path: one bad deployment should not weaken both.

Quick test to apply in a design review: "Assume layer N is completely bypassed. What does the attacker still need to do?" If the answer is "nothing", the layers were redundant.

Worked example: SQL injection against an enterprise web app in a hybrid cloud (on-prem datacenter plus a public cloud)

In the hybrid case, keep the layers consistent across both sides: the same segmentation rules expressed as code in the datacenter and the cloud, one identity provider but separate break-glass (emergency) admin accounts per environment, and one log destination so the picture is not split.

The same idea in a streaming pipeline (Kafka brokers feeding Spark jobs)

Kafka is a system that carries streams of records between programs (brokers are its servers; producers write records and consumers read them), and Spark jobs process those records. The network zone limits who can reach the brokers; TLS plus authentication on every producer and consumer; per-topic ACLs (access control lists) so each job reads only its topics; schema validation at ingest so poisoned records are rejected; encryption at rest; audit logs of topic access.

Pitfalls

• Counting layers rather than testing independence.
• Layers nobody monitors: a layer that fails silently is no layer.
• Validating only from outside. Test each layer assuming the previous one failed (an assume-breach test, where the tester starts with an internal foothold).