The database will be in private subnets.
I hear some version of this sentence in many architecture reviews.
It sounds responsible. It looks good in the diagram. The database sits inside a VPC, the subnet is labeled private, and there is no arrow coming from the Internet.
The diagram looks complete, so we could end the review here. But “it is private” should be only the beginning of the conversation.
How does a developer investigate a problem in non-production? How does CI/CD apply a schema migration or a hotfix? How do we create and revoke database users? Where does emergency access come from when the normal path is broken?
Private is a default, not the finished design
Applications need to read and write data. Pipelines need to deploy changes. Developers need feedback while building features (and even some flexibility for innovation). Operators need to diagnose failures. A useful system will always have access paths; the architecture decides whether those paths are intentional or improvised.
The objective is therefore not to make a resource impossible to reach. It is to make it reachable only by the right actor, in the right context, with the right permissions and for the right amount of time.
That is what I mean by: Private by default, accessible by design.
Private does not mean hidden. It means restricted—but the conversation often ends before anyone defines those restrictions.
We already use database credentials, cloud identities, Git keys, API keys, usernames, and passwords to restrict access. In this context, private means there is no anonymous or ambient access. It does not mean there is no access at all.
A private subnet only answers the routing question
A private subnet usually means that its resources do not have a direct route from the Internet. They may still have outbound Internet access through NAT, and they may be reachable from many other networks connected to the VPC.
That configuration only answers one question: Can traffic from point X get there?
It does not tell us who initiated the request, what they can do after connecting, how long their access should last, or whether we can explain their actions later.
A firewall can allow a connection to port 5432. It cannot decide whether the caller should be able to run DROP TABLE, read customer data, or create a database user. A VPN can put an employee on a connected network, but it does not make every employee an authorized database administrator.
The network is an important filter, but it is not an identity provider.
An access model should add four things to network reachability:
- Identity: every human and workload authenticates as itself.
- Authorization: that identity receives only the actions and data it needs.
- Context and time: environment, source, approval, and expiration narrow the access.
- Auditability: sensitive connections and actions leave evidence.
Calling a resource private without defining these controls is confusing network placement with security.
What this looks like for a database
For a production database, the baseline is usually straightforward: no public endpoint, private subnets across multiple availability zones, encryption in transit and at rest, and inbound rules that reference approved application or proxy security groups instead of broad CIDR ranges.
The rest of the design becomes clearer when we describe each actor explicitly:
| Actor | Access path | Identity and authority |
|---|---|---|
| Application | Connection from the service runtime to its database | Service identity; access only to its own schema and required operations |
| Developer | Brokered session to a non-production database | SSO identity mapped to a limited database role; session expires |
| Deployment job | Ephemeral job launched in the target network | Federated pipeline identity; permission to run migrations, nothing else |
| Operator | Approved, just-in-time session through an access proxy | Named admin role that expires; connection and queries are audited |
| Emergency operator | Separate break-glass path that is tested before an event | Temporary elevated role; immediate alert and mandatory post-use review |
The exact tools will vary. The important part is that every path carries an identity and ends with a purpose-specific database role. Network access does not automatically grant database authority.
This is especially important for developers. “Ask someone to temporarily open the security group” is not an access design. Non-production can provide a convenient, self-service path without making access anonymous or permanent. Production can require stronger approval without depending on a shared admin password.
All of these paths should exist before an incident.
CI/CD is an actor too
Architecture diagrams often show application-to-database traffic and forget the pipeline entirely.
If the database is reachable only from the application runtime, teams eventually invent another route: a long-lived credential in the CI system, a script executed from a laptop, or a security group rule nobody remembers to close.
CI/CD needs both parts of the access model: a controlled network path and its own authenticated identity. The network path might come from a runner inside the VPC, a connected runner network, or a deployment job launched inside the target environment. The identity might use OIDC federation and a short-lived cloud role instead of a static access key.
For database changes, I prefer versioned migrations executed as a dedicated deployment job. The job enters through an approved path, assumes a migration-specific identity, obtains temporary credentials, applies the expected change, emits logs, and terminates.
The application should not need owner permissions because migrations exist. The pipeline should not receive unrestricted access because deployment is automated.
Database user management can follow the same pattern: an administrative workflow assumes a narrowly scoped role and runs from a controlled environment. Engineers do not need permanent connectivity or a shared master password simply to create, rotate, or revoke a user.
S3 changes the mechanism, not the model
S3 does not live inside our private subnets in the same way a database does, but the same access model still applies.
Block Public Access establishes the default. Bucket policies and IAM define which identities can perform which operations. VPC endpoint conditions can constrain the network context. KMS controls who can decrypt the data. Access logs preserve evidence.
When a person or external system genuinely needs one object, a short-lived pre-signed URL may be the correct design. It does not make the bucket public; it grants a narrow capability for one operation and a limited period of time.
That is the distinction: public access is ambient; designed access is explicit.
Architecture reviews need paths, not labels
When someone says “the database is private,” the next version of the diagram should show more than subnet colors. For every application, person, pipeline, or external system, the review should be able to answer:
- Through which network path does this actor connect?
- How is the actor authenticated without a long-lived shared secret?
- What is the minimum authority it receives?
- Which context is required, and when does the access expire?
- How is the access granted, audited, and revoked?
- What is the controlled path when the normal one is unavailable?
- What happens if a credential is compromised?
If the design cannot answer these questions, the resource is not securely private. It is merely difficult to reach.
The real design begins after “private”
Difficult systems create unofficial paths. People share credentials, run changes from laptops, leave tunnels open, and request permissions broader than the task requires. Accessibility is not the opposite of security; a well-designed path is often what keeps people from bypassing it.
The network reduces the surface area. Identity tells us who is crossing the boundary. Authorization limits what they can do. Context and expiration contain the risk. Auditing makes the result explainable. None of these controls replaces the others, and together they turn a private resource into an operable system.
Private by default gives us a safe foundation. Accessible by design gives applications, developers, pipelines, and operators an intentional way through it. Together, they make the secure path the easiest path—even when production is on fire.
“The database is in private subnets” is a good opening statement. It means the design has started, not that it is finished.
Good architecture does not eliminate access. It makes access deliberate, limited, temporary, and explainable.