PostgreSQL has no connections left | Labs | DevOps in Practice

orders-api intermittently reports remaining connection slots are reserved while PostgreSQL CPU stays low.

Measure connection pressure by application and state before increasing any limit.

Try: psql -c 'select application_name,state,count(*) from pg_stat_activity group by 1,2;'

Terminal command

PostgreSQL has connection headroom again.

The API pool is bounded across replicas and abandoned transactions are no longer accumulating.

Lesson

Treat database connections as a shared capacity budget. Size pools across all replicas, preserve operator headroom, and use transaction timeouts as a guardrail—not as a substitute for fixing transaction lifecycle bugs.

Edit configmap/orders-api-db

Bound the pool per replica and ensure abandoned transactions are closed.

Set DB_POOL_MAX to 15 and DB_IDLE_TRANSACTION_TIMEOUT to 60s.

Key concepts

Connections are a shared budget

Every replica competes for a finite database limit.

What it is

PostgreSQL max_connections is divided among applications, operators, maintenance, and reserved superuser slots.

In this incident

Three API replicas can each open 60 sessions against a database capped at 100.

Why it matters

Pool size must be budgeted across replicas, not tuned in isolation.

Idle in transaction is not harmless idle

An open transaction can retain locks and old row versions while doing no work.

What it is

idle in transaction means the client began a transaction and then stopped sending statements without commit or rollback.

In this incident

Twenty-eight orders-api sessions are abandoned inside transactions.

Why it matters

Timeouts bound damage, but application transaction handling still needs correction.

Preserve operational headroom

A database at 98% connection use has no room for recovery work.

What it is

Headroom is unused capacity reserved for bursts, failover, maintenance, and operator access.

In this incident

The bounded pool leaves more than half the connection budget free.

Why it matters

Raising max_connections can increase memory pressure without fixing an oversized client pool.