Skip to main content

Postgres

Written by Aron Day

Almost every request touches Postgres. Spaces, workflows, users, run history, and live UI sync all depend on it. Alarm on the database itself, not only the application pods.

The chart’s in-cluster Postgres is the postgres StatefulSet with a 10 GiB PVC named pgdata-postgres-0. Production installs should prefer a managed PostgreSQL 18 instance with wal_level=logical, as described in the Helm installation guide. Use that provider’s CPU, memory, storage, and connection alarms when you do; the thresholds below still apply.

Alarms to set

Alarm

When to fire

What it means

CPU sustained

Above 80% for 10 minutes

The instance is too small, or queries are backing up.

CPU spike

Above 95% for 2 minutes

A burst. Repeat spikes mean the instance needs more CPU.

Memory

Less than about 10% free, or the pod holding near its limit, for 15 minutes

Postgres will start evicting cache, then get OOM-killed.

Connections

Sustained above about 80 connections (in-cluster default max_connections is 100)

Clients are leaking or the pool is undersized. On a managed instance, alarm at about 80% of that instance’s limit.

Disk

The Postgres volume above 60% full

WAL and data share this volume. A full disk stops writes and can take the whole product down.

For the in-cluster PVC, scrape kubelet volume stats on pgdata-postgres-0 the same way as Blobstore disk. If you run a Postgres exporter, also alarm on pg_stat_activity connection count and on remaining disk from pg_database_size.

Managed Postgres extras worth turning on: free storage, replication slot lag, and “oldest inactive logical replication slot.” 3B’s Zero sync uses logical replication. A stuck slot retains WAL until it fills the volume.

Confirming from the cluster

In-cluster, the database runs in postgres-0:

kubectl -n 3b exec postgres-0 -- psql -U postgres -c "SELECT count(*) FROM pg_stat_activity;"
kubectl -n 3b exec postgres-0 -- psql -U postgres -c "SELECT slot_name, active, pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) AS retained_bytes FROM pg_replication_slots;"
kubectl -n 3b exec postgres-0 -- psql -U postgres -c "SELECT pg_database_size('3b'), pg_database_size('zero_internal');"

Do not put the Postgres password on the command line. The chart’s postgres container is already authenticated for these as the postgres superuser.

What to do

  • CPU or memory pressure on in-cluster Postgres: move to a larger node, or cut over to a managed instance. The in-cluster database is a singleton and does not scale out.

  • Connections: find idle sessions in pg_stat_activity, then raise the instance size or the pool — do not raise max_connections as the first move.

  • Disk: expand pgdata-postgres-0 the same way as Blobstore disk, using StatefulSet postgres and postgres.storage.size. On the single-server install, grow the VM disk instead.

Did this answer your question?