StormaticsStormatics

PostgreSQL Wait Events: A Production Diagnostic Guide

A query plan tells you what PostgreSQL intended to do. A wait event tells you what it actually did with its time. Most performance work I see in production starts one layer too high, in the EXPLAIN output, when the cheaper answer is already sitting in pg_stat_activity.This is a refresh of a guide I wrote in late 2024. Two things have changed since then. PostgreSQL 17 added pg_wait_events as a first-class catalog view. PostgreSQL 18 shipped asynchronous I/O and changed how I/O wait should be read in production. Both shifts deserve their own treatment, and I have folded them into this update along with five production cases from work my team did over the last two quarters.
Read More

Long-running transactions, job queues, and the cascade that wreaks havoc

A scheduled PostgreSQL migration step held an open transaction snapshot for hours during the initial data copy. A job queue running at production write velocity began to slow down. Twelve hours later, the queue was seven million rows deep, the primary was pinned at 100 percent CPU across 24 cores, and customer support was fielding complaints about delayed transactions.
Read More

You have a Patroni leader election. You are only halfway to PostgreSQL high availability.

A PostgreSQL primary loses power at 2am. Writes resume in under thirty seconds. The on-call engineer reads the alert in the morning, sees that the cluster healed itself, and goes back to coffee. That is the outcome PostgreSQL high availability is supposed to deliver.A working Patroni cluster, on its own, gets you partway there. The leader election runs. A standby gets promoted. The cluster state in etcd stays consistent. Then the application keeps trying to reach an IP address that points at the wrong node now, the old primary needs a manual rejoin, and the on-call engineer is on a conference bridge instead of in bed.
Read More

The best PostgreSQL databases are boring on purpose

The calmest PostgreSQL deployments in production share one trait. They are boring. Pages stay quiet. Dashboards stay green. The on-call engineer reads a book on Tuesday night. And the people running those databases will tell you, plainly, that boring is the achievement. Think about flying for a minute. The flight everyone wants is the one where the captain says hello, the meal shows up on time, and a few hours later, the wheels touch down in the right city. That flight is boring. It is also a small miracle. Behind that boring flight sits decades of compounded discipline. Pilots with thousands of simulator hours.
Read More

PostgreSQL High Availability on OCI: Why Your Failover Passes Every Test But Breaks in Production

If you have built PostgreSQL high availability clusters on AWS or Azure, you have probably gotten comfortable with how virtual IPs work. You assign a VIP, your failover tool moves it, and your application reconnects to the new primary. Clean. Simple. Done.Then you try the same thing on Oracle Cloud Infrastructure and something quietly goes wrong.The cluster promotes. Patroni (or repmgr, or whatever you are using) does its job. The standby becomes the new primary. But the VIP does not follow. Your application keeps sending traffic to the old node — the one that just failed. From the outside, it looks like the database is down. From the inside, everything looks green.
Read More

Thinking of PostgreSQL High Availability as Layers

High availability for PostgreSQL is often treated as a single, big, dramatic decision: “Are we doing HA or not?”That framing pushes teams into two extremes:- a “hero architecture” that costs a lot and still feels tense to operate, or - a minimalistic architecture that everyone hopes will just keep running.A calmer way to design this is to treat HA and DR as layers. You start with a baseline, then add specific capabilities only when your RPO/RTO and budget justify them.Let us walk through the layers from “single primary” to “multi-site DR posture”.Start with outcomesBefore topology, align on three things:1. Failure scope a. A database host fails b. A zone or data center goes away c. A full region outage happens d. Human error2. RPO (Recovery Point Objective) a. We can tolerate up to 15 minutes of data loss b. We want close to zero3. RTO (Recovery Time Objective) a. We can be back in 30 minutes b. We want service back in under 2 minutesHere is my stance (and it saves money!): You get strong availability outcomes by layering in the right order.
Read More

PostgreSQL Materialized Views: When Caching Your Query Results Makes Sense (And When It Doesn’t)

Your dashboard queries are timing out at 30 seconds. Your BI tool is showing spinners. Your users are refreshing the page, wondering if something's broken. You've indexed everything. You've tuned shared_buffers. You've rewritten the query three times. The problem isn't bad SQL - it's that you're forcing PostgreSQL to aggregate, join, and scan millions of rows every single time someone opens that report.
Read More

Archives

Categories