Key Takeaways 

• Signal 1 (SIGHUP) normally tells PostgreSQL to reload its configuration. A log line saying a process was terminated by signal 1 means that process didn’t handle the signal and died from it.
• received SIGHUP, reloading configuration files is a harmless reload. server process (PID …) was terminated by signal 1: Hangup is a death report.
• When any child of the postmaster is killed by a signal, PostgreSQL drops every connection, resets shared memory, and replays WAL. The postmaster keeps its process ID, so the server never appears to restart.
• Running kill -9 on a single backend triggers the same server-wide reset. Use pg_cancel_backend() first, and pg_terminate_backend() only if needed.

Imagine you’re reviewing your PostgreSQL logs and you find this line:

LOG:  server process (PID 48213) was terminated by signal 1: Hangup

Right after it, PostgreSQL went through crash recovery. Every connection dropped, the application threw errors for about half a minute, and then everything came back as if nothing had happened.

So you start investigating. The server never restarted, and the main PostgreSQL process still has the same process ID it had yesterday. Nobody ran a deployment, and nobody on the team admits to touching anything. You search the log for a configuration reload and find nothing. A few days later, the same line appears again at a completely different time.

To make it stranger, signal 1 is the signal PostgreSQL uses to reload its configuration. Normally it causes no harm at all: PostgreSQL re-reads its settings and carries on. So why is it showing up in a crash log?

To answer that, we need to understand what signals are and how PostgreSQL reacts to them. Let’s start with the basics.

What is a Signal?

A signal is a small notification sent to a running program, either by the operating system or by another program. It carries no data. It simply tells the program that something happened, such as “stop”, “cancel what you’re doing,” or “your terminal has disconnected”.

Every signal has both a name and a number, and the two are interchangeable. Commands usually use the name, while PostgreSQL’s log shows the number plus a short description. These are the signals a PostgreSQL DBA is most likely to encounter:

So signal 1: Hangup in the log simply means SIGHUP. If you’ve ever pressed Ctrl+C in psql or cancelled a slow query with pg_cancel_backend(), you’ve already sent a signal: behind the scenes, PostgreSQL delivers SIGINT to the session running your query.

Same Signal, Different Reactions

A signal doesn’t do anything by itself. What happens next depends entirely on the program that receives it. A program can:

• run its own code for that signal, called a signal handler (this is what PostgreSQL does), 
• ignore the signal, or
• do nothing special, in which case Linux applies the default action. For signal 1, the default action is to end the program.

You’ve probably seen that default action yourself. You SSH into a server, start a long pg_dump, and then your Wi-Fi drops. When you log back in, pg_dump is gone. When your session closed, the system sent signal 1 to the programs running in it. pg_dump has no handler for signal 1, so Linux applied the default action and ended it.

This is why many DBAs start long jobs like this:

nohup pg_dump mydb > mydb.sql &

nohup stands for “no hang-up”. It tells the job to ignore signal 1, so the job keeps running even after your session closes.

PostgreSQL works differently. Its source code includes a signal handler for signal 1. When signal 1 arrives, that code runs, and instead of exiting, the process re-reads its configuration and carries on. In other words, PostgreSQL’s processes are trained to handle signal 1. Most other programs aren’t, and signal 1 ends them.

You see PostgreSQL’s training at work every time you reload it. Running SELECT pg_reload_conf(); sends signal 1 to the main PostgreSQL process, which passes it on to the other PostgreSQL processes. They pick up the new settings, and none of them die.

Meet the Postmaster

To understand why one signal can affect a whole server, we need a quick look at how PostgreSQL runs. It doesn’t run as a single process. It runs as a family of processes:

postgres -D /var/lib/postgresql/data                  ← the postmaster (the parent)

 ├─ postgres: checkpointer

 ├─ postgres: walwriter

 ├─ postgres: autovacuum launcher

 └─ postgres: app_user appdb 10.0.0.15(51432) idle    ← a backend (one per connection)

• The postmaster is the parent. It starts first, accepts connections, starts all the other processes, and keeps an eye on them. The program is called postgres these days, but the parent process is still known by its historical name, the postmaster.
• Backends serve the clients. Every connection gets its own.
• Background processes handle housekeeping, such as checkpoints and autovacuum.

All of these processes share one area of memory, called shared memory, where they keep cached data, locks, and other shared state. Shared memory is one reason PostgreSQL is fast. It’s also the reason one failing process can affect every connection.

If you’d like a deeper tour of these processes, our earlier post PostgreSQL Internals Part 3: Understanding Processes in PostgreSQL covers each one in detail.

Two Log Lines That Look Alike But Aren’t

Back to our log. PostgreSQL writes two very different lines that both involve SIGHUP: 

The first line is a receipt. The second is a death report.

Don’t let the label fool you: “server process” only means this process was a child of the postmaster. It doesn’t mean the process was part of PostgreSQL.

On newer PostgreSQL versions, the death report may use a more specific label, such as client backend, instead of server process.

In our story there’s no receipt, only the death report. That tells us something important: nobody reloaded PostgreSQL. Signal 1 reached some other process, and that process died from it.

Why One Death Resets Everything

When a process dies, Linux cleans up after it. It frees the process’s own memory, closes its files, and reports the death to its parent: “your child died, and here’s why.”

What Linux can’t clean up is PostgreSQL’s shared memory, because Linux has no idea what’s inside. A process that died suddenly might have been holding a lock, or been halfway through writing a data page. The postmaster has no way to check, so it has to assume the worst.

PostgreSQL would rather be safe than sorry with your data. So when the postmaster learns that a child was killed by a signal (or exited in some other unexpected way), it:

1. tells every other process to stop,
2. resets shared memory,
3. replays the write-ahead log (WAL) to bring the database back to a consistent state.

Here’s how that looks in the log (trimmed slightly):

LOG:  server process (PID 48213) was terminated by signal 1: Hangup

LOG:  terminating any other active server processes

WARNING:  terminating connection because of crash of another server process

LOG:  all server processes terminated; reinitializing

LOG:  database system was not properly shut down; automatic recovery in progress

LOG:  redo starts at …

LOG:  database system is ready to accept connections

The WARNING line is what every other session receives, and it’s usually the first thing an application team reports. The postmaster itself never stops, so its process ID stays the same, and the server never appears to restart. The database restarted without restarting. (This automatic recovery is controlled by the restart_after_crash setting, which is on by default.)

There’s one more detail here, and it’s the key to our mystery. The postmaster reacts to how a child died, not to what that child was. If a process that isn’t of PostgreSQL and ever ends up as a child of the postmaster, the same rule applies to it:

• That process has no handler for signal 1, so signal 1 kills it.
• The postmaster sees one of its children killed by a signal.
• It assumes shared memory may be damaged, and it resets everything, even though the process that died never touched shared memory.

The Everyday Version: Kill -9

You don’t need a mystery to trigger this reset. The most common cause is a well-meaning kill -9.

A query runs too long, someone finds its process ID in top, and runs kill -9 on it. The query stops, along with every other connection on the server, and the log shows terminated by signal 9: Killed, followed by the same crash recovery. In other words, kill -9 on one backend affects the whole database, not just the query you aimed at. The Linux out-of-memory killer also uses signal 9, which is why running out of memory can trigger exactly the same reset.

The safe way is to let PostgreSQL do the stopping. First, find the process ID:

SELECT pid, usename, state, now() – query_start AS running_for, query

FROM pg_stat_activity

WHERE state = ‘active’

ORDER BY running_for DESC;

Then cancel the query, and end the session only if you need to:

— Step 1: cancel the query, keep the connection

SELECT pg_cancel_backend(48213);

— Step 2: if that’s not enough, end the session

SELECT pg_terminate_backend(48213);

Both functions go through PostgreSQL’s own signal handling, so the process cleans up after itself and only that one session is affected. To use them, you must be a superuser, be connected as the same role as the target session, or be a member of the pg_signal_backend role.

Reaching for kill -9 is usually a decision made under pressure. If you want a calmer plan for those first minutes, join our webinar on October 28: The First 30 Minutes of a PostgreSQL Incident: a Production Triage Playbook.