df and du disagree by 158 GiB

A df/du gap means blocks are allocated to the filesystem but no pathname accounts for them:

1. Unlinked-but-open files. lsof +L1 /var/lib/pgsql (or lsof | grep -i deleted). The process still holds the inode, so ext4 keeps the blocks; no directory entry exists, so du cannot see it. Survives here - nothing in the setup rules it out, and a crashed archiver or backup tar is the classic holder.
2. Permission-limited du. du skips directories it cannot read and prints 'Permission denied' to stderr. Discriminator: run as root, or du -xsh and actually read stderr. Does not survive: the root run returned 198G against 199G used, so the non-root 41G was simply an undercount. That number must be thrown away before drawing any conclusion.
3. Files hidden under a mount point. If a filesystem is mounted over a directory that already held data, that data still occupies the parent filesystem but is invisible to du walking the mounted tree. Discriminator: du -x (stays on one filesystem) versus du without -x, or mount | grep pgsql. Does not survive here: nothing has been mounted in months.

Also worth eliminating quickly: reserved blocks (ext4's default 5% is ~10 GiB - it explains 'root can still write' but cannot produce a 158 GiB gap, and the setup already gives you the real explanation) and sparse files, which make du smaller than df but are not in play once the root du agrees with df to within 1 GiB.

So the surviving answer is (1): ~158 GiB of deleted files held open. The 1 GiB df figure is genuinely free.

Identify first, act second:

tr '' ' ' < /proc/PID/cmdline; echo
ls -l /proc/PID/fd | grep deleted
lsof +L1 /var/lib/pgsql

/proc/PID/fd maps file descriptor numbers to the deleted paths; the cmdline tells you what the process actually is (a stale archiver, a tar, a pg_basebackup wrapper).

Reclaim, in order of preference:

1. Let it finish or terminate it cleanly. kill -TERM PID, wait, re-check df. The kernel frees the inodes when the last descriptor closes. This is the only option with no corruption risk.
2. Truncate through /proc, only for a file the process does not read back. : > /proc/PID/fd/N (as root, or the owning user) releases the blocks without killing the process. Safe for a log file that is only appended to; unsafe for anything the process might later read, seek in, or has mmaped - truncating a database or index file that way yields silent corruption or SIGBUS.

Never unlink further, rm the spool, or reboot the host casually as a 'fix': a reboot does free the space but destroys the evidence of which process leaked, and the same job will do it again next week. After reclaiming, confirm with df -h and add the PID's parent job to monitoring.

vgs; lvs                    # confirm 150 GiB free extents in vg_pg and the LV's current size
lvextend -L +100G /dev/vg_pg/lv_pg
resize2fs /dev/vg_pg/lv_pg  # ext4, mounted, grows online
df -h /var/lib/pgsql

Or, on a current RHEL 8, the one-step form lvextend -r -L +100G /dev/vg_pg/lv_pg, which calls fsadm and runs the right resize for ext4 or XFS. Using -r splits the choice of resize tool from the operator, which is usually what you want.

Traps:

• -L +100G versus -L 300G. One is relative, one absolute. Typing the absolute form when you meant relative shrinks or wildly over-allocates; a typo is refused only if it exceeds free extents, so it can still succeed and destroy data if the filesystem is grown later.
• Forgetting the second step. lvextend grows the block device; ext4 inside it stays 200 GiB until resize2fs runs. df will not budge and the next person re-runs lvextend.
• Wrong filesystem tool. resize2fs is ext2/3/4 only; XFS needs xfs_growfs on a mounted filesystem and cannot shrink at all. Check with lsblk -f before typing.
• Order. resize2fs before lvextend has nothing to grow into.

Finally, this host was at 100% because growth monitoring watched only the database, not the deleted-file behaviour of the archiver; add both an lsof +L1 check and an LV free-space alert.