PCI findings on an EPAS cluster under PEM management

Split the work into privilege, then credential format, then enforcement - each step is independently safe.

1. Reduce privilege first, because it is the actual risk. Create service roles per application tier with only the required object privileges, grant them to grouped roles, and remove SUPERUSER from app_super after building the mapping. Existing sessions keep their privileges until they reconnect, so coordinate: change the role, then let the pooler recycle connections in a controlled wave, and have the ability to re-grant fast if a missing privilege surfaces.
2. Then migrate password storage: ALTER SYSTEM SET password_encryption = 'scram-sha-256'; SELECT pg_reload_conf(); and reset passwords. This is the subtle part - when a role's password is set while password_encryption is scram, the server stores a SCRAM verifier but retains the old MD5 hash, so existing md5 pg_hba lines keep working while you roll through 40 roles. Verify with SELECT rolname, left(rolpassword, 14) FROM pg_authid WHERE rolpassword IS NOT NULL; - you want SCRAM-SHA-256$ for endpoints that matter.
3. Only when every role that needs to authenticate has a SCRAM verifier, change pg_hba.conf to scram-sha-256 and SELECT pg_reload_conf();. That reload needs no restart and applies to new connections; pooled connections survive until recycled, so a bad line shows up as new connections failing while old ones work. Validate before reloading with SELECT * FROM pg_hba_file_rules WHERE error IS NOT NULL;.
4. Finish with auditing (edb_audit DDL and connection logging, or pgaudit) so the next review has evidence, and alert on failed authentications.

Never change the auth method and the role model in the same window: if logins break you will not know which change caused it.

ssl = on is a postmaster-level parameter, so it cannot be applied by reload - that is the constraint driving the answer, and pretending otherwise is how audits fail.

Options:

• Terminate TLS at the pooler. Configure the pooler with a certificate and require TLS from clients, so every 38 application node to database path is encrypted on the wire between the node and the pooler, and the pooler-to-database hop is inside a segmented network. This closes most of the finding immediately with no database restart. It does not close all of it: pg_stat_ssl will still show the pooler sessions as unencrypted, and any client bypassing the pooler is still in the clear.
• Restrict the residual exposure while you wait: set hostssl lines for the office and admin subnets and remove host all all 10.20.0.0/16 md5 so only TLS-capable paths remain for privileged users. Reload applies it.
• Schedule the change window for ssl = on plus ssl_min_protocol_version = TLSv1.2, certificate deployment on all nodes, and hostssl-only rules for the application subnet. Budget for the restart plus the replication reconnect, and rehearse the certificate chain on a standby first - an expired or mispermissioned key produces a database that starts and then refuses every connection.

Recommendation: pooler TLS plus tight pg_hba now, documented as an interim control with a dated change window for the postmaster restart. State the residual risk explicitly rather than claiming the finding is closed; the auditor's real question is whether you know what remains unencrypted.

The management plane holds credentials to 140 monitored instances, so it is the highest-leverage compromise path in the estate.

• Console transport: put HTTPS in front of the PEM console (or terminate TLS at a reverse proxy you control), restrict it to the admin VLAN, and stop exposing it to the office subnet. An HTTP console means session cookies and any credentials typed into it cross the office network in clear text.
• Agent credentials: a shared database credential for agent registrations means one leaked value compromises every monitored instance, and it also destroys attribution - you cannot tell which agent did what. Issue per-agent or per-group monitoring roles with the minimum privileges PEM needs (statistics and monitoring functions, not superuser), and rotate the shared value out.
• Privilege review on the PEM server itself: its repository database holds monitoring history and configuration for the whole estate, so treat it as a production database - patched, backed up, restricted, and not sharing credentials with anything else.
• Auditing and alerting: enable PEM's own audit trail for console logins and configuration changes, alert on new agent registrations, and include the PEM server in the same vulnerability and patch process as the database estate.

Sequencing: fix exposure (VLAN restriction, HTTPS) in hours, then credential separation, then auditing. The reason it ranks alongside the database findings is that the SUPERUSER role and the shared agent credential reach the same 140 instances, but the shared credential is one secret guarding all of them.