Six weeks to the QSA: patching, root SSH, auditd, SELinux, and a vendor who will not move
Forty RHEL 8 servers run a vendor-supplied Java application in cardholder-data scope. A CIS Level 1 scan puts the fleet at 64 percent compliance. Open findings include: PermitRootLogin yes and PasswordAuthentication yes on every host; TLS 1.0 and 1.1 still enabled on the Tomcat connector; a system-wide umask of 022; auditd installed but disabled because a 2021 vendor benchmark showed a 12 percent throughput loss; SELinux in Permissive on all 40 hosts because the vendor will not support Enforcing; and the application binds ports 80 and 443 and, per the install guide, must run as root. Nine hosts are missing patches for a remotely exploitable OpenSSL CVE published last week. The QSA assessment begins in six weeks. Change control requires ten business days' notice and application-owner approval, and the application owner has refused any downtime before the assessment. The vendor supports only the documented configuration.
1. Sequence the six weeks. Justify the order, and explain how you address the application owner's refusal of downtime for the CVE patching.
Read the worked answer
Order by exploitability first, then by lead time, then by effort.
Week 1: the OpenSSL CVE on the nine hosts. Remotely exploitable, in cardholder scope, already public - this is the finding a QSA will treat as an active risk. Use the emergency change path.
Week 1-2: SSH hardening. PermitRootLogin no with a documented break-glass account, PasswordAuthentication no, AllowGroups, deployed by configuration management to one canary host first. Validate by opening a second session before closing the first; a lockout here is self-inflicted and unrecoverable remotely.
Week 2-3: auditd back on with tuned rules, and measure the actual performance delta on a canary rather than trusting a 2021 number taken on different hardware. PCI requires audit logging; 'disabled for performance' is not a defensible answer without a compensating control.
Week 3-4: TLS 1.0 and 1.1 off, after inventorying which clients negotiate them from Tomcat logs. Run a warn-only phase first.
Week 4-5: umask, file permissions, remaining CIS items.
Week 5-6: evidence pack and dry-run walkthrough with the QSA; leave buffer.
The downtime objection. The owner is refusing an outcome that the fix does not require. Patching a load-balanced fleet can be done as a rolling restart: drain one host, patch, restart, verify, return it to the pool, move to the next. Users see no outage. Bring the owner a schedule that names each host, the expected 5-10 minute drain per host, and the rollback snapshot, and ask them to approve that rather than approve 'downtime'. If there is no load balancer, the conversation is about a maintenance window and the risk acceptance, not about whether to patch.
Never bundle the CVE patch with an SSH or SELinux change in the same window; if something breaks you need to know which change did it.
2. The vendor will not support SELinux Enforcing or auditd. How do you respond without simply leaving the hosts in Permissive?
Read the worked answer
Separate the two findings, because they are not equivalent.
auditd is not negotiable. It is the mechanism that produces the evidence the assessment depends on. Re-enable it with a tuned configuration - a larger backlog buffer, rate limiting, and rules scoped to what matters (identity files, sudoers, the application unit, authentication events) rather than every syscall - and measure the throughput impact on a canary. If the real delta is 2 percent, the vendor objection is closed with data. If it is 12 percent, that is a capacity decision for the application owner, documented with numbers, not a reason to run blind.
SELinux can be partially enforced. Host-wide Permissive protects nothing. Instead, keep the host Enforcing and mark only the vendor application's domain permissive: semanage permissive -a <vendor_domain>. Everything else on the host stays confined while the application runs under a documented exception. Collect AVCs during normal operation, review them, and build a policy with audit2allow only after understanding each denial. This is a defensible partial control and materially better than the current state.
If the vendor still refuses, move to formal risk acceptance: the application owner, not the security team, signs a dated acceptance naming the specific risk and an expiry or review date, supported by compensating controls - tighter network segmentation around the cardholder zone, WAF in front of the application, file integrity monitoring on the application binaries and config, and enhanced logging shipped off-host. Record it on the compensating-control worksheet so the QSA reviews a decision rather than discovering an exception.
The failure mode to avoid is quietly leaving 40 hosts in Permissive and hoping the scan does not ask.
3. The application must bind 80 and 443 and the vendor says it must run as root. What do you actually implement, and how does that interact with the TLS finding?
Read the worked answer
Running as root is a consequence of binding privileged ports, not a real application requirement. Several options remove it:
- systemd ambient capability. Run the service as a dedicated
appsvcuser withAmbientCapabilities=CAP_NET_BIND_SERVICE,NoNewPrivileges=yes, and a hardened unit. Declarative, survives upgrades, and scoped to the one service. setcap cap_net_bind_service=+epon the java binary. Works, but applies to every invocation of that binary and must be reapplied after each JDK update - a silent failure waiting to happen. Prefer option 1.- Terminate in front. Put HAProxy or nginx on 443 and let the application listen on an unprivileged 8080 behind it.
Recommend option 3 here, because it solves two findings in one change: the proxy can enforce TLS 1.2 and above with a controlled cipher list, retiring the Tomcat TLS 1.0/1.1 connector at the same time, and the application drops to an unprivileged user behind it. If the proxy is not acceptable, option 1 is the fallback.
Audit trail. Once the application runs as appsvc, nobody needs root for routine operations: use named accounts with sudo for service management, enable log_output or iolog on those sudo rules, keep systemd journal shipped to the central log platform, and add auditd watches on /etc/passwd, /etc/sudoers and /etc/sudoers.d, the unit file, and the application binary. Verify with ausearch -k app_unit that a restart is recorded.
Validation before the assessment. Confirm the application does not write to its install directory, move logs and state to /var/lib/appsvc with correct ownership, and test the full path - external client, proxy, application, database - on one host before rolling out.