420 Certificates, 90-Day Lifetimes, and a PQC Mandate
After a certificate expired on a customer portal and caused 38 minutes of downtime, a financial services firm inventories 420 public TLS certificates across 14 business units, 60 BIG-IP devices, and three clouds. Only 180 certificates are issued through ACME; the rest are manually purchased from four CAs, with lifetimes dropping to 90 days next quarter. Private keys for 30 high-value certificates must reside in an HSM. The firm also runs an IoT fleet of 12,000 devices that use TLS 1.2 and cannot be updated before 2027. A post-quantum pilot enabled the X25519MLKEM768 hybrid group on a test virtual server; 8% of handshakes from older Android and IoT clients failed, while modern browsers negotiated successfully. The CISO wants 'PQC everywhere' before the next board meeting and asks whether to disable TLS 1.2 and all non-hybrid groups. The platform team has limited change windows and no centralized certificate inventory today.
1. Outline a 90-day certificate lifecycle remediation. What do you automate first, how do you handle HSM-bound keys, and what evidence proves the expiry risk is controlled?
Read the worked answer
Treat this as an inventory and ownership problem before it is an ACME problem. In the first 30 days, discover every certificate and endpoint: BIG-IP profiles, cloud load balancers, CDN, API gateways, and IoT registration authorities. Reconcile against the 420 known certificates and assign a business owner, environment, and renewal method to each. Add external synthetic handshakes and certificate-transparency monitoring so a missing internal inventory item still triggers an alert. Set alerts at 45, 30, 14, 7, and 3 days, not just at expiry.
Next, automate the high-blast-radius certificates first: customer portal, authentication, payment, and partner APIs. Use ACME where the CA and client support it, with DNS-01 or HTTP-01 as appropriate. For the 30 HSM-bound certificates, the key must remain non-exportable, so use an ACME client that can sign with a PKCS#11 or HSM key, or a CA integration that wraps the CSR; do not export the key to a file to make automation easier. Automate renewal at 30 days remaining, with a tested deploy and reload step plus rollback.
For the remaining manual certificates, create a renewal queue with named owners and a monthly service review. Do not use wildcards to reduce the count unless the risk is accepted; wildcards concentrate blast radius and complicate ownership.
Evidence that risk is controlled: a live dashboard reconciled to the CMDB, 100% ownership, automated renewal success rate, synthetic handshake success, a game day that replaces a certificate, and a documented manual exception list with expiry dates. The two-hour change window should be for cutover tooling, not routine renewals.
2. Evaluate the CISO's request to enable PQC everywhere, disable TLS 1.2, and remove non-hybrid groups. What rollout do you recommend and why?
Read the worked answer
PQC everywhere is the right direction but the wrong immediate instruction. The pilot shows 8% handshake failures on older Android and IoT clients, and 12,000 IoT devices cannot be updated before 2027. Disabling TLS 1.2 and all non-hybrid groups would disconnect those devices and likely some partners; it would also create an outage that undermines the crypto program.
Recommendation: use TLS 1.3 with the hybrid X25519MLKEM768 group preferred, followed by X25519 and P-256 as fallback, on profiles that serve modern clients. Keep TLS 1.2 enabled with classical groups on separate profiles for the IoT and legacy populations, and set a dated retirement plan tied to device replacement. Hybrid, not pure ML-KEM, is the safer transition because it retains a classical assumption if the new algorithm is later broken. Do not remove all non-hybrid groups until client telemetry shows negligible fallback.
Validate with ClientHello inspection and canary virtual servers. Measure handshake latency, CPU, and failure rate by client family. Some clients fail because of larger handshake messages or middleboxes, so test the full path, not just the BIG-IP. Prioritize harvest-now-decrypt-later exposure: external customer data, long-lived secrets, and partner links first; internal low-risk apps later.
Board-level answer: PQC is a program with milestones, not a switch. The measurable target is the percentage of externally exposed, high-value traffic negotiated with a hybrid group, plus a funded IoT replacement date. Keep the CISO's intent, but make the rollout risk-based and reversible.
3. How do you handle the 12,000 TLS 1.2 IoT devices and 30 HSM certificates in the same estate without weakening high-value services?
Read the worked answer
Segment by client population and risk, not by one global TLS profile. Create separate virtual servers and client SSL profiles for IoT, partner, public web, and high-value internal services. The IoT profile can keep TLS 1.2 with a restricted cipher list, require client certificates where devices support it, and terminate in a dedicated network zone. The public web profile prefers TLS 1.3 hybrid. The high-value profile uses the HSM-backed keys, strict cipher suites, and the strongest feasible groups. This prevents 12,000 devices from dictating the security level of customer-facing services.
For HSM certificates, use dedicated HSM partitions per environment, dual control for key generation, and automated renewal that never exports the private key. Test HSM failover and the renewal workflow. Keep HSM-backed services out of bulk wildcard automation.
For IoT, add compensating controls: private CA and certificate pinning where possible, device identity allowlists, anomaly detection on destination and volume, and network segmentation so a compromised device cannot reach other tenants. If devices cannot do mTLS, use network access control and per-device rate limits.
Migration path: inventory client capabilities from telemetry, publish a compatibility matrix, and set a date when TLS 1.2 is disabled only for populations that have migrated. Monitor handshake failures by profile. Rollback is profile-level, not estate-wide. This is slower than a global flag, but it is the only way to satisfy the CISO without breaking the IoT fleet.