Incomplete Chain, Two Brands, One IP and a Legacy Client Fleet

A leaf-only certificate file is enough for many desktop browsers because they fetch the missing intermediate from the URL in the certificate's AIA field. iOS and many Android clients do not; they must be sent a chain they can build, and when they cannot they reject the certificate. That is precisely the desktop-fine, mobile-broken split.

Remediation:

1. Obtain the intermediate certificate(s) from the issuing CA rather than from a browser cache.
2. Rebuild the certificate object so the file contains the leaf first, then the intermediates in order - they are transmitted in file order. The private key remains a separate object and must match the leaf.
3. Include the root only if a specific client population demands it; normally it is unnecessary and inflates every handshake.
4. Reload and verify from outside the network: openssl s_client -connect 203.0.113.10:443 -servername portal.example.com -showcerts should show the full chain in the right order.
5. Then confirm on an actual iOS device. Desktop verification is what concealed the original fault, so it cannot be the acceptance test.
6. Check expiry on every certificate in the chain, not just the leaf - an expired intermediate produces an identical client complaint and is easy to miss.

Make chain rebuilding a mandatory step in the renewal runbook, or the next renewal reintroduces the fault silently.

Three options, in order of preference:

1. SNI-based certificate selection. The client names the host in the handshake and the ADC chooses the matching certificate and template. Verify that the running ACOS release supports the behaviour you need, then test with openssl s_client -servername shop.example.net. The catch: clients that do not send SNI - the Android 5 fleet, some embedded devices, older libraries - receive the default certificate, so whichever brand is default will show a name mismatch to those users. That must be an explicit decision, not a side effect.

2. One certificate with both names (SAN/UCC). Operation is simple and it works for non-SNI clients, but brand and legal teams often refuse a shared certificate, and every future addition means reissuing.

3. Interim port split. Serve one brand on a different port with a redirect on 443 until the second IP arrives. Ugly for users, but it removes the non-SNI ambiguity and is honest about the constraint.

How the legacy fleet decides it: if those 8% matter and cannot be upgraded within six weeks, pure SNI leaves them with a mismatch on one brand, so option 2 or 3 carries the day. If the fleet is small and owned, SNI plus a documented list of affected devices is the cleanest path.

The audit requirement and the legacy fleet are in direct conflict; a single client-ssl template can enforce one or the other, not both. The enforcement point is the template, and templates bind per virtual port - which is what makes a clean split possible.

Preferred: a second listener, for example 203.0.113.10:8443, with a legacy template permitting TLS 1.0/1.1 and the ciphers those tablets need, restricted by source range and documented as an exception with an expiry. The main :443 moves to TLS 1.2 and above with a modern cipher set, and the audit is satisfied for the production service.

Not preferred: weakening the main template. It fails the audit and widens exposure for every user in order to serve 8%.

Before the deadline, quantify the 8% by client and by owner. Android 5 tablets are usually a fixed, enumerable fleet with a refresh date; if it is forty devices, a hard cutover plus a communications plan beats a second listener that nobody will ever retire. Whatever you choose, evidence it with a real device and with openssl s_client -tls1_1 showing the intended refusal.

The backend log observation is unrelated to TLS: the servers see the ADC as the source. Add an X-Forwarded-For insert in the HTTP profile and log that header on the backends; do not weaken security to recover client addresses, and do not expect the ADC to preserve source IPs for a proxied flow.