Certificate swap breaks StoreFront pass-through, aggregation and external HTML5
Contoso Health runs two independent Citrix Virtual Apps and Desktops Sites, each fronted by its own three-server StoreFront cluster. Internal users reach https://citrix.contoso.com, which DNS resolves to the nearest cluster; external users arrive through a NetScaler Gateway pair. In June the infrastructure team replaced the internal-CA certificate on both StoreFront clusters with a public-CA wildcard for *.contoso.com and removed the old internal root from the clusters' trusted store. Since then three things happen. Windows Workspace app users on the clinical VLAN are prompted for credentials three times, and roughly one in twenty receives "Cannot complete your request". A change that added the second Site's Delivery Controllers to the primary store, to give users a single icon set, produced 60 duplicate application icons and pushed enumeration from 4 seconds to 26 seconds. And the reception thin clients, which launch published desktops through the browser, work internally but fail externally. Nobody has changed the Gateway configuration.
1. Diagnose the repeated credential prompts and the intermittent "Cannot complete your request" on the clinical VLAN.
Read the worked answer
The June change bundled two independent risks - certificate identity and root trust - and the symptom pattern points at both.
Repeated prompts are a client-side trust and zone problem. The Workspace app only performs pass-through authentication when the store URL is in a trusted intranet zone and the TLS certificate validates silently for the name the client actually uses. Check, in order:
- The URL configured in the Workspace app - if clients use a short name or an internal-only FQDN, a *.contoso.com wildcard will not cover it.
- The full certificate chain served by IIS: a missing intermediate breaks validation and forces prompts even on a public certificate.
- That the new binding has a private key and SNI is configured correctly for the store's HTTPS binding.
- That CRL/OCSP endpoints are reachable from the clinical VLAN; a blocked revocation check produces exactly this intermittent, per-client behaviour.
"Cannot complete your request" is a StoreFront-to-dependency failure, not a client failure. Removing the internal root from the clusters' trusted store will break any TLS call StoreFront makes to a component still using an internal-CA certificate - Delivery Controller XML traffic if it was moved to HTTPS, or the Gateway callback URL. Inspect the store's Manage Delivery Controllers transport setting and the Manage Citrix Gateways callback URL, and confirm each endpoint's certificate chains to a root the StoreFront servers still trust.
Method: reproduce from one clinical VLAN client while watching StoreFront logs and a packet capture, then test with the exact FQDN the production client uses. One in twenty failing usually means a subset of servers with an incomplete binding - compare all six StoreFront servers, not just one.
2. Evaluate the change that added the second Site's controllers to the primary store. What is causing the duplicates and the enumeration delay, and what should be done?
Read the worked answer
The delay is expected physics. A store bound to controllers in both Sites issues broker requests to both on every enumeration, and the remote Site adds WAN round trips for authentication, resource lists and per-icon retrieval. A rise from 4 s to 26 s is consistent with cross-site enumeration; it will also add jitter whenever the WAN degrades, and it makes every user's icon list dependent on the far data center - the exact dependency the two-Site design was built to avoid.
**The duplicates are an aggregation-**naming problem. StoreFront merges resources that share a name and type within a store; if the two Sites publish the same applications under different display names (or one as a different application type or with a different command line), they are treated as separate resources and both icons appear. Before changing anything, compare the published names, client types and command lines between Sites - 60 duplicates out of a shared set is usually a naming-convention drift, not a StoreFront bug.
Judgement: revert. A single global icon set is only worth the WAN dependency if users genuinely need apps from both Sites. If locality matters (and the whole architecture says it does), keep one store per Site, each enumerating only its local controllers, and expose them through GSLB or two documented URLs. If a truly global icon set is a hard requirement, publish the shared applications from one Site and use the other as DR - do not aggregate two Sites across a 130 ms link.
Measure the fix: enumeration from a Dallas client against the Dallas store, target under 8 s, plus a check that duplicate names, if any, are deliberate.
3. Reception thin clients work internally but fail externally after the change. Where do you look, and what is the most likely cause?
Read the worked answer
Internal success narrows the fault to the Gateway hop. The browser-based client authenticates at the Gateway, and then two things must work: the Gateway must reach the StoreFront callback URL, and the session's ICA traffic must be proxied over the Gateway to an STA-registered VDA.
Most likely causes, in order of probability after a certificate change:
- Callback URL trust or resolution. The URL configured under Manage Citrix Gateways (for example https://citrix.contoso.com/Citrix/StoreAuth) must be resolvable from the DMZ and must present a certificate the Gateway trusts. If the callback now uses a name outside the wildcard's SAN, or the internal root was removed from the Gateway, external authentication fails at exactly this point - internally the browser talks to StoreFront directly and never exercises this path.
- STA trust. The Gateway opens a TLS connection to the STA servers on 443; if those still use internal-CA certificates and the root is gone, the Gateway cannot validate the ticket issuer.
- WebSocket handling. The HTML5 client needs WebSocket traffic through the Gateway. A proxy or profile change can silently block it, which presents as a connect failure after successful authentication.
Method: from outside, request the callback URL directly and inspect the chain; read the Gateway's authentication and AAA logs for the failing transaction; confirm STA reachability by opening a TCP/TLS session from the Gateway to each STA. Then test with a browser that is not on the corporate proxy.
Remedy: bind a certificate whose SAN covers every name in use, serve full chains on both ends, restore the internal root only on the servers that need it for internal-CA dependencies, and document that callback and STA certificates are part of the StoreFront certificate lifecycle.