Air-Gapped Upgrade and License Renewal in a 10-Hour Window

Pre-flight (before the window, on the portable drive): confirm every component is at 10.9.1 and healthy (portal health check, server site health, data store status), confirm free disk on all six machines, and run a full webgisdr export as a version-consistent backup in addition to VM snapshots. Verify the 11.3 media and the 11.3 license files are on the drive and that checksums match.

Order - portal, then server, then data store. Components must end at the same version, and the later tiers depend on the earlier ones. Portal first: upgrade both portal nodes, then run the portal configuration/upgrade so the content directory and config store are migrated. ArcGIS Server second: upgrade both machines and confirm the site is healthy before touching the data stores. ArcGIS Data Store last: upgrade the primary, then the standby, and let the data store upgrade complete before re-enabling publishing. Doing the data store first would leave it expecting a server it cannot serve.

Licensing. Licensing is a separate step from installing media:

• Portal: obtain the 11.3 .json license file through My Esri from a connected machine, carry it in, then import it in Organization > Settings > Licensing (or the Portal Administrator Directory). The license is stored in the portal config store, so it applies across both portal nodes automatically.
• ArcGIS Server: import its 11.3 license file in Server Manager > Licensing, or run the configure/authorize script with the import option, on each machine.
• ArcGIS Data Store: no license file is required.

Finally, assign user types (Creator/Professional) and add-on licenses to the 40 named users. The license file defines entitlements; it does not assign them, and users will be blocked from publishing until assignment is done.

That message is conclusive: the administrator ran the online authorization path. In 10.9.1 and 11.x, the setup/licensing wizard offers an option to authorize against ArcGIS Online (or an ArcGIS license server). On an air-gapped network that call can never succeed, and because nothing was written, the junior admin's attempt left no partial state - which is good news.

The offline method is file-based import, not authorization:

1. From a machine with internet access, sign in to My Esri and generate the license file for the correct product and the correct version. A 10.9.1 file will not license 11.3, and a Portal for ArcGIS file will not license ArcGIS Server.
2. Carry the file in on the portable drive and place it on the machine running the configuration.
3. Portal: Organization > Settings > Licensing > Import (or https://portal/sharing/rest/portals/self/importLicense via the Administrator Directory). Verify the expiration date afterwards and confirm the status page shows the expected user-type counts.
4. ArcGIS Server: Server Manager > Licensing > Import, on every machine in the site.
5. Remove the license files from the portable drive after import; they are sensitive.

Two details worth checking before the window. First, make sure no proxy or firewall rule is configured to silently retry an online call and hang the wizard for minutes. Second, verify that the license file's user-type and app entitlements match what the agency actually purchased, because discovering a missing add-on entitlement at 02:00 with no internet is unrecoverable. Print or save the current license status page before starting so you can compare afterwards.

Rollback posture: the upgrade is effectively all-or-nothing. A partially upgraded deployment - 11.3 portal and servers with a 10.9.1 data store still upgrading - is an unsupported, unstable state, and you cannot roll back the portal alone and keep the upgraded servers. The only safe rollback is a consistent restore of all six machines to snapshots taken at the same instant, before any component was touched. Attempting a selective restore of one tier is how organizations turn a failed window into a multi-day outage and data loss in hosted feature layers.

Decision rule at hour 7: if the data store upgrade is progressing (logs advancing, no errors), let it run and use the remaining hours for validation. If it is genuinely stalled, stop, record the exact state and error text for Esri Support, and execute the coordinated restore of all six snapshots at once - do not spend the window improvising a repair.

What should have been done differently with no spare hardware:

• Clone, don't guess. VM clones restored to an isolated network are a legitimate test environment and cost only storage. Rehearse the full sequence there, including the license import and one publish test.
• Rehearse the restore. A snapshot you have never restored is a hope, not a rollback plan. The DBA's inexperience with restores was the single largest risk in the plan.
• Sequence for the worst case. Front-load the longest, riskiest step and build explicit go/no-go checkpoints with a decision owner who is not the person doing the clicking.
• Budget realistic time. Two portal nodes, two servers, and a data store pair plus validation rarely fit comfortably in 10 hours when each tier needs a health verification step.