Instance Healthy After Reboot, Portal Unreachable

Console access proves the instance is running, the OS booted, and credentials are not the problem. It rules out hypervisor failure, a failed boot, and lost SSH keys. It does not prove anything about networking, listeners, or the address the outside world should use.

Order:

1. Look up the instance's current public IP in the provider console or API. Auto-assigned addresses are usually released on stop and a different one assigned on start. The engineer connects to the old literal 203.0.113.45 and times out; that single fact explains both the SSH and the HTTPS symptom simultaneously.
2. From the serial console run ip -4 addr to confirm the internal address and ss -tlnp to see whether anything is listening on 443 and 3000.
3. Only then examine security groups and ACLs. They are unchanged and the ACL allows all, so changing rules would be action without a hypothesis.

Reason about the error type: timeouts mean packets are going nowhere or being silently dropped, while connection refused means the host answered and nothing was listening. Both symptoms being timeouts, with unchanged firewall config, points at the destination address.

Refused means the packet reached the host and the host sent a TCP reset because nothing is bound to 443. nginx was started by hand and never enabled, so the reboot left it stopped. The reboot did not break nginx; it revealed that its running state was never persistent.

Fix by making both tiers supervised: write a unit for the Node service, for example ExecStart=/usr/bin/node /opt/portal/server.js, WorkingDirectory=/opt/portal, User=portal, EnvironmentFile=/etc/portal.env, Restart=always, After=network-online.target, then systemctl daemon-reload && systemctl enable --now portal. Enable the distribution's nginx unit with systemctl enable --now nginx. Verify in layers: ss -tlnp for 3000 and 443, curl -I http://127.0.0.1:3000 for the app, curl -kI https://127.0.0.1 for TLS and proxying, then test from outside.

The error taxonomy matters: refused means no listener; 502 means nginx is listening but cannot reach the upstream, which would point at the Node process, the proxy_pass address, or SELinux/AppArmor blocking the proxy connection.

The root cause is that the running system existed only as a sequence of manual actions. Encode it: a cloud-init user-data script or a configuration-management role that installs packages, writes the systemd units and nginx site config, enables both services, and deploys the application artefact at a pinned version. Better still, bake a machine image so a replacement instance is identical by construction.

Separate addressing from the instance: attach a reserved static IP or put a load balancer with health checks in front so the DNS record never points at an address that can be reassigned. If DNS must follow the instance, keep the TTL low and automate the update.

Add external synthetic monitoring that requests the public URL every minute and alerts on failure, rather than relying on provider status checks that only test the hypervisor and guest agent. Practise the failure: reboot the instance in staging as a routine drill, and treat "survives reboot" as an acceptance criterion for any new service.