Consul Associate cheat sheet

Every node runs exactly one agent in either client or server mode; only servers hold Raft state and vote in the consensus quorum.

Raft quorum for N servers is (N/2)+1: a 3-server cluster tolerates 1 failure, a 5-server cluster tolerates 2, a 2-server cluster tolerates 0.

Servers use Raft for consensus and Serf gossip for membership; clients take part in LAN gossip only and forward RPCs to servers.

Default ports: server RPC 8300, LAN Serf 8301, WAN Serf 8302, HTTP 8500, HTTPS 8501, DNS 8600, gRPC 8502, gRPC-TLS 8503.

A datacenter is a single LAN gossip pool with its own servers; the catalog and KV store are not replicated between datacenters.

Read modes are default (leader-lease based), consistent (round-trip to the leader, linearizable), and stale (any server, possibly outdated).

The catalog is the authoritative record of nodes, services, and checks; agents anti-entropy sync local registrations into it.

Cross-datacenter lookups are handled by prepared query failover blocks or by forwarding the request to servers in the remote datacenter.

bootstrap_expect must carry the same value on every initial server (normally 3 or 5); bootstrap triggers automatically once that many servers have joined.

Using bootstrap = true on more than one server can produce separate clusters; bootstrap_expect is the supported production method.

Server essentials: server = true, data_dir, node_name, datacenter, bind_addr or advertise_addr, and encrypt for gossip.

retry_join accepts addresses, DNS names, or cloud auto-join strings and retries indefinitely; retry_join_wan joins the WAN gossip pool.

The gossip key from consul keygen is symmetric and must be identical on every agent in the same gossip pool before joining.

A newly joined server is a non-voter; Autopilot promotes it to voter only after it is stable and has caught up on the Raft log.

enable_auto_encrypt lets servers distribute TLS certificates to clients so client agents can speak TLS to servers automatically.

Clients run local health checks and cache catalog data but never store Raft state; they send all write RPCs to servers on port 8300.

Services are defined in JSON/HCL files in the config directory or submitted to /v1/agent/service/register; each needs a unique name and optional id.

Supported check types include script, HTTP, TCP, TTL, gRPC, Docker, alias, and h2ping.

A TTL check stays passing only while the service calls /v1/agent/check/pass/<check_id> within the TTL; otherwise it becomes critical.

Health states are passing, warning, and critical; DNS and API calls with the ?passing filter return only passing instances.

DNS format is .service..consul; SRV records return port and node, and ..service.consul filters by tag.

Prepared queries are stored in the catalog, runnable over DNS or HTTP, and support tag filters, near=_agent sorting, and remote-datacenter failover.

Anti-entropy runs periodically on each agent to reconcile locally registered services and checks with the catalog on the servers.

Catalog endpoints include /v1/catalog/services, /v1/catalog/service/, /v1/catalog/node/, and /v1/health/service/.

enable_tag_override allows an external system to modify service tags without the local agent overwriting them during anti-entropy.

KV endpoints are GET, PUT, and DELETE on /v1/kv/; ?recurse returns a prefix, ?keys returns key names only, ?raw returns value bytes only.

Check-and-set uses ?cas=; ?cas=0 creates the key only if absent, and a mismatched index returns HTTP 409 Conflict.

Each entry exposes CreateIndex, ModifyIndex, LockIndex, Flags, and the base64-encoded Value.

Values are capped at 512 KB per key; keys are UTF-8 strings and hierarchical prefixes are separated with /.

Sessions provide locking: POST /v1/session/create with TTL and behavior release or delete, then write with ?acquire=.

Blocking queries pass ?index= and ?wait=; the server holds the request until data changes or the wait expires, capped at 10 minutes.

KV data is scoped to a single datacenter and is not replicated across datacenters.

The /v1/txn endpoint applies multiple KV operations atomically within one request.

With ACLs enabled the default policy is deny; acl_default_policy = allow restores permissive behavior for unauthenticated requests.

Token types are management (unrestricted) and client tokens bound to policies and, in newer versions, roles and service identities.

Policy rules use resource blocks (node, service, key, keyring, operator, query, session, mesh) with access levels read, write, list, or deny.

Run consul acl bootstrap once to create the initial management token; it is returned only at creation time and must be stored securely.

Gossip encryption uses a symmetric key from consul keygen set in encrypt; every agent in the same gossip pool must share it.

Rotate gossip keys with consul keyring -install, then -use, then -remove; -relay-factor controls how quickly the new key propagates.

Server TLS requires verify_incoming, verify_outgoing, and verify_server_hostname set true, plus ca_file, cert_file, and key_file.

Clients supply ACL tokens via the X-Consul-Token header, the token query parameter, or the CONSUL_HTTP_TOKEN environment variable.

ACLs are written in the primary_datacenter and replicated to secondary datacenters, which cannot modify the ACL store locally.

Intentions define which source services may connect to which destinations; with Connect enabled, connections with no matching intention are denied.

Intention sources and destinations may be a service name or the '*' wildcard, and the action is allow or deny.

A sidecar proxy is registered as -sidecar with kind connect-proxy and started by consul connect envoy -sidecar-for .

Connect establishes mutual TLS between proxies; certificates come from the built-in CA or Vault and are rotated automatically before expiry.

Mesh gateways proxy Connect traffic between datacenters and network segments and are started with consul connect envoy -mesh-gateway.

Terminating gateways expose external non-Consul services into the mesh; ingress gateways expose mesh services to external clients.

L7 traffic management uses config entries: service-defaults, service-resolver, service-router, and service-splitter.

Envoy sidecars receive xDS configuration from the local agent's gRPC endpoint on port 8502, or 8503 when gRPC TLS is enabled.

The sidecar's Envoy admin listener defaults to 127.0.0.1:19000 and is separate from the proxy's service listener.

consul snapshot save writes a Raft snapshot containing the KV store, catalog, ACLs, sessions, prepared queries, and intentions.

consul snapshot restore must target the same datacenter and overwrites existing state; snapshots are taken from the leader.

Autopilot tunables include server_stability_timeout, last_contact_threshold, max_trailing_logs, min_quorum, and cleanup_dead_servers.

Autopilot promotes stable non-voting servers to voters and, when cleanup_dead_servers is true, removes failed servers that are not needed for quorum.

Upgrade servers one at a time, confirming health with consul operator raft list-peers before moving on, then upgrade client agents.

consul members shows gossip membership; consul operator raft list-peers shows server ID, address, voter state, and Raft protocol version.

The telemetry stanza configures statsd, dogstatsd, or prometheus_retention_time; /v1/agent/metrics exposes metrics over HTTP.

GET /v1/status/leader returns the current Raft leader address (empty when none is elected) and /v1/status/peers returns the peer set.

consul leave gracefully removes a node from gossip, while consul force-leave evicts an unreachable node that is still listed as a member.