Scoping six teams in one tenant without splitting the platform

Against, unless there is a hard regulatory isolation requirement that internal scoping genuinely cannot satisfy.

• Traces break at the boundary. If Payments runs in its own environment, every call from checkout into payments becomes an unresolved external call in the checkout environment. You lose the end-to-end chain exactly where incidents are most expensive. Cross-environment correlation is not a substitute for a single trace.
• Configuration drift doubles. Alerting profiles, maintenance windows, dashboards, tagging rules, and privacy settings all have to be maintained twice and will diverge within a quarter.
• It does not actually deliver least privilege. Access control is per environment, so a separate Payments environment gives everyone with access to it everything - including the six engineers who shouldn't see it. You would then have to build scoping inside that environment anyway.
• Duplicate host-unit consumption if any host is monitored by both environments during migration, which is exactly the overlap window that lasts longest.

A separate environment is defensible for regulatory data residency, or where the blast radius of a config error must be isolated - not as an access-control mechanism.

Three layers, in order:

1. Automatic tagging as the labelling layer. Derive tags from host groups, Kubernetes namespace and labels, cloud metadata, and service ownership - never manual tags, which rot within weeks and silently break scoping rules. One consistent naming convention (team:payments, env:prod) that everyone inherits for free.
2. A management zone (or a segment in a Grail-native tenant) as the enforcement boundary. This is what actually restricts what a group can see. Group rules combine the tags from layer 1. Note the common trap: tags by themselves do not restrict access - they only feed rules and let people filter. A user with tenant-wide access can still search outside the tag.
3. IAM group → policy bound to that zone/segment. The 12 Payments engineers get a policy scoped to the Payments zone; the SRE group gets broad access for triage. Keep the number of zones small and rule-generated so a new namespace is scoped automatically rather than by ticket.

Separately: "payloads must not be visible" is a data-privacy control, not a visibility control. Scoping the zone does not mask captured request data. That requirement needs masking/exclusion on the request-attribute capture rules for those services, applied globally, and verified by opening a PurePath as an authorised admin and checking what is actually recorded.

Instrument them infrastructure-only (agent without full-stack code monitoring) and be explicit about the trade:

• You keep: host and process availability, CPU/memory/disk/network metrics, file system and log-level signals, process presence alerting, and capacity trending.
• You lose: PurePaths, service-level response times and failure rates, code-level method hotspots, and backtrace-level exception analysis. Those hosts will show as gaps in any service dependency map - make sure that is understood before an incident, not during one.

Verify against the agent support matrix first. If the OS version is outside the supported range you may need to pin an agent version or fall back to generic/extensions-based monitoring for the metrics you actually need. Do not leave them unmonitored by default; a silent host is worse than a known-limited one.

Then give the platform team an explicit expiry: these hosts are monitored at reduced fidelity until replacement on date X. Reduced-fidelity monitoring that nobody remembers is how an EOL estate survives three more years.