Allocating shared and idle cost in a multi-tenant cluster
Your company runs three production Kubernetes clusters in one cloud account: 240 nodes across general-purpose, memory-optimized, and spot node pools, costing $412,000 last month. OpenCost reports that 18% of spend ($74,160) is unallocated: control-plane fees, cluster-wide logging and ingress, DaemonSets, and idle capacity from over-provisioned node pools. Namespace labels exist for 78% of pods; nine legacy workloads in the payments and fulfillment namespaces have no team label and cannot be changed before the next release freeze. Finance has announced that chargeback begins next quarter and requires 95% of cost allocated to a team; the platform team argues for an explicit unowned bucket. Two business units already dispute their shares, claiming shared services are charged twice. The CFO wants a defensible method documented before the first invoice.
1. How should the $74,160 of unallocated cost be treated? Recommend an allocation method and explain how it avoids double-charging.
Read the worked answer
Split the $74,160 into three pools and allocate each once:
- Shared services with a measurable driver - ingress by request count or bytes, logging by ingested volume, and control-plane cost per cluster split by namespace CPU requests. Use the driver, not CPU, whenever CPU is not what causes the cost.
- System overhead - DaemonSets and kube-system consume node-hours, so allocate them by node-hours or by a per-node surcharge, not by tenant usage.
- Idle capacity - the residual after direct and shared allocation. Keep it as a visible line owned by the platform team. Do not spread it silently across tenants; that hides the sizing decision and is the likely source of the double-charge complaint.
Publish two views: direct (what a team consumed) and fully loaded (direct + shared + a labeled idle surcharge). Use fully loaded for budgeting and direct for behavior. If finance insists idle be spread, spread it by CPU requests because requests drive provisioning, but keep the idle component visible. Reconcile all lines to the invoice monthly and document the hierarchy: direct costs first, shared costs at the lowest common denominator, residual last.
2. How do you assign the nine unlabeled legacy workloads without guessing, and what control prevents this recurring?
Read the worked answer
Unlabeled pods still have cgroup usage and a node identity. Map pod → node → node pool → instance type and price the node-hours; that gives a defensible cost even without a team label. Then assign ownership through the deployment's GitOps repository or namespace ownership record. Have the application owner and finance sign a one-page mapping table for the nine workloads - a one-time control, not an estimate.
For anything genuinely unknown, create an unmapped bucket owned by the platform team, capped at, say, 2% of the invoice and with a 30-day remediation date. Report it weekly. Do not split the residual evenly across namespaces or by pod count: payments and fulfillment are likely heavy, and an even split creates a new dispute rather than resolving one.
Prevent recurrence with an admission policy that requires team and cost-center labels on every new deployment, with a default that routes to the platform bucket. Track data quality as the percentage of pod-hours labeled, not the percentage of pods, because a long-running unlabeled pod distorts the result far more than a short-lived one.
3. Finance demands 95% allocated to product teams; engineering proposes an 80% target plus an unowned pool. What target and governance do you recommend?
Read the worked answer
Separate two questions: does every dollar have an owner, and how much is charged to product teams? You can reach 100% on ownership by assigning shared and idle cost to the platform team, while product-team allocation lands at 92–95%. That is defensible and keeps the incentive to shrink idle capacity.
Recommendation:
- Target: 100% of the invoice has a named owner; at least 95% allocated to product teams; no more than 5% in a platform idle-and-shared pool with a quarterly reduction plan.
- Dispute control: publish the methodology before the first invoice, show direct versus fully loaded views, and give teams a 10-business-day challenge window with a named arbiter.
- Behavior: charge product teams on requests plus their share of shared drivers; charge the platform team for idle above an agreed buffer, such as 15% headroom. That makes the platform team accountable for provisioning, not only the tenants.
Forcing 95% by spreading idle would hit the CFO's number but hide the real savings opportunity and damage trust. A capped, reported 5% platform pool is not a loophole; it is an explicit accountability line.