Testing a Zero Trust Appliance Claim Against NIST and CISA
A state benefits agency serves 5,000 employees and 2.5 million residents. About 60% of workloads remain on-premises, 20% are in one cloud, and the rest are SaaS. MFA covers 70% of users, but there is no authoritative device inventory, VPN is required for most internal apps, and the network is flat across nine floors. The agency must align with NIST SP 800-207 and uses the CISA Zero Trust Maturity Model for reporting. A vendor demonstrates a single appliance and claims it makes the agency NIST 800-207 compliant and mature across all CISA pillars. The CIO is interested because the procurement deadline is 60 days away. The security team has 12 staff, limited cloud engineering, and a mandate to show measurable progress this fiscal year. The agency has an Okta tenant for cloud apps but Active Directory remains authoritative for on-premises.
1. Evaluate the vendor claim that one appliance makes the agency NIST SP 800-207 compliant and mature across all CISA pillars.
Read the worked answer
The claim is not credible as stated. NIST SP 800-207 defines a logical architecture, not a certification: it expects a policy engine, policy administrator, and policy enforcement points, with dynamic access decisions based on identity, device, and context. A single appliance can be a PEP or a policy engine, but it cannot by itself supply authoritative identity, device inventory, data classification, or governance. It also cannot make the agency mature across all CISA pillars, because the CISA Zero Trust Maturity Model assesses identity, devices, networks, applications and workloads, data, and visibility and analytics, with cross-cutting governance.
The agency facts show gaps: 70% MFA, no device inventory, VPN for most internal apps, and a flat network. The vendor should be asked for evidence: which NIST tenet each feature supports, whether the policy engine is separate from the PEP, how it consumes identity and device signals, how it logs decisions, and how it handles non-agent devices. A product can accelerate a pillar, but maturity requires process, inventory, and governance. Treat compliant as a marketing term unless it maps to a specific control and an auditable evidence source.
2. Using CISA ZTMM, identify the two most important gaps and a 12-month sequence. Explain why not all pillars move at once.
Read the worked answer
Most important gaps: identity and device visibility. Identity: MFA at 70% means 30% of users, likely including privileged or legacy-app users, can authenticate with weaker methods; the agency should move to phishing-resistant MFA and centralized identity for on-premises apps, not just cloud. Devices: no authoritative inventory means the agency cannot make device-based access decisions; it needs an inventory fed by MDM/EDR and network discovery, with a posture baseline.
Sequence: months 0-3 identity and device inventory; 3-6 policy engine and SSO for the highest-risk on-prem apps; 6-9 network segmentation for a small enclave and VPN reduction; 9-12 data classification and visibility metrics. Do not move all pillars at once because the agency has 12 staff and limited cloud engineering; spreading effort across six pillars would produce demos without operations. CISA model is cumulative, but the agency can advance one pillar from initial to advanced while others remain traditional. The sequence should follow dependency: device and identity signals are inputs to policy; network enforcement without them is guesswork. Report progress with evidence per pillar, not a single maturity score.
3. What pilot success criteria should the agency set before expanding the vendor product?
Read the worked answer
Set criteria tied to a bounded use case: access to three high-risk on-premises benefits applications. Success should require: 100% of pilot users authenticated with phishing-resistant MFA; device posture evaluated for at least 90% of pilot endpoints; access decisions logged with user, device, resource, and policy version; unauthorized-access attempts blocked in a red-team test; median access latency within an agreed threshold; and no increase in help-desk tickets above a baseline. Also require an exit plan: can the agency remove the appliance without losing identity or logging? And a data-portability test for policy and logs.
Reasoning: a pilot should test whether the product fills the agency gaps, not whether it can be installed. The vendor claim is about compliance and maturity; the pilot should produce evidence for one CISA pillar and one NIST tenet at a time. If the appliance cannot consume the agency device inventory or separate policy decision from enforcement, that is a design gap, not a training issue. The 60-day procurement deadline should not force a sole-source decision; a short proof of concept with clear criteria is a better risk control.