The Assessment the Sponsor Ordered
A national energy utility has run a PMO for four years. Its director has commissioned an assessment and told you the deliverable is "a maturity level, scored against a recognised model". Twenty-two of 60 project managers responded to the survey; the PMO director supplied the interview list. Sponsors describe the PMO as "bureaucracy that arrives two weeks after we needed the decision", while project managers say the templates and the risk register genuinely help. Delivery data tells a third story: 70% of projects are on time against baseline, but baselines are re-set on average 2.3 times per project. The CIO, who funds the PMO, wants the result in six weeks and has asked whether to cut the PMO's headcount.
1. What are the validity problems in the evidence you have, and how do you close them within six weeks?
Read the worked answer
Three distinct problems, each needing a different fix.
Self-selected non-response. 22 of 60 is a 37% response rate and you cannot tell whether respondents skew to the engaged or the aggrieved. Close it by sending a short census instrument to the remaining 38 (five minutes, five questions) and reporting the response bias explicitly rather than pretending the 22 are representative.
A sponsor-supplied interview list. The PMO director choosing whom you interview guarantees a favourable sample, whether or not that was the intent. Redraw it yourself from the portfolio: sponsors of the ten largest projects, the five projects with the worst re-baselining, plus a random sample of five. Add two business unit heads who do not sit on the steering committee.
A gamed performance metric. 70% on-time against a baseline that moves 2.3 times per project is not a delivery measure; it is a measure of how often the plan was rewritten. Replace it with schedule predictability (baseline change frequency, forecast accuracy measured at mid-point) and decision latency, which is the actual complaint. Test the phrase "two weeks after we needed the decision" on a sample of gate records: date requested, date decided.
Triangulate four sources - survey, interviews, artefacts (gate records, risk logs, decision logs), delivery data - and report where they disagree, because the disagreement is the finding.
2. The director wants a maturity score. What do you actually deliver, and how do you handle that disagreement?
Read the worked answer
A maturity score answers "how do we compare to a model", not "is this PMO worth what it costs". The CIO's question is the second one. Deliver a findings report in three parts: what the PMO does that stakeholders value, with evidence and rough magnitude; what it does whose cost exceeds its return, with evidence; and a small set of changes with expected effects and a way to tell whether they worked.
If a score is genuinely needed - for external benchmarking, or because the funding conversation needs a comparable number - provide it, but put it in an appendix with its limitations stated: model-dependence, self-report bias, threshold arbitrariness, and the fact that level movement is slow and weakly linked to outcomes.
Handling the director: agree the criteria for a good assessment at the outset - defensible, useful, acted on. A bare level is not actionable and invites the CFO to ask "so what", which is the outcome the director is trying to avoid. If the director wants the level as the headline for political reasons, that is their call as the client; the underlying evidence must still be in the report so the number is not the only artefact.
What I would not do is soften findings to protect the PMO. That is the failure mode that gets PMOs defunded two years later, when the same issues resurface with less credibility attached.
3. How do you answer the CIO's headcount question without overreaching?
Read the worked answer
You cannot responsibly answer it from a six-week assessment, because you have no workload or demand data - so say that, and then give the CIO what they actually need.
Reframe to the decision: the CIO is asking whether headcount is the right lever. Offer a service-level view instead of an org-chart view. Sample two weeks of actual effort by service (status collection, decision support, benefits work, tooling administration). Add the demand queue: how many requests, how long to respond. Then state what would not be done if two roles were removed, and what would improve if two roles were redeployed.
Flag the confounder directly: the sponsor complaint is about latency, not volume. Cutting heads is likely to worsen the thing sponsors are complaining about, because decision turnaround depends on having someone available to work the decision. Recommend a decision point rather than a number - for example, revisit headcount after a 90-day service-mix change, judged on decision turnaround and forecast accuracy.
Being explicit about the limits of your evidence is not a weakness in front of a CIO; giving a headcount number the data cannot support is.