Measurement System Before Baseline
A Green Belt is measuring 'order processing time' in a shared services team. The team logs a start time when the request lands in the queue and an end time when it is closed. A pilot of 30 orders was timed independently by two analysts using the same log data and by the team lead watching the desk. The two analysts agreed within 2 minutes on 28 of 30 orders. The team lead's timings differed from the log-based timings by 15–40 minutes on 22 of 30 orders, always longer. The team's target is to cut average processing time from 3.2 days to 2.0 days. The Green Belt has not yet run a formal Gage R&R.
1. Assess the measurement system and decide whether to proceed to baseline data collection. Justify with the evidence given.
Read the worked answer
Two distinct questions, and they give different answers:
Repeatability/reproducibility of the log-based measure: the two analysts agreed within 2 min on 28/30 - that is good agreement between appraisers on the same source. This suggests the log definition is stable when read consistently.
Validity against the process: the team lead's observation is systematically longer (22/30, always longer). That is not random error - it is a bias between two definitions of 'processing time'. The log measures queue-arrival to closure; the lead is probably observing actual work time including rework or excluding waiting. Neither is wrong, but they measure different things.
Decision: do not baseline yet. You cannot tell whether the 3.2-day figure reflects the process or the logging convention. Actions:
- Agree an operational definition of start and end (does 'closed' mean customer-confirmed or system-closed?).
- Run a proper Gage R&R on the agreed definition, not on two competing ones.
- Investigate the 15–40 min gap: if it is systematic rework not captured in the log, that is a finding for Analyse, not measurement error to discard.
A 3.2→2.0 day target (37.5% reduction) is large enough that measurement bias of this size could account for a meaningful share of the apparent gap.
2. The team lead argues the log data is 'the official system, so it's correct'. How do you handle this?
Read the worked answer
Separate 'official' from 'valid'. Official means it is the record of authority for reporting; valid means it measures the thing the project is trying to improve.
- If the project goal is customer-perceived turnaround, the log (queue to closure) is closer to what the customer experiences. The lead's observation may be measuring only touch time.
- If the goal is team workload, the lead's measure may be more relevant.
So the question is not which is correct but which one the goal refers to. Recommend: adopt the log as the primary metric (it is what the customer sees and what the target is expressed against), and use the lead's observation as a diagnostic to explain why logged time is long - e.g. if the gap is rework, that is a candidate root cause for Analyse.
Document the definition in the charter/measurement plan so the 2.0-day target is unambiguous. Do not let the disagreement stall the project; convert it into a defined measure plus a diagnostic.