The Number for the Annual Report

The figure has four separate problems, and each one changes what can be claimed.

Denominator shifting. The 41% is computed on completers, not on the 640 enrollees. Completers are a self-selected group who attended six sessions - plausibly more motivated and more likely to succeed. The headline number therefore describes the best-case subset, not the program's effect. Recomputing on all enrollees using an intention-to-treat convention, where non-responders count as not quit, gives roughly 214 completers of follow-up times the self-reported rate - a much smaller number - or, stated conservatively, no more than 33% of enrollees with any usable outcome, and far fewer confirmed quits.

Timing and measure. Seven-day point prevalence at the last session is the weakest standard. It is short, it is measured at a moment of maximum social desirability, and it is self-reported with no biochemical verification. Cessation literature distinguishes point prevalence from prolonged abstinence and typically reports six- or twelve-month follow-up.

Missing follow-up. Only 33% completed the six-month survey. The two-thirds who did not respond are not missing at random - relapse is a common reason to stop answering calls. Treating non-responders as quitters would be indefensible; treating them as non-quitters is conservative and honest.

No comparison group. Without one, the 41% cannot be attributed to the program. Some employees quit without any program, and secular trends in tobacco taxation and smoke-free policy move quit rates independently.

What the figure can support: the program reached 640 employees and engaged 371 of them intensively, and among those who completed both the program and follow-up, a meaningful share reported abstinence. That is a delivery and reach story, not an effectiveness claim.

With $12,000 and no randomization, aim for a defensible pre-post design with comparison where comparison is free.

Design. A single-group pre-post with the next cohort, plus a comparison assembled from existing data rather than purchased: for example, quit attempts among employees at an employer that offers only the quitline, or state quitline outcomes supplied at no cost by the state health department. Call it a quasi-experimental comparison, not a trial.

Measurement schedule. Baseline at enrollment, end of treatment, and six months. Keep the instrument to the minimum tobacco surveillance standard - 30-day and 7-day point prevalence, quit attempts, cigarettes per day - so results can be compared with published benchmarks.

Attrition control. This is where the $12,000 should go. Pre-collect two contact methods and a collateral contact at enrollment; schedule follow-up dates at session six; use text-message links; offer a $10 incentive for completing the six-month survey; allow telephone completion with a live interviewer. Raising follow-up from 33% to 70% does more for credibility than any statistical technique.

Analysis. Report the denominator explicitly: enrolled, completed, followed up, and quit, at each time point. Report both a completer rate and an all-enrolled rate, and label which is which. No multivariate modeling is needed at this n, and with this budget it would be a false sophistication.

Process measures. Reach (enrolled per eligible employee, by shift), adoption (employers agreeing to continue), implementation fidelity (sessions delivered as designed), and maintenance. RE-AIM dimensions are cheap to collect and answer the funder's actual question: should this continue?

Reserve a portion of the budget for data management and a short written report. Unanalyzed data is the most common way small evaluation budgets are wasted.

Give each audience the same facts with different emphasis, and never present two different truths.

To the program officer. Lead with what is solid: 640 employees enrolled across nine employers, 371 completed treatment, and a conservative all-enrolled estimate that you can defend. Offer two numbers with explicit labels - 'x% of all enrolled employees reported being quit at six months, counting non-responders as not quit; y% of those we reached reported quit' - and state plainly which one belongs in the annual report and why. Funders accept conservative numbers well; they do not forgive inflated ones discovered later. Offer a one-paragraph methods note for the report.

To the medical director. Agree that the current design cannot prove effectiveness, and then negotiate the constraint. Randomization at the employee level is often not feasible in a benefit program - employees talk to each other, and withholding a benefit raises an ethical issue. Offer a stepped-wedge alternative: staggered rollout by site or shift so that people waiting serve as a comparison, with the wait limited to weeks, not months. That gets closer to causal inference without denying anyone the program.

On the three employers leaving. Do not bury this. Analyze it, because it is the most actionable finding: 33% of employers attributing non-participation to shift scheduling says the delivery model does not fit shift work. Report it as an implementation finding with a proposed fix - on-site or shift-aligned sessions, and a quitline-only option for sites that cannot host groups. Turning a dropout into a design lesson is more persuasive at renewal than a defensible quit rate alone.