Calibrating to the Wrong Year: Weather Data and the Fudge Factor

The annual match is a coincidence, not evidence of calibration. Two seasonal errors of opposite sign are partially cancelling in the annual total.

Root cause - wrong weather year. TMY3 is a synthetic typical year assembled from a composite of months across decades. Its December–February temperatures represent a normal winter; the billing period contained a January about 30% colder than normal on heating degree days. The model therefore cannot see the load the building actually carried. The same logic explains July: a mild actual July against a composite typical July produces the over-prediction.

The residual shape tells you the problem is in the shape of the load, not its magnitude. Chasing the magnitude first is backwards.

Actions, in order:

1. Obtain an actual meteorological year (AMY) file for a station representative of the site, covering the exact billing period, and re-run. This is the single change most likely to collapse both residuals.
2. Verify period alignment. Utility meter read dates rarely fall on the first and last of the month. A read on the 18th shifts roughly two weeks of consumption into the adjacent month and can alone generate 10%+ monthly residuals.
3. Confirm the fuel units and conditions - therms vs. CCF, and any altitude correction on gas volume.
4. Only after 1–3, examine model inputs (schedules, setpoints, economizer operation, boiler staging).

If no AMY exists, regress both bills and model output against HDD/CDD and compare the normalized coefficients rather than raw monthly totals, and say so explicitly in the report.

Criteria. Apply ASHRAE Guideline 14 statistical indices at the interval your data supports (verify thresholds against the edition your program cites):

• Monthly data: normalized mean bias error (NMBE) within ±5% and CV(RMSE) ≤15% of the mean monthly consumption, using a minimum of 12 months.
• Hourly data: NMBE within ±10% and CV(RMSE) ≤30%.

Treating the failing seasons.

• Do not tune inputs to individual months. The sequence is: correct the weather year first, re-run, then re-evaluate. Reporting a statistic on a model driven by the wrong weather is meaningless regardless of how it comes out.
• If a month still fails after the AMY re-run, the diagnosis is now physical: check holiday and vacation schedules, boiler staging and lockout, economizer operation in the shoulder months, and missing end uses (domestic hot water, kitchen gas, process loads, tenant submeters outside the model boundary).
• Look at the sign and autocorrelation of the residuals, not just the aggregate statistic. A model with acceptable annual CV(RMSE) but residuals that are all negative for six consecutive winter months is not calibrated; the errors are systematic, not random.
• Report all twelve months, including the failures, with a written explanation of each. A calibration report that shows only the months that passed will not survive third-party review.
• Record every adjustment, the evidence for it, and the before/after index in a calibration log attached to the model file.

Reject it. This is a back-solved fudge factor, not a calibration.

Why it is wrong on the physics. Boiler efficiency is a property of the equipment, established by the combustion test, nameplate, and part-load curve - not by whatever number makes the annual gas total match. Raising it to 84% asserts an equipment performance the survey does not support, and it does not fix the load-shape error in the slightest: the model still sees a normal January, it just burns less gas during it.

Why it is wrong on the analysis. The baseline is the counterfactual against which every ECM will be judged. Corrupting it biases every downstream number: a heating-side measure will show savings that are partly an artifact of an artificially efficient baseline plant, and those savings will not appear in the measured data after installation. The error also propagates to the incentive claim, where it becomes a financial and reputational exposure.

What is defensible instead:

• If gas is genuinely unaccounted for, add the missing end use. Domestic hot water, kitchen gas, and process loads are common omissions in office modeling and adding them is a legitimate, documented fix.
• If the boiler survey shows the plant operates at 78% combustion efficiency but distribution losses are unmodeled, add the distribution loss explicitly rather than burying it in the boiler.
• Fix the weather first; most of the 18–22% winter residual will disappear without touching a single HVAC input.

The general rule: calibrate by correcting identified, evidenced discrepancies; never by adjusting a nameplate parameter until the totals match.