Thermocouple Calibration: When an Exact Fit Is the Wrong Fit
A junior engineer is calibrating a type-K thermocouple against a certified bath. Six readings are recorded at 0, 20, 40, 60, 80 and 100 °C, with EMF values 0.000, 0.798, 1.611, 2.438, 3.280 and 4.138 mV. The engineer fits a fifth-degree polynomial through all six points using the normal equations in a spreadsheet and reports coefficients that reproduce every reading exactly. When one EMF value is corrected by 0.02 mV, the cubic coefficient changes from 1.9×10^-6 to 2.8×10^-6 and the predicted EMF at 50 °C shifts by 0.06 mV. The spreadsheet also shows a condition number of about 2×10^10 for the Vandermonde matrix. The engineer proposes adding a sixth-degree term to improve the fit further. A separate cubic fit gives a residual sum of squares of 4.2×10^-4 mV²; a quartic fit reduces this to 3.5×10^-4 mV². The bath temperature is known to ±0.05 °C, and the plant tolerance on indicated temperature is ±0.3 °C.
1. Diagnose why the fitted coefficients are unstable and set out a numerically sound procedure for the calibration fit. State what should be reported alongside the curve.
Read the worked answer
The raw-temperature Vandermonde matrix has columns 1, T, T², ..., T⁵. With T up to 100, those columns differ by about ten orders of magnitude. Forming the normal equations AᵀA squares the condition number, so a 0.02 mV change is amplified into large coefficient swings. A degree-5 polynomial through six noisy points is interpolation of noise, not calibration.
Use x = (T − 50)/50 so x lies in [−1, 1], fit a cubic by QR or SVD on the scaled design matrix, and choose the degree by residual analysis or an extra-sum-of-squares test. Do not add a sixth-degree term. Report the coefficients in the scaled variable, their covariance from the QR/SVD solution, the residual standard error, the condition number actually used, and residuals at independent check temperatures. Also state the valid temperature range and that extrapolation outside it is unsupported.
2. Using the residual sums of squares, decide whether the quartic term is statistically justified at the 5% significance level. State the assumptions and any limitation of the test.
Read the worked answer
Compare the models by an extra-sum-of-squares F-test. The cubic has 6 − 4 = 2 residual degrees of freedom; the quartic has 6 − 5 = 1.
F = [(RSS₃ − RSS₄)/(1)] / [RSS₄/(6 − 5)] = (4.2×10^-4 − 3.5×10^-4) / (3.5×10^-4) = 0.00007 / 0.00035 = 0.20.
At the 5% level, F(1,1) ≈ 161. The computed F is far below that, so the quartic term is not justified. The reduction in RSS is negligible compared with the remaining residual, and the test assumes independent, constant-variance, Gaussian errors. With only one denominator degree of freedom the test has very low power, so the defensible conclusion is to retain the cubic unless independent evidence supports curvature. The bath uncertainty of ±0.05 °C corresponds to only about 0.002 mV, far below the observed residuals, so the dominant error is likely reading noise or model lack-of-fit, which a higher-degree term will not fix.
3. The EMF readout has a repeatability of ±0.010 mV (1σ). Estimate the 95% uncertainty in indicated temperature near 50 °C, where the local sensitivity is 0.041 mV/°C, and judge whether the ±0.3 °C tolerance can be met.
Read the worked answer
Convert the EMF uncertainty to temperature by dividing by the local sensitivity:
σ_T = 0.010 mV / 0.041 mV/°C = 0.244 °C.
A single reading therefore has a 95% uncertainty of about 1.96 × 0.244 = 0.48 °C, which exceeds the ±0.3 °C tolerance. Averaging n independent readings reduces the standard uncertainty by √n. For three readings, σ_mean = 0.244/√3 = 0.141 °C and the 95% value is 1.96 × 0.141 = 0.28 °C, just inside tolerance. The plant should average at least three readings and include the calibration-curve uncertainty in the budget; the improvement is only marginal, so a better readout or shorter repeated calibration interval is advisable.