Strain-Gauge Calibration: Least Squares After a Suspect Reading
A calibration technician records a strain gauge's output y (mV) at five known loads x (kN): (1, 2.10), (2, 3.95), (3, 6.10), (4, 7.85), (5, 10.20). The intended model is y = a + bx. During the run, the technician notices that the fifth reading was taken while a connector was loose and suspects a constant +1.00 mV offset on that reading only. The lab's quality procedure says a slope change of more than 8% after correcting a suspect point must trigger a full recalibration, not just a note in the log. The technician must decide whether to accept the original fit, refit after subtracting the offset, or escalate. A colleague argues that because least squares spreads error across all points, removing the offset cannot change the slope much. The technician needs a quantitative answer and a geometric explanation of why the fitted line changes when one observation is altered.
1. Set up the least-squares normal equations for the five original points and solve for a and b. Show the sums you used.
Read the worked answer
The design matrix has columns of ones and x, so the normal equations are n a + b Σx = Σy, a Σx + b Σx^2 = Σxy. The sums are n = 5, Σx = 15, Σy = 30.20, Σx^2 = 55, and Σxy = 110.70. Thus 5a + 15b = 30.20, 15a + 55b = 110.70. Multiplying the first equation by 3 gives 15a + 45b = 90.60. Subtracting from the second gives 10b = 20.10, so b = 2.01. Then a = (30.20 - 15(2.01))/5 = 0.01. The original fit is y = 0.01 + 2.01x.
2. Repeat the fit after correcting the fifth reading by subtracting 1.00 mV. Compute the percentage change in slope and apply the 8% rule. What should the technician do?
Read the worked answer
The corrected fifth reading is 9.20 mV. The new sums are Σy = 29.20 and Σxy = 105.70; Σx and Σx^2 are unchanged. The normal equations become 5a + 15b = 29.20, 15a + 55b = 105.70. As before, multiply the first by 3: 15a + 45b = 87.60. Subtract from the second: 10b = 18.10, so b = 1.81. Then a = (29.20 - 15(1.81))/5 = 0.41. The percentage change in slope is |1.81 - 2.01| / 2.01 = 0.20/2.01 ≈ 9.95%, which exceeds the 8% threshold. Under the lab procedure the technician should escalate to a full recalibration rather than accept either fit silently; the corrected fit should be recorded, but the trigger shows the calibration is not robust to that suspect point.
3. Explain geometrically why one observation can change the slope by about 10%. Refer to the projection of the data vector onto the column space of the design matrix and to the leverage of the point x = 5.
Read the worked answer
In matrix form the fitted values are the projection of the observation vector y onto the column space of the design matrix X, whose columns are the constant vector and the x-values. Changing one observation moves y, and its projection moves by a corresponding amount weighted by that point's leverage. For simple linear regression the leverage at x_i is h_i = 1/n + (x_i - xbar)^2/Sxx. Here n = 5, xbar = 3, and Sxx = 10, so at x = 5 the leverage is h = 1/5 + (2)^2/10 = 0.6. That is a high-leverage point because it sits far from the mean x. The direct slope-change formula is Δb = (x_i - xbar)/Sxx Δy_i = (2/10)(-1.00) = -0.20, exactly the observed change from 2.01 to 1.81. Least squares does distribute residuals, but a high-leverage observation can still tilt the fitted line strongly because the projection is most sensitive where leverage is large.