A Failing Level Loop and a Weighted Mean from Observations of Unequal Precision
A 3.2 km closed level loop of 8 setups is run with a digital level; the misclosure is +0.024 m. The firm's specification permits a misclosure of 12 mm x sqrt(K), where K is the loop length in kilometres. Separately, a distance between two control points is observed three times, independently, with declared standard deviations: 152.340 m (sigma = 0.005 m), 152.352 m (sigma = 0.010 m) and 152.346 m (sigma = 0.020 m). A junior technician proposes to close the loop by distributing the 24 mm pro rata to setup lengths and to adopt the simple arithmetic mean of the three distances because it treats every observation equally.
1. Is the level loop acceptable under the specification? What is the correct course of action?
Read the worked answer
Tolerance: 12 mm x sqrt(3.2) = 12 x 1.789 = 21.5 mm. The observed misclosure of 24 mm exceeds the tolerance, so the loop fails and must not be adjusted and adopted.
Why the technician's proposal is wrong: the allowable-misclosure formula is a statistical screen built on the assumption that the remaining error is random. A misclosure outside it is evidence that the error model is violated - that a systematic error or blunder is present. Distributing a blunder pro rata simply smears it across every reduced level and makes it invisible while corrupting all of them.
Correct action:
- Reject the loop and investigate. Typical causes: collimation error with unbalanced sight lengths, rod settlement or a sinking tripod, a misread or misrecorded staff reading, refraction over long low sights, or a moving turning point.
- Check that the sum of foresights equals the sum of backsights plus the known difference, and that sight lengths are balanced at each setup.
- Re-observe the loop, ideally with the instrument repositioned so that any residual collimation error reverses sign.
- Only a loop that passes the tolerance should be adjusted, and then by a method consistent with the error model (equal correction per setup if setup lengths are similar; pro rata to length if they are not).
2. Compute the least-squares estimate of the distance and its standard deviation. Compare it with the simple mean and explain the difference.
Read the worked answer
Weights are proportional to inverse variance, w = 1/sigma^2:
| Obs | Value (m) | sigma (m) | sigma^2 | w |
|---|---|---|---|---|
| 1 | 152.340 | 0.005 | 2.5e-5 | 40,000 |
| 2 | 152.352 | 0.010 | 1.0e-4 | 10,000 |
| 3 | 152.346 | 0.020 | 4.0e-4 | 2,500 |
Sum w = 52,500.
Weighted mean = (152.340x40,000 + 152.352x10,000 + 152.346x2,500) / 52,500 = (6,093,600 + 1,523,520 + 380,865) / 52,500 = 7,997,985 / 52,500 = 152.3426 m (report 152.343 m).
Variance of the estimate = 1 / sum(w) = 1/52,500 = 1.905e-5; sigma = 0.0044 m.
Simple mean = (152.340 + 152.352 + 152.346)/3 = 152.3460 m.
Why they differ: the simple mean implicitly assumes equal variances, which is false here. By treating the 20 mm observation as equal to the 5 mm one, it pulls the result 3.4 mm toward the worst measurement - exactly the wrong direction. The weighted mean is pulled toward the strongest observation and, as a bonus, its standard deviation (4.4 mm) is better than even the best single observation (5 mm). The weighting is only as good as the declared sigmas; they should come from the instrument specification or a prior adjustment, not from guesswork.
3. Using residuals and standardized residuals, assess whether any of the three observations should be rejected as a blunder.
Read the worked answer
With the adjusted value x = 152.3426 m:
| Obs | Value | v = value - x | v/sigma |
|---|---|---|---|
| 1 | 152.340 | -0.0026 | -0.52 |
| 2 | 152.352 | +0.0094 | +0.94 |
| 3 | 152.346 | +0.0034 | +0.17 |
Weighted residual check: 40,000(-0.0026) + 10,000(+0.0094) + 2,500(+0.0034) = -104 + 94 + 8.5 = -1.5, i.e. zero within rounding - the solution is consistent.
Judgement: the largest standardized residual is under 1.0, far below the usual 2 or 3 threshold, so there is no statistical basis for rejection. The 20 mm observation looks poor only because of its stated precision, not because it is inconsistent.
Honest limitation to state: three observations with one unknown give redundancy r = 2, so a posteriori variance factor and any data-snooping test rest on very little. With two degrees of freedom a chi-square test has almost no power, and a single standardized residual cannot reliably separate a blunder from an optimistic sigma. The defensible conclusion is to retain all three, use the weighted mean, and treat the declared sigmas as the weak link; with more observations, an a posteriori variance factor (and a w-test / Baarda data snooping) would be the proper tool.