Forty order-dependent failures in a 1,850-test suite

Step 1 - separate state leakage from worker races. Re-run the suite with -n0. If the 40 failures persist, the cause is shared state, not xdist. If they vanish, it is per-worker distribution.

Step 2 - pin the order. Use pytest -p no:randomly and a fixed -p xdist -n0, or install pytest-random-order and set a seed, so a failure is reproducible at all.

Step 3 - bisect the polluter. pytest --sw (stepwise) stops at the first failing test; re-run with --sw from that point and repeatedly drop half the preceding tests (--deselect or -k) until one predecessor remains. The pair <polluter> <victim> reproduces it in two lines.

Step 4 - instrument cheaply. An autouse fixture in conftest.py that snapshots len(billing.audit.EVENTS) before and after each test and asserts no net change will name the polluter directly, without bisection.

Expected root-cause class: mutable state with module, class or session scope - module-level lists/dicts, TestCase class attributes, setUpClass caches, or a session-scoped fixture returning a mutable object. The 23:00 failure is a separate defect: a real-time dependency that the mock never intercepted, so the test only fails when the UTC date rolls over mid-assertion.

from datetime import datetime executes at import time and binds the class object into the consuming module's namespace (billing.invoices.datetime). patch('datetime.datetime') replaces the attribute on the standard-library module, which the already-bound name never re-reads. The test therefore exercises the real clock; it passes for most of the day and fails after 23:00 UTC when the stamped date differs from the expected one.

Correct target - patch where the name is looked up:

@mock.patch("billing.invoices.datetime")
def test_invoice_total_is_stamped(mock_dt):
    mock_dt.now.return_value = datetime(2024, 5, 1, 12, 0, tzinfo=timezone.utc)
    ...
    mock_dt.now.assert_called_once()

Prefer injecting a clock callable into the function under test over patching stdlib names at all; then there is nothing to mis-target.

Automatic detection: the decisive signal is that the double was never used. Assert mock_dt.now.assert_called_once() (or a generic assert_called()) in every test that patches - a mis-targeted patch now fails loudly and immediately. Complement that with a conftest.py fixture that freezes time globally (freezegun) so any unpatched real-clock read is deterministic, and treat mock.patch("datetime...") / mock.patch("time...") as a review flag in CI lint.

Keeping -n 4 is fine, but xdist changes nothing about isolation: each worker is a separate process, so leakage between tests that land in the same worker still happens, and --dist load scatters tests differently on every run. Use --dist loadfile to keep a file on one worker so ordering at least stays stable, and fix the underlying state.

Must change:

• EVENTS = [] in billing.audit → expose a monotonic, per-test sink, or add an autouse fixture that clears it (yield then EVENTS.clear()).
• Class attributes on TestCase subclasses → move into setUp.
• Any session/module-scoped fixture returning a list, dict, session or connection → return a factory, or a copy.deepcopy per test.
• Files written to fixed paths → tmp_path.

Scope rule: a fixture may be broader than function scope only if its value is immutable, or if every test receives a copy. Anything mutable, stateful or order-sensitive is function-scoped. Caches and app factories are the usual exception: build them once per session but reset them in an autouse cleanup fixture.

Finally, delete the six time.sleep(0.5) calls - they encode a race, they do not remove it. Wait on a condition with a deadline (poll a predicate, or unittest.mock wait, or tenacity.retry), so a slow worker becomes a clear timeout instead of a random failure.