Seven flaky tests in a 320-test suite
A Kotlin Multiplatform team has 320 JUnit 5 tests. Seven fail intermittently on CI (about 1 in 25 runs) but never locally. The common pattern: ViewModel tests call runBlocking { viewModel.load() }, the ViewModel uses viewModelScope.launch(Dispatchers.Default), and the test asserts state after delay(1_000) or Thread.sleep(500). A repository test mocks a final Kotlin service with Mockito and occasionally sees MockitoException: Cannot mock/spy class ... final class. The build uses Kotlin 2.0, coroutines-test 1.8, JUnit 5, and Gradle 8.8 with test retry enabled; retries hide the failures. One developer suggests converting everything to runTest and adding Dispatchers.setMain(StandardTestDispatcher()); another warns this will break tests that use withContext(Dispatchers.IO).
1. Explain the root causes of the seven flaky tests and why test retries make the suite less trustworthy.
Read the worked answer
The tests use real time and real dispatchers. runBlocking { viewModel.load() } waits only for the suspend function to return, not for viewModelScope.launch(Dispatchers.Default) to finish. The assertion then runs while the coroutine is still queued, or after an arbitrary delay(1_000)/Thread.sleep(500). On a fast local machine the coroutine wins; on a loaded CI worker it loses. Dispatchers.Default is a shared pool sized to CPU cores, so scheduling latency varies with other tests. If Dispatchers.Main is not replaced, viewModelScope may fall back to a real main dispatcher or throw depending on the artifact. The Mockito failure is separate: Kotlin classes and methods are final by default, and older Mockito cannot mock final classes without the inline mock maker. Retries hide all of this: a 1-in-25 failure becomes a green build after one retry, but the race remains, and the suite now reports success for a test that did not reliably pass. Retries also increase CI time and make failure attribution harder. Treat flakiness as a product bug in the test harness, not as infrastructure noise.
2. Rewrite the ViewModel test strategy: injected dispatchers, `runTest`, `StandardTestDispatcher`, virtual time. Show a sketch.
Read the worked answer
Inject dispatchers instead of reading global ones. The ViewModel takes private val defaultDispatcher: CoroutineDispatcher = Dispatchers.Default and launches with it; the repository takes an ioDispatcher. In the test: runTest { val dispatcher = StandardTestDispatcher(testScheduler); Dispatchers.setMain(dispatcher); val vm = LoaderViewModel(fakeRepo, dispatcher); vm.load(); advanceUntilIdle(); assertEquals(Loaded, vm.state.value) }. runTest installs a TestCoroutineScheduler; StandardTestDispatcher queues coroutines until runCurrent() or advanceUntilIdle(), so assertions are deterministic. Virtual time means a delay(5_000) in production code completes instantly. Dispatchers.setMain is required because viewModelScope uses Dispatchers.Main.immediate. Do not replace Dispatchers.IO with a test dispatcher globally; inject the IO dispatcher and use the same scheduler, or a fake that executes on the test thread. Use advanceUntilIdle for terminal state and runCurrent when you need to assert an intermediate loading state. Add @AfterEach with Dispatchers.resetMain(). The test should fail if the coroutine is not completed, not sleep and hope.
3. Address the Mockito final-class error. What are the options, and what is the minimal safe change?
Read the worked answer
First check the Mockito version. Mockito 5 uses the inline mock maker by default and can mock final Kotlin classes; Mockito 4 needs mockito-inline instead of mockito-core, or the mock-maker-inline resource. The minimal safe change is to upgrade to Mockito 5.x and keep the existing tests, or add mockito-inline if an upgrade is blocked. Alternatively, use MockK, which supports final classes and has coroutine-aware coEvery/coVerify; that is a larger test rewrite but better for suspend functions. Do not add open to production classes solely for mocking: it weakens the API, changes binary shape, and can affect equality and subclassing. For suspend functions, plain Mockito when does not work; use mockito-kotlin with whenever and runBlocking, or MockK coEvery. Also annotate with @ExtendWith(MockitoExtension::class) and avoid Mockito.mock inside a loop. Add a CI step that runs the suite three times without retries to catch residual nondeterminism; if it fails, fix the dispatcher injection rather than adding more retries.