NEC Computer Engineering: Unbounded Blocking in a Motor-Drive RTOS Because the Mutex Is a Binary Semaphore

Unbounded Blocking in a Motor-Drive RTOS Because the Mutex Is a Binary Semaphore

An induction-motor drive runs FreeRTOS on a Cortex-M4 at 168 MHz. Three tasks touch shared resources:

Control - priority 5, period 1 ms, worst-case execution 150 µs, hard deadline equal to its period.
Logger - priority 3, period 5 ms, worst-case execution 1.2 ms, purely CPU-bound (writes to a RAM ring buffer that DMA drains), takes no locks.
Sensor - priority 1, period 20 ms, worst-case execution 300 µs. It takes the I²C bus twice per period: a 120 µs register read, and a 600 µs calibration block read once every 20 ms.

The I²C bus is protected by a handle created with the binary-semaphore API rather than the mutex API; the driver blocks while the transfer runs and does not disable interrupts. Bench tests at low load are clean. On the production machine, about one control cycle in 2000 overruns its 1 ms deadline, and the overruns cluster when the logger is active. The team's first proposal is to raise the Sensor task to priority 4 so that it finishes faster.

1. Explain the failure mechanism and compute the worst-case Control task response time with the current design.

Read the worked answer

Mechanism - unbounded priority inversion. The I²C handle is a binary semaphore, which in FreeRTOS gives no priority inheritance. When Sensor (prio 1) holds the bus and Control (prio 5) becomes ready and blocks on the semaphore, Control is now waiting behind a prio-1 task. Logger (prio 3) is unrelated to the lock but outranks Sensor, so it preempts Sensor while Sensor still holds the bus. Control's blocking time is therefore the critical section plus every execution of every medium-priority task that arrives in the meantime - it is not bounded by the critical section at all.

Worst case:

R_control = C_control + (longest critical section) + (medium-priority interference)

R = 150 µs + 600 µs + 1200 µs = 1950 µs

Against a 1 ms deadline that is a miss by roughly 2×, and the observed 1-in-2000 rate is consistent with the case where all three align - the event is rare precisely because it needs Logger to be released during the 600 µs window.

Why the proposal does not help. Raising Sensor to priority 4 removes Logger's ability to preempt it only if Logger is at 3 and Sensor at 4 - but Sensor then also outranks nothing that matters and, worse, the Sensor task's own 600 µs critical section now delays Control identically. The inversion is still present in principle (Sensor at 4 is still below Control at 5), and the change silently alters the scheduling design without documentation. It treats a symptom.

2. Evaluate priority inheritance versus priority ceiling for this system, quantify the improvement, and state whether the fix is sufficient.

Read the worked answer

Option A - real mutex with priority inheritance. Switch the handle to xSemaphoreCreateMutex(). While Sensor holds the lock and Control blocks on it, Sensor inherits priority 5 and Logger (prio 3) can no longer preempt it:

R = 150 + 600 + overhead(≈10) = 760 µs

Deadline met, with 240 µs (24 %) slack. Inheritance is applied only when contention actually occurs, so it costs nothing in the common case.

Option B - priority ceiling. Assign the mutex a ceiling of 5. Sensor is boosted to 5 for the whole critical section whether or not Control contends. Response is the same 760 µs but the boost is unconditional, and the ceiling protocol additionally gives freedom from deadlock for well-formed critical sections. The cost is that Sensor now runs at priority 5 for 600 µs, blocking anything at priority 5 - tolerable here since Control is the only task at that level.

Is that enough? 760 µs against a 1 ms deadline leaves only 240 µs of headroom, and the budget above excludes interrupt handlers: any ISR that runs during the 600 µs window adds directly to R. A single 200 µs ADC or PWM ISR would consume almost all remaining slack. So the fix is correct but not comfortable.

Recommended structural change: cut the worst-case critical section. Split the 600 µs calibration read into four 150 µs register-group reads, releasing the bus between chunks, or make the I²C transfer interrupt/DMA-driven with a queue so the mutex is held only while programming the transaction. Then:

R = 150 + 150 + 10 = 310 µs - a 69 % deadline utilisation, with room for ISR interference.

Do both: real mutex first (it removes the unbounded term immediately), then reduce the critical section to restore margin.

3. How would you verify the fix and prevent regression?

Read the worked answer

Measure, do not reason from source code.

Instrument the real timeline. Toggle a GPIO at Control task entry and exit and capture with a logic analyzer alongside the Sensor lock/unlock GPIOs. Run until you have captured at least 100 contention events. The maximum observed exit-to-entry interval is the empirical R; compare against 760 µs (inheritance only) and 310 µs (after restructuring).
Deliberately create the worst case. Temporarily extend Logger's execution to 3 ms and Sensor's critical section to 1 ms so contention happens constantly. A design that only survives benign traffic has not been proven. Also run at maximum CPU load and maximum bus traffic simultaneously.
Check the interrupt side. Measure the longest ISR and the longest interrupt-disabled region in the I²C driver and the flash/SysTick paths, then add them to the response-time budget explicitly. Add them to the documented budget, not to tribal knowledge.
Static checks in CI. Grep for xSemaphoreCreateBinary/xSemaphoreCreateCounting used where a lock is intended and fail the build; require every shared-resource handle to be declared with its ceiling priority. Enable configUSE_MUTEXES, and enable stack-overflow and task-runtime-stats hooks.
Runtime guard. Add a deadline-monitor task that counts Control-task overruns and latches the priority of the task that was running when the miss occurred. This turns a rare field event into a diagnosable record, and the overrun counter becomes a release gate.