PCPP2: Interprocess communication: threading, multiprocessing, subprocess

Interprocess communication

This domain covers threading, multiprocessing, and subprocess, and expects you to pick the right concurrency model and synchronise shared state correctly.

Rules. The GIL lets only one thread execute Python bytecode at a time, so threads help I/O-bound work while multiprocessing is required for CPU-bound parallelism. threading.Thread(target=..., args=...), then start(), then join(). Shared mutable state needs Lock/RLock, Event, or Queue; queue.Queue is thread-only, multiprocessing.Queue is pickled across processes. multiprocessing.Pool or concurrent.futures.ProcessPoolExecutor map work over workers; Manager proxies shared objects, Value/Array share memory. On spawn-based platforms the entry point must be guarded with if __name__ == "__main__":.

subprocess. subprocess.run(cmd, capture_output=True, text=True, check=True, timeout=5) returns a CompletedProcess with .returncode, .stdout, .stderr; check=True raises CalledProcessError on a non-zero exit. Popen suits streaming or long-running children - call communicate() to avoid pipe deadlock.

Worked example

r = subprocess.run(["git", "rev-parse", "HEAD"],
                   capture_output=True, text=True, check=True)
sha = r.stdout.strip()

Arguments go in a list, so no shell parsing and no injection. With text=True stdout is str, not bytes. If the repository is missing, check=True raises rather than returning a silently empty string.

Traps

• capture_output=True combined with an explicit stdout= raises ValueError.
• shell=True with an interpolated string is an injection risk.
• Starting a process without join() lets it outlive the parent.
• Assuming multiprocessing.Queue.get() returns - pass timeout for safety.
• Sharing a plain list between processes does not propagate mutations.