The problem
A learner in an online forum wanted a script that keeps opening an external program (Notepad, in their case) in a loop. They wanted the loop to stop when a key is pressed. They used multiprocessing and os.system, and it did not behave. The script hung, and the stop key did nothing useful.
The replies spotted three separate bugs. We rebuilt each one on Linux with Python 3.13 so you can see them happen, then wrote a fix with tests.
-
Process(target=f(path))runsfright away, in the parent, not in a new process. -
os.system()starts the app through a shell and waits for it. Callingterminate()on the worker does not reach the app. - The key check ran once, before the loop even began, so it could never stop anything later.
Bug 1: you called the function instead of passing it
import os, time
from multiprocessing import Process
def work(n):
print("work() running in pid", os.getpid())
time.sleep(n)
p = Process(target=work(1)) # work(1) runs NOW, then returns None
Python must evaluate work(1) before it can hand the result to Process. So work runs in the parent and blocks it. When it returns, target is None. Our run printed the same pid for the parent and for work(). The Process(...) line took 1.0 seconds, and target was None.
Fix: pass the function and its arguments separately.
p = Process(target=work, args=(1,))
If the loop never ends, as in the original post, the parent stays stuck on that line forever. That is why it looked like the script "hangs".
Bug 2: terminate() kills the worker, not the app
Even with the target fixed, this still leaks:
import os, sys, time
from multiprocessing import Process
APP = f'"{sys.executable}" fake_app.py 60'
def run_forever():
while True:
os.system(APP) # shell -> app; blocks until the app exits
if __name__ == "__main__":
p = Process(target=run_forever)
p.start(); time.sleep(1.5)
p.terminate(); p.join()
In our test, the worker was gone after terminate(), but two processes were still running: the shell that os.system started, and the app under it. We also forced the "spawn" start method on Linux (the docs say it is the default on Windows and macOS). There, the script printed its last line but did not exit until the leftover app closed.
This matches the Python docs. os.system() runs the command "in a subshell" by calling C's system(). And Process.terminate() notes that descendant processes "will not be terminated – they will simply become orphaned." The docs also warn that terminate() skips finally blocks and exit handlers. It can also break a pipe, queue or lock the process was using.
Root cause: your code never holds a handle to the app. os.system hides it behind a shell and only gives you an exit status at the end. You can't signal something you can't see.
The fix: own the app with Popen, stop with an Event
Two changes do most of the work:
- Start the app with
subprocess.Popen([...]), using a list of arguments. There is no shell in between, and you get a handle withpoll(),terminate(),kill()andwait(). - Use an
Eventas the stop signal. The loop checks it often, and the loop itself closes the app. Nobody has to kill the loop from outside.
"""Relaunch an external app in a loop until asked to stop, then close it cleanly."""
import subprocess
import threading
def stop_app(proc, grace=3.0):
"""Ask the app to exit, wait a bit, then force it."""
if proc.poll() is not None:
return proc.returncode
proc.terminate() # SIGTERM on POSIX, TerminateProcess on Windows
try:
return proc.wait(timeout=grace)
except subprocess.TimeoutExpired:
proc.kill() # SIGKILL on POSIX
return proc.wait()
def run_app_loop(cmd, stop, max_runs=None, poll=0.2):
"""Start cmd, restart it each time it exits, until `stop` (an Event) is set."""
runs = 0
while not stop.is_set():
if max_runs is not None and runs >= max_runs:
break
proc = subprocess.Popen(cmd) # list of args, no shell in between
runs += 1
try:
while proc.poll() is None: # app still running
if stop.wait(poll): # sleeps up to `poll`, wakes early on stop
break
finally:
stop_app(proc) # runs on stop, Ctrl+C, or an exception
return runs
def watch_for_q(stop):
"""Portable key check: type q and press Enter."""
try:
while True:
if input().strip().lower() == "q":
break
except EOFError: # stdin closed: treat as stop too
pass
stop.set()
if __name__ == "__main__":
import sys
stop = threading.Event()
threading.Thread(target=watch_for_q, args=(stop,), daemon=True).start()
app = [sys.executable, "fake_app.py", "5"] # e.g. ["notepad.exe"] on Windows
print("running; type q + Enter to stop", flush=True)
print("app started", run_app_loop(app, stop), "times")
A few details matter here:
-
stop.wait(poll)works as a sleep that wakes early when the event is set. So the stop is fast, but the loop does not spin the CPU. -
stop_app()tries a polite stop first, then waits with a timeout, and only then forces it. Thefinallymakes sure the app is closed even on Ctrl+C or an error. - The key check runs in its own thread. It loops, so it checks every line you type, not just once. We used
input()(typeqand press Enter) because it works the same on every OS. Packages that read single key presses have their own platform rules, so check their docs before you use one.
Do you even need multiprocessing?
Often you don't. The external app is already a separate process. Your Python loop spends nearly all its time waiting, so one process with a small thread is enough, as above.
Save the code above as supervisor.py.
If the loop really must live in its own process (for example, a larger program starts it), share a multiprocessing.Event. The docs describe it as "a clone of threading.Event". Then ask the worker to stop instead of terminating it:
"""Same idea when the loop must live in its own process."""
import multiprocessing as mp
from supervisor import run_app_loop
def worker(cmd, stop):
run_app_loop(cmd, stop)
if __name__ == "__main__":
import sys, time
stop = mp.Event()
p = mp.Process(target=worker, args=([sys.executable, "fake_app.py", "60", "BB_MARK_MP"], stop))
p.start()
time.sleep(1.5)
stop.set() # ask, don't terminate()
p.join(timeout=10)
if p.is_alive(): # last resort only
p.terminate(); p.join()
print("worker exit code:", p.exitcode)
In our run, the worker exited with code 0 and no app process was left. terminate() stays only as a last resort, after a timeout.
Trade-offs to know
- Grace time vs. speed. A polite stop lets the app save or clean up. A force kill is instant but can lose data. We used 3 seconds. Pick what fits your app.
-
Windows is blunter. The docs say that on Windows,
Popen.terminate()callsTerminateProcess(), andkill()is just an alias for it. So there is no polite-then-force step there. If you need a graceful close on Windows, you need an app-specific method. We tested on Linux only. The Windows notes come from the docs, not from our runs. -
Apps that start their own children.
terminate()reaches only the process you started. In our test, a launcher that started a child left that child running afterterminate(). On POSIX, starting it withstart_new_session=Trueand then callingos.killpg(p.pid, signal.SIGTERM)cleared the whole group (0 left). That option is POSIX only. On Windows, look atcreationflagssuch asCREATE_NEW_PROCESS_GROUPand test it yourself. -
Pipes. If you add
stdout=PIPE, the docs warn thatwait()can deadlock when the app writes a lot of output. Usecommunicate(), or don't pipe output you don't read. -
Restart storms. If the app exits at once (a bad path, for example), the loop will relaunch it over and over. Add a short delay or a
max_runslimit. Also, log errors instead of a bareexcept: pass, which hid problems in the original code.
How to verify the fix
Don't trust "it looks closed". Count processes. We gave each test app a unique marker argument and counted matches with pgrep -f. These five tests all passed:
def alive(mark):
out = subprocess.run(["pgrep", "-f", mark], capture_output=True, text=True).stdout
return len(out.split())
def test_stop_kills_running_app_fast(self):
stop = threading.Event()
cmd = [sys.executable, "fake_app.py", "60", "BB_MARK_T1"]
th = threading.Thread(target=run_app_loop, args=(cmd, stop)); th.start()
time.sleep(1.0); self.assertEqual(alive("BB_MARK_T1"), 1)
t = time.time(); stop.set(); th.join(5)
self.assertFalse(th.is_alive()); self.assertLess(time.time() - t, 2)
self.assertEqual(alive("BB_MARK_T1"), 0)
-
Restarts: an app that exits at once was started exactly 3 times with
max_runs=3. -
Fast stop: a 60-second app was closed in under 2 seconds after
stop.set(), with 0 left. - Force kill: an app that ignores SIGTERM was killed after the 1-second grace time (return code -9).
-
Worker version: the
multiprocessingworker exited with code 0, with 0 app processes left. -
Key stop: typing
qstopped a 5-second app early, in under 2 seconds.
fake_app.py is just time.sleep(seconds), standing in for the real program. On Windows, swap pgrep for Task Manager or tasklist to do the same count.
Official sources
- Python docs, subprocess (Popen, terminate, kill, wait, start_new_session): https://docs.python.org/3/library/subprocess.html
- Python docs, multiprocessing (Process.terminate, Event): https://docs.python.org/3/library/multiprocessing.html
- Python docs, os.system and os.killpg: https://docs.python.org/3/library/os.html
ByteBridge Talent LLC is an IT staffing company. We help companies find and hire IT professionals. Website: https://www.bytebridgetalent.com · Email: info@bytebridgetalent.com
Top comments (1)
I ran your supervisor.py with the functions unchanged (Linux, Python 3.14.4, a 60-second fake_app) and tried an exit your five tests don't cover: the supervisor itself getting SIGTERM, for example
kill <pid>or a parent process callingterminate()on it.SIGINT (Ctrl+C) is fine: KeyboardInterrupt unwinds through the
finally,stop_app()runs, 0 apps left. SIGTERM is not. Python installs no handler for it, so the default action ends the process without running finally blocks: the supervisor exited with -15 and the fake_app was still running a second later, orphaned. Same leak as your Bug 2, one level up. The multiprocessing version hits it in the last-resort branch:p.terminate()on the worker gave exitcode -15 and left the app running.What fixed it in my runs:
With that, SIGTERM exited with 143 and 0 apps were left (calling
stop.set()from the handler instead also worked: exit 0, 0 left). SIGKILL can't be caught, sokill -9still orphaned the app. On Linux, launching it as["setpriv", "--pdeathsig", "TERM", "--", *cmd]has the kernel send TERM to the app when the supervisor dies: 0 left afterkill -9. On Windows a parent'sterminate()is TerminateProcess, which no Python handler sees; a Job Object with kill-on-close is the usual answer there, but I only ran Linux.Does anything besides the keyboard stop the script in your learner's setup, like a parent script or a scheduler?