Hi !
This article deal with Keyboard Interrupt and multi Processess in Python.
Context
I wanted to execute several parallel event processing loops in order to enjoy the power of Python Interpreter. Indeed, multiple threads in Python are executed by the same interpreter, sharing common parts using a Global Interpreter Lock, which means that the power computation is finally shared between the threads. This is not happening with different process. Each process has its own interpreter and can use its complete power computation (with the processor and OS limits). For that purpose, Python offers multiprocessing module as a replacement of threading module with similar tools.
I’ve been dealing with correctly handle Keyboard Interrupt in Python with multiprocessing. As I did not find proper answer to this problem in common Web resources, I found an alternative. Here is how I manage to do this.
Solution
To handle transmission of Keyboard Interrupt handler (Ctrl_C) from top process to its workers children, multiprocessing documentation and common web solutions indicate to intercept KeyboardInterrupt exception and to call the method terminate() to end child process. A SIGINT signal is sent to child process, which needs to be handled with the handler installed by signal.signal(signalnum, handler). However, this generates some tracebacks related to improper termination within multiprocessing code.
In order to exit child process before exiting the main process properly, I used “Event” from multiprocessing. An Event is created during the initialization of the child process. An Event is a flag set by default to False. As the initialization method of the child process is called by the main process, the new created event is shared by the main process as well as the child process. While the Event is not set (False), the child process can execute code. When the user press Ctrl+C, the main process manage this Keyboard Interrupt in its own signal handler and just set the Event (now it is True). When the child process see that the Event is set, it stops its work and terminate. The main process can join its subprocess and exit normally when they are finished.
As child process are a part of the same process group of the main process, they also receive Ctrl+C signal. We need to handle this and ignore it with signal.SIG_IGN.
Example
You can find an simple example of code below.
The main script :
import signal
def manage_ctrlC(*args):
# If you have multiple event processing processes, set each Event.
myProcess1.exit.set()
myProcess2.exit.set()
def toExecute1():
while True:
println("exec1");
def toExecute2():
while True:
println("exec2");
def main():
global myProcess1, myProcess2
# Init process (you can start multiple event processing processes)
myProcess1 = MyProcess(toExecute1)
myProcess1.start()
myProcess2 = MyProcess(toExecute2)
myProcess2.start()
myProcess1.join()
myProcess2.join()
# Manage Ctrl_C keyboard event
signal.signal(signal.SIGINT, manage_ctrlC)
main()
The Process Class :
from multiprocessing import *
import signal
class MyProcess(Process):
def __init__(self, toExecute):
# Call Mother Class Constructor
Process.__init__(self)
# Create shared Event for all process (__init__ is called in the main process)
self.exit = Event()
# Set method to execute
self.toExecute = toExecute
def run(self):
# Children process (in same process group as main process) will
# receive Ctrl-C signal, this is not our logical exit procedure
# (we use shared events tested in processing loops, and proper
# exit).
signal.signal(signal.SIGINT, signal.SIG_IGN)
while not self.exit.is_set():
self.toExecute()
print("Process exited")
I hope you enjoyed and it helped you.
I would like to thanks Laurent Pointal (https://perso.limsi.fr/pointal/accueil) for his help handling this problem.
Best