Gunicorn and haven't read the manual, I figured I'd let you know how your code can more gracefully handle timeouts. Gunicorn timeouts are signaled using the Unix SIGABRT signal. Gunicorn provides an option called graceful-timeout which allows you to have it send a second timeout signal using SIGTERM. This means you can trap the SIGABRT and have a few precious seconds to try and more gracefully handle a timeout rather than just abandoning mid routine to return a generic timeout error.
First, the config. While there are a couple ways to configure Gunicorn, this is how you'd configure it using command line arguments:
# Send SIGABRT at 5 seconds, send SIGTERM at 7 seconds
gunicorn --timeout 5 --graceful-timeout 7 …
# Send SIGABRT at 5 seconds followed by SIGKILL 2 seconds later
gunicorn --timeout 5 …
When it comes to writing Python code to use this config, here's an exception based system to handle graceful exits. First, we define a new interrupt object we can raise and a helper method to raise it.
import signal
import types
class GracefulExit(KeyboardInterrupt):
"""
An interrupt class to allow subroutines to try-except a block of code
that should have a chance to perform logic in the event of a graceful
exit signal (like keyboard interrupt or trapped Unix signal).
Properties:
signum: int
The Unix signal number trapped.
frame: frame
The python stack frame at the moment of interrupt.
"""
def __init__(self, signum: int, frame: types.FrameType):
"""
Store the values for handlers.
Arguments:
signum: int
The Unix signal number trapped.
frame: frame
Python stack frame of execution.
"""
super().__init__(f"{signal.Signals(signum).name} received.")
self.signum = signum
self.frame = frame
@staticmethod
def throw(signum: int, frame: types.FrameType):
"""
Pass to signal.signal() to raise a GracefulExit exception. Eg:
signal.signal(signal.SIGTERM, GracefulExit.throw)
Arguments:
signum: int
The Unix signal number trapped.
frame: frame
Python stack frame of execution.
Raises:
GracefulExit
This is the point…
"""
raise GracefulExit(signum, frame)
Next, when initializing everything we also need to register our handler for the signal:
signal.signal(signal.SIGABRT, GracefulExit.throw)
Finally, whenever we have a sensitive section of code, we can use a try-except block to provide a graceful exit strategy. Our interrupt class inherits from KeyboardInterrupt
which provides us three options:
KeyboardInterrupt
which will also handle our new interrupt;GracefulExit
to ignore KeyboardInterrupt
issues; ortry:
… timeout code …
except GracefulExit:
… clean up code …
In case the inheritance from KeyboardInterrupt
doesn't make sense, you should know that pressing Ctrl+D while using a POSIX terminal sends the current process a SIGINT. This includes things running in CPython which by default raises a KeyboardInterrupt
when this happens.