Performing background tasks in uWSGI by Vadym Yusanenko

With complicated web applications and a lot of microservices it is often a necessity to execute heavy long running tasks occasionally with minimal impact on system performance or request throughput. A lot of solutions exists to address this task but are there simpler alternatives to complex solutions like Celery? When dealing with task execution in a non-distributed system we can rely on fast and efficient uwsgi Spooler system as viable and safe option.

With this topic you'll discover what uwsgi Spooler is and what it can achieve. We will go through such steps:

Look at sub-optimal solutions when dealing with background tasks and their drawbacks in details
Configure uWSGI Spooler subsystem
Set directories for different spooler groups
Configure spooler processes and task priorities
Familiarize with different configuration options: init logic, recycling, timeouts
Use basic inter-process communication subsystem to see progress between processes