Monday, 11 September 2023

Huey Monitor - Error occurred sending signal "executing" - database kills the connection

In a Django 4.1 project I have huey (2.4.5) with django-huey (1.1.1) and django-huey-monitor (0.7.1) set up. Redis broker. 5 queues, including default, and are run in docker containers:

PYTHONBUFFERED=1 python manage.py djangohuey --queue queue_name --no-periodic --force-color

The monitor is initialised as such in tasks/__init__.py:

for queue_name in settings.DJANGO_HUEY["queues"].keys():
    signal(queue=queue_name)(store_signals)
    on_startup(queue=queue_name)(startup_handler)

Since adding django-huey-monitor Sentry has started to catch exceptions in @db_task tasks

Error occurred sending signal "executing"

OperationalError
server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.

Now, I've experienced something like that in the past (5+ years ago) and usually, it helped to add connection.close() at the end of the task imported from django.db thous @db_task should do it anyway. Now everything was running smoothly, @db_task and all, until I added signal monitoring. I tested one @db_task that should be closing the connection anyway, added the connection.close() - that one seems to go silent but another one started acting up.

I'd like to solve it globally and have run out of ideas.

Are the signals misconfigured?



from Huey Monitor - Error occurred sending signal "executing" - database kills the connection

No comments:

Post a Comment