Tuesday, 15 March 2022

Stop the running instances when max_instances is reached

I'm using apscheduler-django and I created a task that loops every 10 seconds.

This function will make a request to an API and save the content to my database (PostgreSQL).

This is my task:

scheduler.add_job(
  SaveAPI,
  trigger=CronTrigger(second="*/10"), 
  id="SaveAPI", 
  max_instances=1,
  replace_existing=True,
)

and my SaveAPI is:

def SaveAPI():
    SPORT = 3
    print('API Queue Started')
    AllMatches = GetAllMatches(SPORT)
    for Match in AllMatches:
        AddToDatabase(Match, SPORT)
    print(f'API Queue Ended')

The GetAllMatches and AddToDatabase are too big and I don't think the implementations are relevant to my question.

My problem is sometimes I will get this error:

Run time of job "SaveAPI (trigger: cron[second='*/10'], next run at: 2022-03-05 23:21:00 +0330)" was missed by 0:00:11.445357

When this happens, it will not get replaced with a new instance because my SaveAPI function doesn't end. And apscheduler will always miss new instances.

I did many tests and function does not have any problem.

How can I make apscheduler stop the last running instance if a new instance is going to be missed?

So if my last instance takes more than 10 seconds, I want to just terminate the instance and create a new one.



from Stop the running instances when max_instances is reached

No comments:

Post a Comment