why TimedRotatingFileHandler does not delete old files?

I am using TimedRotatingFileHandler to create my logs. I want my log files to be created every minute, keep at most 2 log files and delete older ones. Here is the sample code:

import logging
import logging.handlers
import datetime

logger = logging.getLogger('MyLogger')
logger.setLevel(logging.DEBUG)

handler = logging.handlers.TimedRotatingFileHandler(
    "logs/{:%H-%M}.log".format(datetime.datetime.now()), 
    when="M", 
    backupCount=2)

logger.addHandler(handler)
logger.debug("PLEASE DELETE PREVIOUS FILES")

If I run this code multiple times (with a minute interval) I get multiple files in my logs directory like so:

21-01.log
21-02.log
21-03.log
...

This seems strange to me, since I set backupCount=2 which indicates that at most 2 files should be saved and older files should be deleted. However when I start my application with 2 files or more in the log folder, old files are not deleted.

Why TimedRotatingFileHandler does not delete old files? Is there any way I can set TimedRotatingFileHandler to delete older files?

from why TimedRotatingFileHandler does not delete old files?