How to monitor a filtered version of a metric in EarlyStopping callback in tensorflow?

I always had this problem. When training neural networks, the validation loss can be noisy (sometimes even the training loss if you are using stochastic layers such as dropout). This is especially true when the dataset is small.

This makes that when using callbacks such as EarlyStopping or ReduceLROnPlateau, these are triggered too early (even using large patience). Also, sometimes I don't want to use large patience in the ReduceLROnPLateau callback.

A solution to this is instead of directly monitoring a certain metric (e.g. val_loss), to monitor a filtered version (across epochs) of the metric (e.g. exponential moving average of val_loss). However, I do not see any easy way to solve this because the callbacks only accept metrics that not depend on the previous epochs. I have tried using a custom training loop to reproduce the functionality of these callbacks with my custom filtered metric, but I don't think it is the correct way. Is there another (simpler) way to do the monitor the filtered version of the loss in the callbacks, without reimplementing the whole functionality of the callbacks?

Edit:

This is what I mean by monitoring a filtered version of a metric. The current EarlyStopping works something like this:

best_loss = float('inf')
best_epoch = 0
for epoch in range(n_epochs):
    # ...
    new_loss = # compute loss of current epoch
    if new_loss < best_loss:
        best_loss = new_loss
        best_epoch = epoch
    if epoch - best_epoch > patience:
        break

Monitoring the filtered metric would be like this:

best_loss = float('inf')
filtered_loss = 10 # example initial value
best_epoch = 0
for epoch in range(n_epochs):
    # ...
    new_loss = # compute loss of current epoch
    filtered_loss = 0.1*new_loss + 0.9*filtered_loss
    if filtered_loss < best_loss:
        best_loss = filtered_loss
        best_epoch = epoch
    if epoch - best_epoch > patience:
        break

from How to monitor a filtered version of a metric in EarlyStopping callback in tensorflow?