Tuesday, 4 September 2018

Keras ImageDataGenerator Fit causes memory leak

I'm on Keras 2.2.2 and I'm trying to generate augmentations of my training data with zca_whitening and an ImageDataGenerator. But when I try to fit the generator (which is mandatory when using zca_whitening) the python process eats more and more memory (100Gb+) until it gets killed by the system.

This small example can cause the leak:

import numpy as np
from keras.preprocessing.image import ImageDataGenerator

def cause_leak():
    idg = ImageDataGenerator(zca_whitening = True)
    random_sample = np.random.random((1, 250, 250, 3))
    idg.fit(random_sample)

cause_leak()



from Keras ImageDataGenerator Fit causes memory leak

No comments:

Post a Comment