How can I make Keras Models fit method execute a generator in the main thread? From the docs, it looks like that setting workers=0 would execute the code in the main thread.
workers Integer. Used for generator or keras.utils.Sequence input only. Maximum number of processes to spin up when using process-based threading. If unspecified, workers will default to 1. If 0, will execute the generator on the main thread.
However when I do:
import tensorflow as tf
import threading
model = tf.keras.Sequential([tf.keras.layers.Dense(1)])
model.compile(loss = "mse", optimizer = "adam")
def gen ():
for i in range(100):
print(threading.current_thread())
yield (tf.random.normal(shape=(100,1)), tf.random.normal(shape = (100,)))
model.fit(gen(), epochs = 1, workers = 0, verbose = 0, steps_per_epoch = 3)
I get
<_MainThread(MainThread, started 140516450817920)>
<_DummyThread(Dummy-5, started daemon 140514709206784)>
<_DummyThread(Dummy-4, started daemon 140514717599488)>
<tensorflow.python.keras.callbacks.History at 0x7fcc1e8a8d68>
Which I interpret as only the first step in the iterator has been executed in the main thread.
In my use case this is problematic because I need that the code inside the generator to always be executed in the main thread otherwise the program crashes.
from Keras fit with generator function always execute in the main thread
No comments:
Post a Comment