Friday 20 November 2020

Speed up the initial TensorFlow startup

Each time I run a Python code using TensorFlow (CPU), such as:

import keras

I see this:

2020-10-30 15:27:20.518894: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found
2020-10-30 15:27:20.518894: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2020-10-30 15:27:23.713077: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'nvcuda.dll'; dlerror: nvcuda.dll not found
2020-10-30 15:27:23.713077: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: UNKNOWN ERROR (303)
2020-10-30 15:27:23.716077: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: User1-PC
2020-10-30 15:27:23.716077: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: User1-PC
2020-10-30 15:27:23.729078: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x10cad0c0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-10-30 15:27:23.729078: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
Using TensorFlow backend.

If I sum all the waiting time, there is ~ 10 seconds of waiting time.

Is there a way to speed up this process? Especially if I'm using TensorFlow for inference (and not training), I don't want to have to wait 10 seconds on each startup of the engine.


NB: Of course, when my code will be ready, I'll keep the process using TensorFlow constantly running, and I'll use some sort of inter-process communication, to avoid restart the whole program.

My question here is mostly for the prototyping stages, when you often need to restart the program: when prototyping, having to wait 10 or 15 seconds on each script start is highly unconvenient.



from Speed up the initial TensorFlow startup

No comments:

Post a Comment