Why is Multi-agent Deep Deterministic Policy Gradient (MADDPG) running slowly and taking only 22% from the GPU?

I tried to run the Distributed Multi-Agent Cooperation Algorithm based on MADDPG with prioritized batch data code with increasing the number of agents to be 12 agents but it takes a lot of times to train 3500 episodes. I have tried different setting and I'm using:

Cuda 10.1

   # MADDPG 
    gpu_options = tf.GPUOptions(allow_growth=True,per_process_gpu_memory_fraction=0.7)

    # config=tf.ConfigProto(gpu_options=gpu_options, log_device_placement=False)
    with tf.Session(config=tf.ConfigProto(gpu_options=gpu_options, log_device_placement=True)) as sess:

and the GPU takes only 22% but the CPU takes 100%

I'm working on :

Operating System: Windows 10 Pro 64-bit (10.0, Build 18363) (18362.19h1_release.190318-1202) Processor: Intel(R) Core(TM) i7-9700 CPU @ 3.00GHz (8 CPUs), ~3.0GHz Memory: 32768MB RAM

The parameters of the MADDPG:

        parser = argparse.ArgumentParser(description='provide arguments for DDPG agent')

        # agent parameters
        parser.add_argument('--actor-lr', help='actor network learning rate', default=0.001)
        parser.add_argument('--critic-lr', help='critic network learning rate', default=0.001)
        parser.add_argument('--gamma', help='discount factor for critic updates', default=0.99)
        parser.add_argument('--tau', help='soft target update parameter', default=0.01)
        parser.add_argument('--buffer-size', help='max size of the replay buffer', default=1000000)
        parser.add_argument('--minibatch-size', help='size of minibatch for minibatch-SGD', default=128)

        # run parameters
        #parser.add_argument('--env', help='choose the gym env- tested on {Pendulum-v0}', default='MountainCarContinuous-v0')
        parser.add_argument('--random-seed', help='random seed for repeatability', default=1234)
        parser.add_argument('--max-episodes', help='max num of episodes to do while training', default=3500)
        parser.add_argument('--max-episode-len', help='max length of 1 episode', default=100)
        parser.add_argument('--render-env', help='render the gym env', action='store_true')
        parser.add_argument('--use-gym-monitor', help='record gym results', action='store_true')
        parser.add_argument('--monitor-dir', help='directory for storing gym results', default='./results/videos/video1')
        parser.add_argument('--summary-dir', help='directory for storing tensorboard info', default='./results/4vs2/tfdata_maddpg/')
        parser.add_argument('--modelFolder', help='the folder which saved model data', default="./results/4vs2/weights_proposed/")
        parser.add_argument('--runTest', help='use saved model to run', default=False)
        parser.add_argument('--m-size', help='M size', default=128)
        parser.add_argument('--n-size', help='N size', default=64)

        parser.set_defaults(render_env=False)
        parser.set_defaults(use_gym_monitor=True)

        args = vars(parser.parse_args())

I also tried ubuntu 18.14 and when I run to check the GPU

 from __future__ import absolute_import, division, print_function, unicode_literals

    import tensorflow as tf
    print("Num GPUs Available: ", len(tf.config.experimental.list_physical_devices('GPU')))
    tf.debugging.set_log_device_placement(True)

    # Create some tensors
    a = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
    b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])
    c = tf.matmul(a, b)

    print(c)



    from tensorflow.python.client import device_lib

    print("g",device_lib.list_local_devices())

I got

Num GPUs Available:  0
2020-01-21 12:22:14.368278: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
Tensor("MatMul:0", shape=(2, 2), dtype=float32)
2020-01-21 12:22:14.481498: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-01-21 12:22:14.481901: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x44a1750 executing computations on platform CUDA. Devices:
2020-01-21 12:22:14.481917: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): GeForce RTX 2080 Ti, Compute Capability 7.5
g [name: "/device:CPU:0"
device_type: "CPU"
memory_limit: 268435456
locality {
}
incarnation: 16981832204773765455
, name: "/device:XLA_GPU:0"
device_type: "XLA_GPU"
memory_limit: 17179869184
locality {
}
incarnation: 10632369433792260465
physical_device_desc: "device: XLA_GPU device"
, name: "/device:XLA_CPU:0"
device_type: "XLA_CPU"
memory_limit: 17179869184
locality {
}
incarnation: 11745586653788522436
physical_device_desc: "device: XLA_CPU device"
]

from Why is Multi-agent Deep Deterministic Policy Gradient (MADDPG) running slowly and taking only 22% from the GPU?