Monday 2 August 2021

Why is model variables initialization method called twice in tensorflow?

I'm trying to figure out the implementation of SGD in tensorflow.

In the apply_gradients method

with ops.name_scope_v2(self._name):
  # Create iteration if necessary.
  with ops.init_scope():
    self._create_all_weights(var_list)

just before self._create_all_weights(var_list) I added some logging

with ops.name_scope_v2(self._name):
  # Create iteration if necessary.
  with ops.init_scope():
    tf.print('apply_gradients::with ops.init_scope: ', var_list[0].numpy(), var_list[1].numpy())
    self._create_all_weights(var_list)

and then I got the initialized weights and bias twice, which I don't understand. Why is this method called twice?

To verify my guess, I disabled all my logging except

  def apply_gradients(self,
                      grads_and_vars,
                      name=None,
                      experimental_aggregate_gradients=True):
  tf.print('apply_gradients is called')

each time I ran my script I got only one of that logging

apply_gradients is called

Why is weights and bias initialization method called twice in tensorflow?



from Why is model variables initialization method called twice in tensorflow?

No comments:

Post a Comment