I'm trying to figure out the implementation of SGD in tensorflow.
In the apply_gradients
method
with ops.name_scope_v2(self._name):
# Create iteration if necessary.
with ops.init_scope():
self._create_all_weights(var_list)
just before self._create_all_weights(var_list)
I added some logging
with ops.name_scope_v2(self._name):
# Create iteration if necessary.
with ops.init_scope():
tf.print('apply_gradients::with ops.init_scope: ', var_list[0].numpy(), var_list[1].numpy())
self._create_all_weights(var_list)
and then I got the initialized weights and bias twice, which I don't understand. Why is this method called twice?
To verify my guess, I disabled all my logging except
def apply_gradients(self,
grads_and_vars,
name=None,
experimental_aggregate_gradients=True):
tf.print('apply_gradients is called')
each time I ran my script I got only one of that logging
apply_gradients is called
Why is weights and bias initialization method called twice in tensorflow?
from Why is model variables initialization method called twice in tensorflow?
No comments:
Post a Comment