Basically I have a custom call method that computes some interesting intermediate values that are important enough to track when training, but not important enough to provide as an additional return value. My model looks something like
class MyModel(tf.keras.Model):
def __init__(self, *args, **kwargs):
self.intermediate_values: List[numpy.ndarray] = []
super().__init__(*args, **kwargs)
def call(self, inputs, training=False):
# Do some linear algebra with inputs and different layers etc.
intermediate_tensor = some_intermediate_result
if training and some_other_condition:
self.intermediate_values.append(intermediate_tensor.numpy())
# Do some more stuff with intermediate_tensor and different layers etc.
return final_result
This of course throws AttributeError: 'Tensor' object has no attribute 'numpy'
since, as I understand it, evaluating to numpy is only applicable to "eager tensors" and here intermediate_tensor
is technically a node in the model call graph(?or whatever the correct nomenclature is).
Directly storing a copy of intermediate_tensor
also of course doesn't solve the problem since its still not an eager tensor.
A callback won't work since this in right in the middle of the call
method.
Maybe a custom metric class that gets set/managed entirely internally in the MyModel
class?
I just can't seem to figure out how to actually extract the values for later analysis when the graph is executed.
from `AttributeError: 'Tensor' object has no attribute 'numpy'` tracking intermediate values in custom call method
No comments:
Post a Comment