Tuesday, 25 July 2023

`AttributeError: 'Tensor' object has no attribute 'numpy'` tracking intermediate values in custom call method

Basically I have a custom call method that computes some interesting intermediate values that are important enough to track when training, but not important enough to provide as an additional return value. My model looks something like

class MyModel(tf.keras.Model):
    def __init__(self, *args, **kwargs):
        self.intermediate_values: List[numpy.ndarray] = []
        super().__init__(*args, **kwargs)


    def call(self, inputs, training=False):

        # Do some linear algebra with inputs and different layers etc.

        intermediate_tensor = some_intermediate_result

        if training and some_other_condition:
            self.intermediate_values.append(intermediate_tensor.numpy())

        # Do some more stuff with intermediate_tensor and different layers etc.

        return final_result

This of course throws AttributeError: 'Tensor' object has no attribute 'numpy' since, as I understand it, evaluating to numpy is only applicable to "eager tensors" and here intermediate_tensor is technically a node in the model call graph(?or whatever the correct nomenclature is).

Directly storing a copy of intermediate_tensor also of course doesn't solve the problem since its still not an eager tensor.

A callback won't work since this in right in the middle of the call method.

Maybe a custom metric class that gets set/managed entirely internally in the MyModel class?

I just can't seem to figure out how to actually extract the values for later analysis when the graph is executed.



from `AttributeError: 'Tensor' object has no attribute 'numpy'` tracking intermediate values in custom call method

No comments:

Post a Comment