A model that I am working on should be predicting quite a lot of variables simultaneously (>1000). Therefore I would like to have a small neural network at the end of the network for each output.
In order to do this compactly, I would like to find a way to create a sparse trainable connection between two layers in the neural network within the Tensorflow framework.
Only a small portion of the connection matrix should be trainable: It is only the parameters that are part of the block-diagonal.
For example: 
The connection matrix is the following:
The trainable parameters should be in the place of the 1's.
from Tensorflow custom layer: Creating a sparse matrix with trainable parameters
No comments:
Post a Comment