Thursday, 18 November 2021

Efficient way to get "neuron-edge-neuron" values in a neural network

I'm working on a visual networks project where I'm trying to plot several node-edge-node values in an interactive graph.

I have several neural networks (this is one example):

import torch
import torch.nn as nn
import torch.optim as optim

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.fc1 = nn.Linear(1, 2)
        self.fc2 = nn.Linear(2, 3)
        self.fc3 = nn.Linear(3, 1)

    def forward(self, x):
        x1 = self.fc1(x)
        x = torch.relu(x1)
        x2 = self.fc2(x)
        x = torch.relu(x2)
        x3 = self.fc3(x)
        return x3, x2, x1

net = Model()

How can I get the node-edge-node (neuron-edge-neuron) values in the network in an efficient way? Some of these networks have a large number of parameters. Note that for the first layer it will be input-edge-neuron rather than neuron-edge-neuron.

I tried saving each node values after the fc layers (ie x1,x2,x3) so I won't have to recompute them, but I'm not sure how to do the edges and match them to their corresponding neurons in an efficient way.

The output I'm looking for is a list of lists of node-edge-node values. Though it can also be a tensor of tensors if it's easier. For example, in the above network from the first layer I will have 2 triples (1x2), from the 2nd layer I will have 6 of them (2x3), and in the last layer I will have 3 triples (3x1). The issue is matching nodes (ie neurons) values (one from layer n-1 and one from layer n) with the corresponding edges in an efficient way.



from Efficient way to get "neuron-edge-neuron" values in a neural network

No comments:

Post a Comment