Sunday, 23 September 2018

Converting a Sequence to Sequence Model from Keras to PyTorch

I was working with Sequence to Sequence models in Pytorch.

The Encoder convert a (batch_size X input_features X num_of_one_hot_encoded_classes) -> (batch_size X input_features X hidden_size)

The Decoder will take this input sequence and convert it into (batch_size X output_features X num_of_one_hot_encoded_classes)

An example would be like-

enter image description here

So on the above example, I would need to convert the 22 input features to 10 output features. In Keras it could be done with a RepeatVector(10).

An Example -

model.add(LSTM(256, input_shape=(22, 98)))
model.add(RepeatVector(10))
model.add(Dropout(0.3))
model.add(LSTM(256, return_sequences=True))

Although, I'm not sure if it's the proper way to convert the input sequences into the output ones.

So, my questions are -

  • What's the proper and fast way to convert the input sequences to output ones. eg. converting from (batch_size, 22, 98) -> (batch_size, 10, 98)
  • What will be the Pytorch implementation of the above Keras code?

Encoder Code snippet (Written in Pytorch) -

class EncoderRNN(nn.Module):
    def __init__(self, input_size, hidden_size):
        super(EncoderRNN, self).__init__()
        self.hidden_size = hidden_size
        self.lstm = nn.LSTM(input_size=input_size, hidden_size=hidden_size,
          num_layers=1, batch_first=True)

    def forward(self, input):
        output, hidden = self.lstm(input)
        return output, hidden



from Converting a Sequence to Sequence Model from Keras to PyTorch

No comments:

Post a Comment