Sunday 18 October 2020

How can I add tf.keras.layers.AdditiveAttention in my model?

I am working on a machine language translation problem. The Model I am using is:

    Model = Sequential([
          Embedding(english_vocab_size, 256, input_length=english_max_len, mask_zero=True),
          LSTM(256, activation='relu'),
          RepeatVector(german_max_len),
          LSTM(256, activation='relu', return_sequences=True),
          Dense(german_vocab_size, activation='softmax')
    ])

Here,english_vocab_size and english_max_len are the total number of english words in the english vocabulory and number of words in each english sentence respectively. And the same is with german_vocab_size and german_max_len.

Now, how can I add tf.keras.layers.AdditiveAttention layer in this Model?

Edit - I tried a lot to find good tutorials of implementing tf.keras.layers.AdditiveAttention layer on an nlp task, but couldn't find any. So, I think if someone can explain how can I put the tf.keras.layers.AdditiveAttention layer in this model, the person would be the first person to give a very clear explanation on how to use tf.keras.layers.AdditiveAttention as it would be then very clear implementation on how to use the tf.keras.layers.AdditiveAttention layer !



from How can I add tf.keras.layers.AdditiveAttention in my model?

No comments:

Post a Comment