I am working on a machine language translation problem. The Model I am using is:
Model = Sequential([
Embedding(english_vocab_size, 256, input_length=english_max_len, mask_zero=True),
LSTM(256, activation='relu'),
RepeatVector(german_max_len),
LSTM(256, activation='relu', return_sequences=True),
Dense(german_vocab_size, activation='softmax')
])
Here,english_vocab_size
and english_max_len
are the total number of english words in the english vocabulory and number of words in each english sentence respectively. And the same is with german_vocab_size
and german_max_len
.
Now, how can I add tf.keras.layers.AdditiveAttention
layer in this Model?
Edit - I tried a lot to find good tutorials of implementing tf.keras.layers.AdditiveAttention
layer on an nlp task, but couldn't find any. So, I think if someone can explain how can I put the tf.keras.layers.AdditiveAttention
layer in this model, the person would be the first person to give a very clear explanation on how to use tf.keras.layers.AdditiveAttention
as it would be then very clear implementation on how to use the tf.keras.layers.AdditiveAttention
layer !
from How can I add tf.keras.layers.AdditiveAttention in my model?
No comments:
Post a Comment