This page has two scripts
When should one use 1st method shown below vs 2nd? As nli-distilroberta-base-v2 trained specially for finding sentence embedding wont that always be better than the first method?
training_stsbenchmark.py1 -
from sentence_transformers import SentenceTransformer, LoggingHandler, losses, models, util
#You can specify any huggingface/transformers pre-trained model here, for example, bert-base-uncased, roberta-base, xlm-roberta-base
model_name = sys.argv[1] if len(sys.argv) > 1 else 'distilbert-base-uncased'
# Use Huggingface/transformers model (like BERT, RoBERTa, XLNet, XLM-R) for mapping tokens to embeddings
word_embedding_model = models.Transformer(model_name)
# Apply mean pooling to get one fixed sized sentence vector
pooling_model = models.Pooling(word_embedding_model.get_word_embedding_dimension(),
pooling_mode_mean_tokens=True,
pooling_mode_cls_token=False,
pooling_mode_max_tokens=False)
model = SentenceTransformer(modules=[word_embedding_model, pooling_model])
training_stsbenchmark_continue_training.py -
from sentence_transformers import SentenceTransformer, LoggingHandler, losses, util, InputExample
model_name = 'nli-distilroberta-base-v2'
model = SentenceTransformer(model_name)
from sentence transformer using huggingface/transformers pre-trained model vs SentenceTransformer
No comments:
Post a Comment