NghddlmZddlmZddlZddlmZmZddlmZGddej Z dS)) annotations)IterableN)Tensornn)SentenceTransformerc@eZdZd fd Zdd Zedd ZxZS)MSELossmodelrreturnNonect||_tj|_dS)ug Computes the MSE loss between the computed sentence embedding and a target sentence embedding. This loss is used when extending sentence embeddings to new languages as described in our publication Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation. For an example, see `the distillation documentation <../../examples/training/distillation/README.html>`_ on extending language models to new languages. Args: model: SentenceTransformerModel References: - Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation: https://arxiv.org/abs/2004.09813 - `Training > Model Distillation <../../examples/training/distillation/README.html>`_ - `Training > Multilingual Models <../../examples/training/multilingual/README.html>`_ Requirements: 1. Usually uses a finetuned teacher M in a knowledge distillation setup Inputs: +-----------------------------------------+-----------------------------+ | Texts | Labels | +=========================================+=============================+ | sentence | model sentence embeddings | +-----------------------------------------+-----------------------------+ | sentence_1, sentence_2, ..., sentence_N | model sentence embeddings | +-----------------------------------------+-----------------------------+ Relations: - :class:`MarginMSELoss` is equivalent to this loss, but with a margin through a negative pair. Example: :: from sentence_transformers import SentenceTransformer, SentenceTransformerTrainer, losses from datasets import Dataset student_model = SentenceTransformer("microsoft/mpnet-base") teacher_model = SentenceTransformer("all-mpnet-base-v2") train_dataset = Dataset.from_dict({ "english": ["The first sentence", "The second sentence", "The third sentence", "The fourth sentence"], "french": ["La première phrase", "La deuxième phrase", "La troisième phrase", "La quatrième phrase"], }) def compute_labels(batch): return { "label": teacher_model.encode(batch["english"]) } train_dataset = train_dataset.map(compute_labels, batched=True) loss = losses.MSELoss(student_model) trainer = SentenceTransformerTrainer( model=student_model, train_dataset=train_dataset, loss=loss, ) trainer.train() N)super__init__r rr loss_fct)selfr __class__s `/var/www/html/ai-engine/env/lib/python3.11/site-packages/sentence_transformers/losses/MSELoss.pyrzMSELoss.__init__ s4v    sentence_featuresIterable[dict[str, Tensor]]labelsrcJt|dkrYtjfd|Dd}||t|dS|dd}||S)NcFg|]}|dS)sentence_embedding)r ).0inputsrs r z#MSELoss.forward..Ns,#m#m#mQWDJJv$6$67K$L#m#m#mrr)dimr)lentorchcatrrepeatr )rrr embeddingss` rforwardzMSELoss.forwardKs  ! !A % %#m#m#m#m[l#m#m#mstuuuJ==V]]3?P;Q;QST-U-UVV VZZ 1! 4556JK }}Z000rstrcdS)Na @inproceedings{reimers-2020-multilingual-sentence-bert, title = "Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2020", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/2004.09813", } )rs rcitationzMSELoss.citationUs   r)r rr r )rrrrr r)r r&)__name__ __module__ __qualname__rr%propertyr) __classcell__)rs@rr r sr=%=%=%=%=%=%~1111   X     rr ) __future__rcollections.abcrr!rrsentence_transformersrModuler r(rrr3s""""""$$$$$$ 555555VVVVVbiVVVVVr