§ ÑìNg|ãóp—ddlmZddlmZddlmZmZddlmZddl m Z m Z Gd„dej ¦«Z d S) é)Ú annotations)ÚIterable)ÚTensorÚnn)ÚSentenceTransformeré)ÚBatchHardTripletLossÚ$BatchHardTripletLossDistanceFunctioncóX‡—eZdZejdfdˆfd„ Zdd „Zdd„Zedd„¦«Z ˆxZ S)ÚBatchAllTripletLosséÚmodelrÚmarginÚfloatÚreturnÚNonecór•—t¦« ¦«||_||_||_dS)a¦ BatchAllTripletLoss takes a batch with (sentence, label) pairs and computes the loss for all possible, valid triplets, i.e., anchor and positive must have the same label, anchor and negative a different label. The labels must be integers, with same label indicating sentences from the same class. Your train dataset must contain at least 2 examples per label class. Args: model: SentenceTransformer model distance_metric: Function that returns a distance between two embeddings. The class SiameseDistanceMetric contains pre-defined metrics that can be used. margin: Negative samples should be at least margin further apart from the anchor than the positive. References: * Source: https://github.com/NegatioN/OnlineMiningTripletLoss/blob/master/online_triplet_loss/losses.py * Paper: In Defense of the Triplet Loss for Person Re-Identification, https://arxiv.org/abs/1703.07737 * Blog post: https://omoindrot.github.io/triplet-loss Requirements: 1. Each sentence must be labeled with a class. 2. Your dataset must contain at least 2 examples per labels class. Inputs: +------------------+--------+ | Texts | Labels | +==================+========+ | single sentences | class | +------------------+--------+ Recommendations: - Use ``BatchSamplers.GROUP_BY_LABEL`` (:class:`docs `) to ensure that each batch contains 2+ examples per label class. Relations: * :class:`BatchHardTripletLoss` uses only the hardest positive and negative samples, rather than all possible, valid triplets. * :class:`BatchHardSoftMarginTripletLoss` uses only the hardest positive and negative samples, rather than all possible, valid triplets. Also, it does not require setting a margin. * :class:`BatchSemiHardTripletLoss` uses only semi-hard triplets, valid triplets, rather than all possible, valid triplets. Example: :: from sentence_transformers import SentenceTransformer, SentenceTransformerTrainer, losses from datasets import Dataset model = SentenceTransformer("microsoft/mpnet-base") # E.g. 0: sports, 1: economy, 2: politics train_dataset = Dataset.from_dict({ "sentence": [ "He played a great game.", "The stock is up 20%", "They won 2-1.", "The last goal was amazing.", "They all voted against the bill.", ], "label": [0, 1, 0, 0, 2], }) loss = losses.BatchAllTripletLoss(model) trainer = SentenceTransformerTrainer( model=model, train_dataset=train_dataset, loss=loss, ) trainer.train() N)ÚsuperÚ__init__Úsentence_embedderÚtriplet_marginÚdistance_metric)ÚselfrrrÚ __class__s €úl/var/www/html/ai-engine/env/lib/python3.11/site-packages/sentence_transformers/losses/BatchAllTripletLoss.pyrzBatchAllTripletLoss.__init__ s:ø€õT ‰Œ×ÒÑÔÐØ!&ˆÔØ$ˆÔØ.ˆÔÐÐóÚsentence_featuresúIterable[dict[str, Tensor]]Úlabelsrcóp—| |d¦«d}| ||¦«S)NrÚsentence_embedding)rÚbatch_all_triplet_loss)rrrÚreps rÚforwardzBatchAllTripletLoss.forward\s7€Ø×$Ò$Ð%6°qÔ%9Ñ:Ô:Ð;OÔPˆØ×*Ò*¨6°3Ñ7Ô7Ð7rÚ embeddingscó|—| |¦«}| d¦«}| d¦«}||z |jz}tj|¦«}| ¦«|z}d||dk<||dk}| d¦«} | ¦«| dzz }|S)a]Build the triplet loss over a batch of embeddings. We generate all the valid triplets and average the loss over the positive ones. Args: labels: labels of the batch, of size (batch_size,) embeddings: tensor of shape (batch_size, embed_dim) margin: margin for triplet loss squared: Boolean. If true, output is the pairwise squared euclidean distance matrix. If false, output is the pairwise euclidean distance matrix. Returns: Label_Sentence_Triplet: scalar tensor containing the triplet loss érrg¼‰Ø—²Òœ<)rÚ unsqueezerr Úget_triplet_maskrÚsizeÚsum) rrr%Ú pairwise_distÚanchor_positive_distÚanchor_negative_distÚ triplet_lossÚmaskÚvalid_tripletsÚnum_positive_tripletss rr"z*BatchAllTripletLoss.batch_all_triplet_loss`sÒ€ð×,Ò,¨ZÑ8Ô8ˆ à,×6Ò6°qÑ9Ô9ÐØ,×6Ò6°qÑ9Ô9Ðð ,Ð.BÑBÀTÔEXÑXˆ õ$Ô4°VÑ<Ô<ˆØ—z’z‘|”| lÑ2ˆ ð*+ˆ \ AÒ%Ñ&ð& l°UÒ&:Ô;ˆØ .× 3Ò 3°AÑ 6Ô 6Ðð $×'Ò'Ñ)Ô)Ð-BÀUÑ-JÑKˆ àÐrÚstrcó—dS)Na @misc{hermans2017defense, title={In Defense of the Triplet Loss for Person Re-Identification}, author={Alexander Hermans and Lucas Beyer and Bastian Leibe}, year={2017}, eprint={1703.07737}, archivePrefix={arXiv}, primaryClass={cs.CV} } ©)rs rÚcitationzBatchAllTripletLoss.citation‹s €ð ð r)rrrrrr)rrrrrr)rrr%rrr)rr3) Ú__name__Ú __module__Ú __qualname__r Úeucledian_distancerr$r"Úpropertyr6Ú __classcell__)rs@rr r sžø€€€€€ð=ÔOØð M/ðM/ðM/ðM/ðM/ðM/ðM/ð^8ð8ð8ð8ð)ð)ð)ð)ðVð ð ð ñ„Xð ð ð ð ð rr N) Ú __future__rÚcollections.abcrÚtorchrrÚ)sentence_transformers.SentenceTransformerrr r ÚModuler r5rrúrBs«ðØ"Ð"Ð"Ð"Ð"Ð"à$Ð$Ð$Ð$Ð$Ð$àÐÐÐÐÐÐÐàIÐIÐIÐIÐIÐIà\Ð\Ð\Ð\Ð\Ð\Ð\Ð\ðJðJðJðJðJ˜"œ)ñJôJðJðJðJr