NgbpddlmZddlZddlZddlmZddlmZmZddl m Z Gddej Z dS)) annotationsN) OrderedDict)Tensornn)import_from_stringcVeZdZddfd Zdd Zdd Zd ZddZedZ xZ S)AsymT sub_modulesdict[str, list[nn.Module]]allow_empty_keyboolc@||_||_t}|D]J\}}t |t s|g}t |D]\}}|||dzt|z<Kt |dS)a This model allows to create asymmetric SentenceTransformer models, that apply different models depending on the specified input key. In the below example, we create two different Dense models for 'query' and 'doc'. Text that is passed as {'query': 'My query'} will be passed along along the first Dense model, and text that will be passed as {'doc': 'My document'} will use the other Dense model. Note, that when you call encode(), that only inputs of the same type can be encoded. Mixed-Types cannot be encoded. Example:: word_embedding_model = models.Transformer(model_name) pooling_model = models.Pooling(word_embedding_model.get_word_embedding_dimension()) asym_model = models.Asym({'query': [models.Dense(word_embedding_model.get_word_embedding_dimension(), 128)], 'doc': [models.Dense(word_embedding_model.get_word_embedding_dimension(), 128)]}) model = SentenceTransformer(modules=[word_embedding_model, pooling_model, asym_model]) model.encode([{'query': 'Q1'}, {'query': 'Q2'}] model.encode([{'doc': 'Doc1'}, {'doc': 'Doc2'}] #You can train it with InputExample like this. Note, that the order must always be the same: train_example = InputExample(texts=[{'query': 'Train query'}, {'doc': 'Document'}], label=1) Args: sub_modules: Dict in the format str -> List[models]. The models in the specified list will be applied for input marked with the respective key. allow_empty_key: If true, inputs without a key can be processed. If false, an exception will be thrown if no key is specified. -N) r r ritems isinstancelist enumeratestrsuper__init__) selfr r ordered_dictnamemodelsidxmodel __class__s ]/var/www/html/ai-engine/env/lib/python3.11/site-packages/sentence_transformers/models/Asym.pyrz Asym.__init__ s:'."}} '--// < >C/$u++2FF). X&(,U (> H%%,,X6666  7 ,1133 # #OHek3x==AAJ K T 2 2 2 2 JJz " " " " "',,{M::C& Q Q Q UY I(!0#4d6J"K                       s=#E--E14E1texts!list[str] | list[tuple[str, str]]c t|dtstdd}|D]?}tt |\}}||}||ksJ@|j|dj|fi|S)z-Tokenizes a text and maps tokens to token-idsrzDAsym. model requires that texts are passed as dicts: {'key': 'text'}N)rdictAttributeErrornextiterrr tokenize)rrMkwargs module_keylookupr&texts rrTz Asym.tokenizecs%(D)) i !ghh h  * *F!$v||~~"6"677NHd!% z)))))7t +A.7HHHHHrcJttj|d5}t j|}dddn #1swxYwYi}|dD]L\}}t|}|tj||}|||<Mi}|dD]0\} } g|| <| D]#}|| ||$1t|fi|d} | S)Nr0r4r5r6) rCr>r?r@rDloadrrr=r ) input_pathfInconfigmodulesrJ model_type module_classmodulerIkey_name models_listrs rrZz Asym.loadrsv "',,z=99 : : $cYs^^F $ $ $ $ $ $ $ $ $ $ $ $ $ $ $$*7O$9$9$;$; ' ' Hj-j99L!&&rw||J'I'IJJF &GH  %+K%8%>%>%@%@ D D !Hk(*OH %' D D)001BCCCC D_==|(<== sAAA)T)r r r r )r r!)r(r))rMrN) r;r< __qualname__rr'r+rBrT staticmethodrZ __classcell__)rs@rr r s''''''''''''''R : I I I I\rr ) __future__rrDr> collectionsrtorchrrsentence_transformers.utilr Sequentialr rrrms"""""" ######999999xxxxx2=xxxxxr