NgBdZddlmZddlmZmZmZmZddlZ ddl m Z m Z m Z ddlmZddlmZddlmZmZdd lmZdd lmZmZdd lmZd efd ZGddeeZGddeZGddeeZ GddeeZ!dS)z@A chain for comparing the output of two models using embeddings.)Enum)AnyDictListOptionalN)AsyncCallbackManagerForChainRunCallbackManagerForChainRun Callbacks) Embeddings)pre_init) ConfigDictField)Chain)PairwiseStringEvaluatorStringEvaluatorRUN_KEYreturnc ddlm}n5#t$r( ddlm}n#t$rtdwxYwYnwxYw|S)zaCreate an Embeddings object. Returns: Embeddings: The created Embeddings object. rOpenAIEmbeddingstCould not import OpenAIEmbeddings. Please install the OpenAIEmbeddings package using `pip install langchain-openai`.)langchain_openair ImportError%langchain_community.embeddings.openairs h/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain/evaluation/embedding_distance/base.py_embedding_factoryrs 5555555   N N N N N N N   Q   O N    s ;;5;;c&eZdZdZdZdZdZdZdZdS)EmbeddingDistancea Embedding Distance Metric. Attributes: COSINE: Cosine distance metric. EUCLIDEAN: Euclidean distance metric. MANHATTAN: Manhattan distance metric. CHEBYSHEV: Chebyshev distance metric. HAMMING: Hamming distance metric. cosine euclidean manhattan chebyshevhammingN) __name__ __module__ __qualname____doc__COSINE EUCLIDEAN MANHATTAN CHEBYSHEVHAMMINGrrr*s3FIIIGGGr/rcfeZdZUdZeeZeed<ee j Z e ed<e de eefde eeffdZed Zedeefd Zd edefd Zde defdZedejdejdejfdZedejdejdejfdZedejdejdejfdZedejdejdejfdZ edejdejdejfdZ!dejde"fdZ#dS)_EmbeddingDistanceChainMixina0Shared functionality for embedding distance evaluators. Attributes: embeddings (Embeddings): The embedding objects to vectorize the outputs. distance_metric (EmbeddingDistance): The distance metric to use for comparing the embeddings. )default_factory embeddings)defaultdistance_metricvaluesrc|d}g} ddlm}||n#t$rYnwxYw ddlm}||n#t$rYnwxYw|st dt |t|r# ddl}n#t$rt dwxYw|S)zValidate that the TikTok library is installed. Args: values (Dict[str, Any]): The values to validate. Returns: Dict[str, Any]: The validated values. r3rrrNzThe tiktoken library is required to use the default OpenAI embeddings with embedding distance evaluators. Please either manually select a different Embeddings object or install tiktoken using `pip install tiktoken`.) getrrappendrr isinstancetupletiktoken)clsr6r3types_rr<s r_validate_tiktoken_installedz9_EmbeddingDistanceChainMixin._validate_tiktoken_installedHs:ZZ --   9 9 9 9 9 9 MM* + + + +    D   N N N N N N MM* + + + +    D  Q  j%-- 0 0     !I  s-5 AAA"" A/.A/!B&&CT)arbitrary_types_allowedcdgS)zgReturn the output keys of the chain. Returns: List[str]: The output keys. scorer.selfs r output_keysz(_EmbeddingDistanceChainMixin.output_keysxs yr/resultcVd|di}t|vr|t|t<|S)NrBr)rDrFparseds r_prepare_outputz,_EmbeddingDistanceChainMixin._prepare_outputs-6'?+ f  $WoF7O r/metricc tj|jtj|jtj|jtj|jtj |j i}||vr||Std|)zGet the metric function for the given metric name. Args: metric (EmbeddingDistance): The metric name. Returns: Any: The metric function. zInvalid metric: ) rr)_cosine_distancer*_euclidean_distancer+_manhattan_distancer,_chebyshev_distancer-_hamming_distance ValueError)rDrJmetricss r _get_metricz(_EmbeddingDistanceChainMixin._get_metricsp  $d&;  ')A  ')A  ')A  %t'=   W  6? "88899 9r/abcj ddlm}n#t$rtdwxYwd|||z S)zCompute the cosine distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.ndarray: The cosine distance. r)cosine_similarityzThe cosine_similarity function is required to compute cosine distance. Please install the langchain-community package using `pip install langchain-community`.g?)langchain_community.utils.mathrWr)rTrUrWs rrLz-_EmbeddingDistanceChainMixin._cosine_distancesi  H H H H H H H   6   &&q!,,,,s #cFtj||z S)zCompute the Euclidean distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Euclidean distance. )nplinalgnormrTrUs rrMz0_EmbeddingDistanceChainMixin._euclidean_distancesy~~a!e$$$r/cTtjtj||z S)zCompute the Manhattan distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Manhattan distance. )rZsumabsr]s rrNz0_EmbeddingDistanceChainMixin._manhattan_distance vbfQUmm$$$r/cTtjtj||z S)zCompute the Chebyshev distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Chebyshev distance. )rZmaxr`r]s rrOz0_EmbeddingDistanceChainMixin._chebyshev_distancerar/c2tj||kS)zCompute the Hamming distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Hamming distance. )rZmeanr]s rrPz._EmbeddingDistanceChainMixin._hamming_distanceswqAvr/vectorsc||j}||ddd|ddd}|S)zCompute the score based on the distance metric. Args: vectors (np.ndarray): The input vectors. Returns: float: The computed score. r)rSr5reshapeitem)rDrfrJrBs r_compute_scorez+_EmbeddingDistanceChainMixin._compute_scoresd!!$"677wqz))!R00'!*2D2DQ2K2KLLQQSS r/N)$r%r&r'r(rrr3r __annotations__rr)r5r rstrrr?r model_configpropertyrrEdictrIrS staticmethodrZndarrayrLfloatingrMrNrOrPfloatrlr.r/rr1r1<su#U3EFFFJ FFF).7H7O)P)P)PO&PPP )$sCx.)T#s(^)))X)V: $LT#YXdt :"3:::::*-BJ-2:-"*---\-( %rz %bj %R[ % % %\ % %rz %bj %R[ % % %\ % %rz %bj %R[ % % %\ % RZ BJ 2;   \  bj U      r/r1ceZdZdZedefdZedefdZede efdZ dde ee fde ede ee ffd Z dde ee fde ede ee ffd Zddddd d d ede edede e ede e ee fdede defdZddddd d d ede edede e ede e ee fdede defdZdS)EmbeddingDistanceEvalChaina"Use embedding distances to score semantic difference between a prediction and reference. Examples: >>> chain = EmbeddingDistanceEvalChain() >>> result = chain.evaluate_strings(prediction="Hello", reference="Hi") >>> print(result) {'score': 0.5} rcdS)zReturn whether the chain requires a reference. Returns: bool: True if a reference is required, False otherwise. Tr.rCs rrequires_referencez-EmbeddingDistanceEvalChain.requires_references tr/c"d|jjdS)N embedding_ _distancer5valuerCs revaluation_namez*EmbeddingDistanceEvalChain.evaluation_namesAD06AAAAr/c ddgS)eReturn the input keys of the chain. Returns: List[str]: The input keys. prediction referencer.rCs r input_keysz%EmbeddingDistanceEvalChain.input_keys sk**r/Ninputs run_managerctj|j|d|dg}||}d|iS)a0Compute the score for a prediction and reference. Args: inputs (Dict[str, Any]): The input data. run_manager (Optional[CallbackManagerForChainRun], optional): The callback manager. Returns: Dict[str, Any]: The computed score. rrrBrZarrayr3embed_documentsrlrDrrrfrBs r_callz EmbeddingDistanceEvalChain._callsW( O + +VL-A6+CV,W X X  ##G,,r/cK|j|d|dgd{V}tj|}||}d|iS)a:Asynchronously compute the score for a prediction and reference. Args: inputs (Dict[str, Any]): The input data. run_manager (AsyncCallbackManagerForChainRun, optional): The callback manager. Returns: Dict[str, Any]: The computed score. rrNrBr3aembed_documentsrZrrlrDrrembeddedrfrBs r_acallz!EmbeddingDistanceEvalChain._acall)s~99 L !6+#6 7        (8$$##G,,r/F)r callbackstagsmetadatainclude_run_inforrrrrrkwargsc R|||d||||}||S)aEvaluate the embedding distance between a prediction and reference. Args: prediction (str): The output string from the first model. reference (str): The reference string (required) callbacks (Callbacks, optional): The callbacks to use. **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. rrrrrrrrI rDrrrrrrrrFs r_evaluate_stringsz,EmbeddingDistanceEvalChain._evaluate_strings?sG2",9EE-    ##F+++r/cvK|||d||||d{V}||S)aAsynchronously evaluate the embedding distance between a prediction and reference. Args: prediction (str): The output string from the first model. reference (str): The output string from the second model. callbacks (Callbacks, optional): The callbacks to use. **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. rrNacallrIrs r_aevaluate_stringsz-EmbeddingDistanceEvalChain._aevaluate_stringsask2zz",9EE- "        ##F+++r/N)r%r&r'r(rpboolryrnrrrrrrr rrrr rqrrr.r/rrwrwsrDXBBBBXB+DI+++X+=A  S#X 89  c3h     0BF  S#X =>  c3h     4$(#$(-1!& , , , ,C= ,  , tCy! ,4S>* , , ,  , , , ,L$(#$(-1!& , , , ,C= ,  , tCy! ,4S>* , , ,  , , , , , ,r/rwceZdZdZedeefdZedefdZ dde ee fde e de ee ffdZ dde ee fde ede ee ffd Zdddd d d ed edede eede e ee fdede defdZdddd d d ed edede eede e ee fdede defdZdS)"PairwiseEmbeddingDistanceEvalChainaUse embedding distances to score semantic difference between two predictions. Examples: >>> chain = PairwiseEmbeddingDistanceEvalChain() >>> result = chain.evaluate_string_pairs(prediction="Hello", prediction_b="Hi") >>> print(result) {'score': 0.5} rc ddgS)rr prediction_br.rCs rrz-PairwiseEmbeddingDistanceEvalChain.input_keyssn--r/c"d|jjdS)Npairwise_embedding_r|r}rCs rrz2PairwiseEmbeddingDistanceEvalChain.evaluation_namesJT%9%?JJJJr/Nrrctj|j|d|dg}||}d|iS)aCompute the score for two predictions. Args: inputs (Dict[str, Any]): The input data. run_manager (CallbackManagerForChainRun, optional): The callback manager. Returns: Dict[str, Any]: The computed score. rrrBrrs rrz(PairwiseEmbeddingDistanceEvalChain._calls\( O + + %vn'=>     ##G,,r/cK|j|d|dgd{V}tj|}||}d|iS)a/Asynchronously compute the score for two predictions. Args: inputs (Dict[str, Any]): The input data. run_manager (AsyncCallbackManagerForChainRun, optional): The callback manager. Returns: Dict[str, Any]: The computed score. rrNrBrrs rrz)PairwiseEmbeddingDistanceEvalChain._acalls~99 L !6.#9 :        (8$$##G,,r/F)rrrrrrrrrrrc R|||d||||}||S)aEvaluate the embedding distance between two predictions. Args: prediction (str): The output string from the first model. prediction_b (str): The output string from the second model. callbacks (Callbacks, optional): The callbacks to use. tags (List[str], optional): Tags to apply to traces metadata (Dict[str, Any], optional): metadata to apply to **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. rrrr rDrrrrrrrrFs r_evaluate_string_pairsz9PairwiseEmbeddingDistanceEvalChain._evaluate_string_pairssG4",lKK-    ##F+++r/cvK|||d||||d{V}||S)aAsynchronously evaluate the embedding distance between two predictions. Args: prediction (str): The output string from the first model. prediction_b (str): The output string from the second model. callbacks (Callbacks, optional): The callbacks to use. tags (List[str], optional): Tags to apply to traces metadata (Dict[str, Any], optional): metadata to apply to traces **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. rrNrrs r_aevaluate_string_pairsz:PairwiseEmbeddingDistanceEvalChain._aevaluate_string_pairssk8zz",lKK- "        ##F+++r/r)r%r&r'r(rprrnrrrrrr rrrr rrqrrr.r/rrrs=.DI...X.KKKKXK =A  S#X 89  c3h     4BF  S#X =>  c3h     6 $$(-1!&!,!,!,!, !,  !, tCy! !,4S>*!,!,!, !,!,!,!,P $$(-1!&#,#,#,#, #,  #, tCy! #,4S>*#,#,#, #,#,#,#,#,#,r/r)"r(enumrtypingrrrrnumpyrZ langchain_core.callbacks.managerrr r langchain_core.embeddingsr langchain_core.utilsr pydanticr rlangchain.chains.baserlangchain.evaluation.schemarrlangchain.schemarrrnrr1rwrr.r/rrsFF,,,,,,,,,,,, 100000))))))&&&&&&&&''''''PPPPPPPP$$$$$$J*T$ttttt5tttnN,N,N,N,N,!=N,N,N,bL,L,L,L,L, "9L,L,L,L,L,r/