Ng`UdZddlmZmZmZmZmZmZmZddl m Z ddl m Z ddl mZddlmZddlmZddlmZmZdd lmZmZdd lmZdd lmZmZdd lmZdd l m!Z!ddl"m#Z#m$Z$m%Z%ddl&m'Z'ddl(m)Z)m*Z*m+Z+ddl,m-Z-m.Z.ddl/m0Z0m1Z1de2deefdZ3ie)j4e%e)j5e$e)j6e#e)j7ee)j8e.e)j9ee)j:e-e)j;ee)j<ee)j=ee)j>e1e)j?e0e)j@ee)jAee)jBee)jCee)jDee)jEe!e)jFe'e)jGeiZHee)eee*ee ee+ffeId<ddde)dee dedee e+ffdZJddddee)dee deeKdedeee e+ff d ZLdS)!z Loading datasets and evaluators.)AnyDictListOptionalSequenceTypeUnion)BaseLanguageModel)Chain)TrajectoryEvalChain)PairwiseStringEvalChain)LabeledPairwiseStringEvalChain)CriteriaEvalChainLabeledCriteriaEvalChain)EmbeddingDistanceEvalChain"PairwiseEmbeddingDistanceEvalChain)ExactMatchStringEvaluator)JsonEqualityEvaluatorJsonValidityEvaluator)JsonEditDistanceEvaluator)JsonSchemaEvaluator)ContextQAEvalChainCotQAEvalChain QAEvalChain)RegexMatchStringEvaluator) EvaluatorType LLMEvalChainStringEvaluator)LabeledScoreStringEvalChainScoreStringEvalChain)PairwiseStringDistanceEvalChainStringDistanceEvalChainurireturnc ddlm}n#t$rtdwxYw|d|}d|dDS)aLoad a dataset from the `LangChainDatasets on HuggingFace `_. Args: uri: The uri of the dataset to load. Returns: A list of dictionaries, each representing a row in the dataset. **Prerequisites** .. code-block:: shell pip install datasets Examples -------- .. code-block:: python from langchain.evaluation import load_dataset ds = load_dataset("llm-math") r) load_datasetzXload_dataset requires the `datasets` package. Please install with `pip install datasets`zLangChainDatasets/cg|]}|Sr().0ds X/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain/evaluation/loading.py z load_dataset..Fs ( ( (!A ( ( (train)datasetsr& ImportError)r#r&datasets r+r&r&'s, )))))))     :    l55566G ( (ww' ( ( ((s #_EVALUATOR_MAPN)llm evaluatorr3kwargsc |tvr9td|dttt|}t |t r ddlm}n5#t$r( ddl m}n#t$rtdwxYwYnwxYw|p |ddd}n&#t$r}td |d |d }~wwxYw|j d d |i|S|d i|S)a<Load the requested evaluation chain specified by a string. Parameters ---------- evaluator : EvaluatorType The type of evaluator to load. llm : BaseLanguageModel, optional The language model to use for evaluation, by default None **kwargs : Any Additional keyword arguments to pass to the evaluator. Returns ------- Chain The loaded evaluation chain. Examples -------- >>> from langchain.evaluation import load_evaluator, EvaluatorType >>> evaluator = load_evaluator(EvaluatorType.QA) zUnknown evaluator type: z Valid types are: r) ChatOpenAIzCould not import langchain_openai or fallback onto langchain_community. Please install langchain_openai or specify a language model explicitly. It's recommended to install langchain_openai AND specify a language model explicitly.zgpt-4*)modelseed temperaturezEvaluation with the z requires a language model to function. Failed to create the default 'gpt-4' model. Please manually provide an evaluation LLM or check your openai credentials.Nr3r() r2 ValueErrorlistkeys issubclassrlangchain_openair7r0&langchain_community.chat_models.openai Exceptionfrom_llm)r4r3r5 evaluator_clsr7es r+load_evaluatorrFcs6&& @y @ @"&~':':'<'<"="= @ @   #9-M-..'  7777777    QQQQQQQ"%?RQ BACC   5}555    &}%88#8888}&&v&&&sN'A.-B4. B 9B?B BB B4B  B44 C>CC)r3config evaluatorsrGc g}|D]B}|r||ini}|t|fd|ii||C|S)aeLoad evaluators specified by a list of evaluator types. Parameters ---------- evaluators : Sequence[EvaluatorType] The list of evaluator types to load. llm : BaseLanguageModel, optional The language model to use for evaluation, if none is provided, a default ChatOpenAI gpt-4 model will be used. config : dict, optional A dictionary mapping evaluator types to additional keyword arguments, by default None **kwargs : Any Additional keyword arguments to pass to all evaluators. Returns ------- List[Chain] The loaded evaluators. Examples -------- >>> from langchain.evaluation import load_evaluators, EvaluatorType >>> evaluators = [EvaluatorType.QA, EvaluatorType.CRITERIA] >>> loaded_evaluators = load_evaluators(evaluators, criteria="helpfulness") r3)getappendrF)rHr3rGr5loadedr4_kwargss r+load_evaluatorsrNsrBFSS /5=&**Y+++2 nYQQCQ;Pf;P;PQQRRRR Mr-)M__doc__typingrrrrrrr langchain_core.language_modelsr langchain.chains.baser 1langchain.evaluation.agents.trajectory_eval_chainr langchain.evaluation.comparisonr *langchain.evaluation.comparison.eval_chainr(langchain.evaluation.criteria.eval_chainrr,langchain.evaluation.embedding_distance.baserr%langchain.evaluation.exact_match.baser!langchain.evaluation.parsing.baserr*langchain.evaluation.parsing.json_distancer(langchain.evaluation.parsing.json_schemarlangchain.evaluation.qarrr%langchain.evaluation.regex_match.baserlangchain.evaluation.schemarrr'langchain.evaluation.scoring.eval_chainrr )langchain.evaluation.string_distance.baser!r"strr&QACOT_QA CONTEXT_QAPAIRWISE_STRING SCORE_STRINGLABELED_PAIRWISE_STRINGLABELED_SCORE_STRINGAGENT_TRAJECTORYCRITERIALABELED_CRITERIASTRING_DISTANCEPAIRWISE_STRING_DISTANCEEMBEDDING_DISTANCEPAIRWISE_EMBEDDING_DISTANCE JSON_VALIDITY JSON_EQUALITYJSON_EDIT_DISTANCEJSON_SCHEMA_VALIDATION REGEX_MATCH EXACT_MATCHr2__annotations__rFdictrNr(r-r+rxsb&&&CCCCCCCCCCCCCCCCCC<<<<<<''''''QQQQQQCCCCCCUUUUUULKKKKKQPPPPPHHHHHHSSSSSSSSSSKKKKKKTTTTTTTTTT )c)d4j))))Hk.0!#:   4  )+I &(C"$7-"$<!#:*,K$&@-/Q!6 !6!"$&?#$(*=88)5l+T%[$:OOPP:(,>'>'>'>' # $>' >'  5/ !" >'>'>'>'H(,! %%%'% # $% TN %  %  %& '( %%%%%%r-