g dZddlZddlmZddlmZmZejeZGddeZ d e iZ d d iZ d diZ dS) z+XNLI utils (dataset loading and evaluation)N)logging) DataProcessor InputExamplec,eZdZdZddZdZdZdZdS) XnliProcessorz Processor for the XNLI dataset. Adapted from https://github.com/google-research/bert/blob/f39e881b169b9d53bea03d2d341b31707a6c052b/run_classifier.py#L207 Nc"||_||_dSN)languagetrain_language)selfr r s ]/var/www/html/ai-engine/env/lib/python3.11/site-packages/transformers/data/processors/xnli.py__init__zXnliProcessor.__init__!s  ,c l|j|jn|j}|tj|d|d}g}t |D]\}}|dkr d|}|d}|d} |ddkrd n|d} t|tstd |d t| tstd | d t| tstd | d | t||| | |S)See base class.Nz$XNLI-MT-1.0/multinli/multinli.train.z.tsvrztrain-r contradictory contradictionTraining input  is not a stringTraining label guidtext_atext_blabel) r r _read_tsvospathjoin enumerate isinstancestr TypeErrorappendr) rdata_dirlglinesexamplesilinerrrrs rget_train_examplesz XnliProcessor.get_train_examples%s_"19T]]t?Rrw||H6e]_6e6e6effgg '' ` `GAtAvvA<rDs 21 ........  H % %8:8:8:8:8:M8:8:8:x M    Ar