Ng lddlmZddlZddlZddlZddlmZGddZGddeZdS) ) annotationsN) InputExamplec<eZdZdZddddejdddfdZd d Zd S) STSDataReadera1Reads in the STS dataset. Each line contains two sentences (s1_col_idx, s2_col_idx) and one label (score_col_idx) Default values expects a tab separated file with the first & second column the sentence pair and third column the score (0...1). Default config normalizes scores from 0...5 to 0...1 rr Tc ||_||_||_||_||_||_||_||_| |_dS)N) dataset_folder score_col_idx s1_col_idx s2_col_idx delimiterquotingnormalize_scores min_score max_score) selfr rrr rrrrrs g/var/www/html/ai-engine/env/lib/python3.11/site-packages/sentence_transformers/readers/STSDataReader.py__init__zSTSDataReader.__init__sJ-*$$" 0""c tj|j|}|drt j|ddnt |d5}tj||j |j }g}t|D]\}}t||j } |jr| |jz |j|jz z } ||j} ||j} |t)|t+|z| | g| |dkrt-||krnd d d n #1swxYwY|S) zJfilename specified which data split to use (train.csv, dev.csv, test.csv).z.gzrtutf8)encodingzutf-8)rr)guidtextslabelrN)ospathjoinr endswithgzipopencsvreaderrr enumeratefloatr rrrrrappendrstrlen) rfilename max_examplesfilepathfIndataexamplesidrowscores1s2s r get_exampleszSTSDataReader.get_examples&s7<< 3X>>  '' 2DIhv 6 6 6 6h111 69:cT^T\RRRDH$T??  Cc$"4566(Y"T^38WXE)) (SWW2DRQSH\a b b bccc!##H (E(EE#               &s#C EEEN)r)__name__ __module__ __qualname____doc__r& QUOTE_NONErr8rrrr sb####,rrc>eZdZdZddddejdddffd ZxZS) STSBenchmarkDataReaderzReader especially for the STS benchmark dataset. There, the sentences are in column 5 and 6, the score is in column 4. Scores are normalized from 0...5 to 0...1 r r Trc \t|||||||||  dS)N) r rrr rrrrr)superr) rr rrr rrrrr __class__s rrzSTSBenchmarkDataReader.__init__DsK )!!'-  r)r9r:r;r<r&r=r __classcell__)rEs@rr@r@?sc          rr@) __future__rr&r$r rrr@r>rrrIs"""""" 22222222j     ]     r