Ng.ddlZddlmZddlmZddlmZddlmZGddej Z Gdd ej Z dS) N)closing)data) PorterStemmer)SnowballStemmerc,eZdZdZdZdZdZdZdS) SnowballTestctdd}|ddksJ|ddksJ|ddksJ|dd ksJ|d d ksJ|d d ksJ|d d ksJ|d d ksJ|ddksJtdd}|d dksJ|dd ksJ|ddksJtd}|ddksJ|ddksJ|ddksJ|dd ksJ|ddksJdS)z this unit testing for test the snowball arabic light stemmer this stemmer deals with prefixes and suffixes arabicTu&الْعَرَبِــــــيَّةuعربuالعربيةu فقالواuقالuالطالباتuطالبuفالطالباتuوالطالباتuالطالبونu اللذانuمنFuاللذuالكلماتuكلمNrstem)self ar_stemmers T/var/www/html/ai-engine/env/lib/python3.11/site-packages/nltk/test/unit/test_stem.py test_arabiczSnowballTest.test_arabic s. %Xt44 GHHHTTTT/00H<<<<~..(::::122j@@@@344 BBBB344 BBBB122j@@@@~...@@@@v&&&0000$Xu55 ~..*<<<<122j@@@@/00H<<<<$X.. GHHHTTTT/00H<<<<~..(::::122j@@@@/00H<<<<<<cZtd}|ddksJdS)Nrussianuавантненькаяuавантненькr )r stemmer_russians r test_russianzSnowballTest.test_russian's6))44##$>??CYYYYYYYrctd}tdd}|ddksJ|ddksJ|ddksJ|ddksJdS)NgermanT)ignore_stopwordsu Schränkeschrankkeinenkeinr )r stemmer_germanstemmer_german2s r test_germanzSnowballTest.test_german+s(22)(TJJJ""=11Y>>>>##M22i????""8,,6666##H--999999rctd}|ddksJ|ddksJdS)Nspanish Visionadovisionalguealgur r stemmers r test_spanishzSnowballTest.test_spanish5sQ!),,||K((H4444||G$$......rcZtd}|ddksJdS)Nenglishzy'syr r%s rtest_short_strings_bugz#SnowballTest.test_short_strings_bug=s2!),,||E""c))))))rN)__name__ __module__ __qualname__rrrr'r+rrrr sb===:ZZZ:::///*****rrc8eZdZdZdZdZdZdZdZdZ dS) PorterTestcttjdd5}|cdddS#1swxYwYdS)Nz*stemmers/porter_test/porter_vocabulary.txtutf-8encoding)rrfindopenread splitlinesr fps r _vocabularyzPorterTest._vocabularyCs  IB C C H H  I     *7799''))  * * * * * * * * * * * * * * * * * *s&A))A-0A-ct|}t||D]@\}}||}||ks Jd||||AdS)N)modez*{} should stem to {} in {} mode but got {})rzipr<r format)r stemmer_modeexpected_stemsr&word true_stemour_stems r_test_against_expected_outputz(PorterTest._test_against_expected_outputKs\222"4#3#3#5#5~FF  OD)||D))HI%%%;BB &%%%  rc*ttjdd5}|t j|ddddS#1swxYwYdS)azTests all words from the test vocabulary provided by M Porter The sample vocabulary and output were sourced from https://tartarus.org/martin/PorterStemmer/voc.txt and https://tartarus.org/martin/PorterStemmer/output.txt and are linked to from the Porter Stemmer algorithm's homepage at https://tartarus.org/martin/PorterStemmer/ z-stemmers/porter_test/porter_martin_output.txtr3r4N) rrr6r7rFrMARTIN_EXTENSIONSr8r9r:s rtest_vocabulary_martin_modez&PorterTest.test_vocabulary_martin_modeXs IE F F K K  L       . ./1E1E1G1G                      ABB B c*ttjdd5}|t j|ddddS#1swxYwYdS)Nz+stemmers/porter_test/porter_nltk_output.txtr3r4) rrr6r7rFrNLTK_EXTENSIONSr8r9r:s rtest_vocabulary_nltk_modez$PorterTest.test_vocabulary_nltk_modejs  IC D D I I  J       . .-rwwyy/C/C/E/E                      rJcttjdd5}|t j|dddn #1swxYwY|t jtjdddS)Nz/stemmers/porter_test/porter_original_output.txtr3r4) rrr6r7rFrORIGINAL_ALGORITHMr8r9r:s rtest_vocabulary_original_modez(PorterTest.test_vocabulary_original_modets IG H H M M  N       . .0"''))2F2F2H2H                    **  , IG H H T7T # # TVV Z\\      sABB B cTtddksJdS)zTest for bug https://github.com/nltk/nltk/issues/1581 Ensures that 'oed' can be stemmed without throwing an error. oedoNrr )r s r test_oed_bugzPorterTest.test_oed_bugs- ##E**c111111rc8t}|ddksJ|ddksJ|dddksJ|ddksJ|dddksJd S) zTest for improvement on https://github.com/nltk/nltk/issues/2507 Ensures that stems are lowercased when `to_lowercase=True` OnonIiF) to_lowercaseGithubgithubNrT)r porters rtest_lowercase_optionz PorterTest.test_lowercase_options {{4  D(((({{33&&&&{{3U{33s::::{{8$$0000{{8%{88HDDDDDDrN) r,r-r.r<rFrIrMrPrUr_r/rrr1r1Bs***   $   0222 E E E E Err1) unittest contextlibrnltkrnltk.stem.porterrnltk.stem.snowballrTestCaserr1r/rrrfs******......6*6*6*6*6*8$6*6*6*r[E[E[E[E[E"[E[E[E[E[Er