NgP8ddlZddlmZddlmZddlmZddlm Z dZGddZ d Z dd Z ej dZddZdZ ddZdZej dejZej dZdZgdd fdZdZedkr edSdS)N)accuracy)map_tag) str2tuple)Treecg}g}|D]M}||}|t|z }|t|z }Nt||S)a| Score the accuracy of the chunker against the gold standard. Strip the chunk information from the gold standard and rechunk it using the chunker, then compute the accuracy score. :type chunker: ChunkParserI :param chunker: The chunker being evaluated. :type gold: tree :param gold: The chunk structures to score the chunker on. :rtype: float )parseflattentree2conlltags _accuracy)chunkergold gold_tags test_tags gold_tree test_trees K/var/www/html/ai-engine/env/lib/python3.11/site-packages/nltk/chunk/util.pyrrsrII// MM)"3"3"5"566 ^I... ^I...  Y * **cheZdZdZdZdZdZdZdZdZ dd Z d Z d Z d Z d ZdZdZdZdS) ChunkScorea; A utility class for scoring chunk parsers. ``ChunkScore`` can evaluate a chunk parser's output, based on a number of statistics (precision, recall, f-measure, misssed chunks, incorrect chunks). It can also combine the scores from the parsing of multiple texts; this makes it significantly easier to evaluate a chunk parser that operates one sentence at a time. Texts are evaluated with the ``score`` method. The results of evaluation can be accessed via a number of accessor methods, such as ``precision`` and ``f_measure``. A typical use of the ``ChunkScore`` class is:: >>> chunkscore = ChunkScore() # doctest: +SKIP >>> for correct in correct_sentences: # doctest: +SKIP ... guess = chunkparser.parse(correct.leaves()) # doctest: +SKIP ... chunkscore.score(correct, guess) # doctest: +SKIP >>> print('F Measure:', chunkscore.f_measure()) # doctest: +SKIP F Measure: 0.823 :ivar kwargs: Keyword arguments: - max_tp_examples: The maximum number actual examples of true positives to record. This affects the ``correct`` member function: ``correct`` will not return more than this number of true positive examples. This does *not* affect any of the numerical metrics (precision, recall, or f-measure) - max_fp_examples: The maximum number actual examples of false positives to record. This affects the ``incorrect`` member function and the ``guessed`` member function: ``incorrect`` will not return more than this number of examples, and ``guessed`` will not return more than this number of true positive examples. This does *not* affect any of the numerical metrics (precision, recall, or f-measure) - max_fn_examples: The maximum number actual examples of false negatives to record. This affects the ``missed`` member function and the ``correct`` member function: ``missed`` will not return more than this number of examples, and ``correct`` will not return more than this number of true negative examples. This does *not* affect any of the numerical metrics (precision, recall, or f-measure) - chunk_label: A regular expression indicating which chunks should be compared. Defaults to ``'.*'`` (i.e., all chunks). :type _tp: list(Token) :ivar _tp: List of true positives :type _fp: list(Token) :ivar _fp: List of false positives :type _fn: list(Token) :ivar _fn: List of false negatives :type _tp_num: int :ivar _tp_num: Number of true positives :type _fp_num: int :ivar _fp_num: Number of false positives :type _fn_num: int :ivar _fn_num: Number of false negatives. c t|_t|_t|_t|_t|_|dd|_|dd|_|dd|_ |dd|_ d|_ d|_ d|_ d|_d|_d|_d |_dS) Nmax_tp_examplesdmax_fp_examplesmax_fn_examples chunk_labelz.*rgF)set_correct_guessed_tp_fp_fnget_max_tp_max_fp_max_fn _chunk_label_tp_num_fp_num_fn_num_count _tags_correct _tags_total_measuresNeedUpdate)selfkwargss r__init__zChunkScore.__init__rs  555555zz"3S99 zz"3S99 zz"3S99 "JJ}d;;     #(   rc4|jr|j|jz|_|j|jz |_|j|jz |_t |j|_t |j|_t |j|_ d|_dSdS)NF) r-rrrr!r lenr'r(r)r.s r_updateMeasureszChunkScore._updateMeasuress  # -}t}4DH}t}4DH}t}4DHtx==DLtx==DLtx==DL',D $ $ $ - -rc |xjt||j|jzc_|xjt||j|jzc_|xjdz c_d|_ t |}t |}n#t$rdx}}YnwxYw|xjt|z c_|xj tdt||Dz c_ dS)aU Given a correctly chunked sentence, score another chunked version of the same sentence. :type correct: chunk structure :param correct: The known-correct ("gold standard") chunked sentence. :type guessed: chunk structure :param guessed: The chunked sentence to be scored. Tc3,K|]\}}||k dVdS)r6Nr7).0tgs r z#ChunkScore.score..s3" " 1aqAvvAvvvv" " rN) r _chunksetsr*r&rr-r ValueErrorr,r2r+sumzip)r.correctguessed correct_tags guessed_tagss rscorezChunkScore.scores GT[$:KLLL  GT[$:KLLL  q #'  -)'22L)'22LL - - -+- ,L<<<  - C --- c" "  l;;" " "    s+B BBc:|jdkrdS|j|jz S)z Return the overall tag-based accuracy for all text that have been scored by this ``ChunkScore``, using the IOB (conll2000) tag encoding. :rtype: float rr6)r,r+r3s rrzChunkScore.accuracys(  q 1!D$444rcl||j|jz}|dkrdS|j|z S)z Return the overall precision for all texts that have been scored by this ``ChunkScore``. :rtype: float r)r4r'r(r.divs r precisionzChunkScore.precision? lT\) !881<#% %rcl||j|jz}|dkrdS|j|z S)z Return the overall recall for all texts that have been scored by this ``ChunkScore``. :rtype: float rr4r'r)rHs rrecallzChunkScore.recallrKr?c||}|}|dks|dkrdSd||z d|z |z zz S)a Return the overall F measure for all texts that have been scored by this ``ChunkScore``. :param alpha: the relative weighting of precision and recall. Larger alpha biases the score towards the precision value, while smaller alpha biases the score towards the recall value. ``alpha`` should have a value in the range [0,1]. :type alpha: float :rtype: float rr6)r4rJrN)r.alphaprs r f_measurezChunkScore.f_measuresd  NN   KKMM 66Q!VV1EAIUa/00rcj|t|j}d|DS)z Return the chunks which were included in the correct chunk structures, but not in the guessed chunk structures, listed in input order. :rtype: list of chunks cg|] }|d Sr6r7r9cs r z%ChunkScore.missed..%%%!%%%r)r4listr!r.chunkss rmissedzChunkScore.misseds7 dh%%f%%%%rcj|t|j}d|DS)z Return the chunks which were included in the guessed chunk structures, but not in the correct chunk structures, listed in input order. :rtype: list of chunks cg|] }|d SrWr7rXs rrZz(ChunkScore.incorrect..r[r)r4r\r r]s r incorrectzChunkScore.incorrects7 dh%%f%%%%rcBt|j}d|DS)z Return the chunks which were included in the correct chunk structures, listed in input order. :rtype: list of chunks cg|] }|d SrWr7rXs rrZz&ChunkScore.correct..r[r)r\rr]s rrAzChunkScore.correct'dm$$%%f%%%%rcBt|j}d|DS)z Return the chunks which were included in the guessed chunk structures, listed in input order. :rtype: list of chunks cg|] }|d SrWr7rXs rrZz&ChunkScore.guessed..r[r)r\rr]s rrBzChunkScore.guessedrercH||j|jzS)NrMr3s r__len__zChunkScore.__len__s# |dl**rcFdtt|zdzS)z` Return a concise representation of this ``ChunkScoring``. :rtype: str z)reprr2r3s r__repr__zChunkScore.__repr__s #T#d))__4zAArcdd|dzddzd|dzddzd|dzddzd|dzdd zS) a- Return a verbose representation of this ``ChunkScoring``. This representation includes the precision, recall, and f-measure scores. For other information about the score, use the accessor methods (e.g., ``missed()`` and ``incorrect()``). :rtype: str zChunkParse score: z IOB Accuracy: rz5.1fz%% z Precision: z Recall: z F-Measure: z%%)rrJrNrTr3s r__str__zChunkScore.__str__s "DDMMOOc$9DDDD FEDNN$4$4s$:EEEE GCDKKMMC$7BBBB DDDNN$4$4s$:CCCC  E rN)rO)__name__ __module__ __qualname____doc__r0r4rErrJrNrTr_rbrArBrirlrnr7rrrr3s<<|)))&---   : 5 5 5 & & & & & &1111& & & & & & &&&&&&&+++BBB     rrcTd}g}|D]}t|trwtj||r+|||f|f|t|z }|dz }t|S)Nrr6) isinstancerrematchlabelappendfreezer2leavesr)r:countrposr^childs rr=r=2s C F eT " " x U[[]]33 > s|U\\^^<=== 3u||~~&& &CC 1HCC v;;rNPS/ctjd}t|gg}||D]Y}|} | ddkryt |dkr%t d|dt|g} |d| || | ddkrMt |d kr%t d |d| ||d| t| |\} } |r|rt||| } |d| | f[t |dkr t d t |d|dS) aB Divide a string of bracketted tagged text into chunks and unchunked tokens, and produce a Tree. Chunks are marked by square brackets (``[...]``). Words are delimited by whitespace, and each word should have the form ``text/tag``. Words that do not contain a slash are assigned a ``tag`` of None. :param s: The string to be converted :type s: str :param chunk_label: The label to use for chunk nodes :type chunk_label: str :param root_label: The label to use for the root of the tree :type root_label: str :rtype: Tree z\[|\]|[^\[\]\s]+r[r6zUnexpected [ at char d]zUnexpected ] at char NzExpected ] at char ) rucompilerfinditergroupr2r>startrxpoprr) sr root_labelsep source_tagset target_tagsetWORD_OR_BRACKETstackrvtextchunkwordtags r tagstr2treer?s(j!455O *b ! ! "E ))!,,..{{}} 7c>>5zzQ !J!J!J!JKKKb))E "I  U # # # LL     !W^^5zzQ !J!J!J!JKKK IIKKKK{b   &&&&%dC00 c E]E!-DDCb   $---- 5zzQ9s1vv999::: 8Orz(\S+)\s+(\S+)\s+([IOB])-?(\S+)?r~PPVPct|gg}t|dD]!\}}|st|}|t d|d|\}}} } || |vrd} | dko| |dk} | dvs| r't|d kr| | d ks| r@t| g} |d | | | |d ||f#|d S) a* Return a chunk structure for a single sentence encoded in the given CONLL 2000 style string. This function converts a CoNLL IOB string into a tree. It uses the specified chunk types (defaults to NP, PP and VP), and creates a tree rooted at a node labeled S (by default). :param s: The CoNLL string to be converted. :type s: str :param chunk_types: The chunk types to be converted. :type chunk_types: tuple :param root_label: The node label to use for the root. :type root_label: str :rtype: Tree  NzError on line rOIrBOrBr) r enumeratesplitstrip_LINE_RErvr>groupsrwr2rrx) r chunk_typesrrlinenolinervrrstate chunk_type mismatch_Irs r conllstr2treerusk$*b ! ! "E!!''$--00&& zz||  t$$ =8f88899 9).&sE:  "z'D'DEc\EjE"IOO4E4E&E D==J=5zzQ  C<<:<R((E "I  U # # # LL    b $%%%% 8OrcNg}|D]} |}d}|D]O}t|trtd||d|d||zfd}Pl#t $r'||d|ddfYwxYw|S)z Return a list of 3-tuples containing ``(word, tag, IOB-tag)``. Convert a tree to the CoNLL IOB tag format. :param t: The tree to be converted. :type t: Tree :rtype: list(tuple) B-z7Tree is too deeply nested to be printed in CoNLL formatrr6I-r)rwrtrr>rxAttributeError)r:tagsr}categoryprefixcontentss rr r s D 3 3 3{{}}HF!  h--$Q Xa[(1+v7HIJJJ   3 3 3 KKq58S1 2 2 2 2 2 3 KsA(A11.B"!B"Fc t|g}|D]n\}}}|)|rtd|||f2|dr/|t|dd||fgv|drt |dksAt |dtr&|d|ddkrA|rtd|t|dd||fg |d||f?|dkr|||f^td ||S) z1 Convert the CoNLL IOB format to a tree. NzBad conll tag sequencerrrrrrzBad conll tag )rr>rx startswithr2rtrw)sentencerrstricttreerpostagchunktags rconlltags2treers  B  D"*<<fh   , !9::: T6N++++   & & < KKXabb\T6N+;<< = = = =   & & <D Q!$r(D118>>##x|33F$%=>>>KKXabb\T6N3C D DEEEERv//// __ KKv ' ' ' ':h::;; ; Krc^dt|D}d|S)z Return a multiline string where each line contains a word, tag and IOB tag. Convert a tree to the CoNLL IOB string format :param t: The tree to be converted. :type t: Tree :rtype: str c8g|]}d|S) )join)r9tokens rrZz!tree2conllstr..s" < < <SXXe__ < < \s*(\s*(?P.+?)\s*\s*)?(\s*(?P.+?)\s*\s*)?(\s*(?P.+?)\s*\s*)?\s*(\s*(?P.+?)\s*\s*)?(?P.*?)\s*\s*\s*z#]*?type="(?P\w+)"ct|gg}|gStjd|D]1}|} |drt |}|td|t|dg}|d|||nE|dr| n|d|#ttf$r,}td| dd |d}~wwxYwt|d krtd |d S) Nz<[^>]+>|[^\s<]+zrr2)rrrpiece_mpiecemres r_ieer_read_textrs *b ! ! "E y ;1155  && (!''..9&%(((QWWV__b11b   ''' U####!!%(( ( b   '''J'   Kw}}KKKK   5zzQ*+++ 8OsCDE/'EE) LOCATION ORGANIZATIONPERSONDURATIONDATECARDINALPERCENTMONEYMEASUREc `t|}|rt|d||d|d|dt|d|dSt||S)ap Return a chunk structure containing the chunked tagged text that is encoded in the given IEER style string. Convert a string of chunked tagged text in the IEER named entity format into a chunk structure. Chunks are of several types, LOCATION, ORGANIZATION, PERSON, DURATION, DATE, CARDINAL, PERCENT, MONEY, and MEASURE. :rtype: Tree rdocnodoctype date_timeheadline)rrrrr) _IEER_DOC_RErvrr)rrrrs r ieerstr2treer's8 1A .#AGGFOOZ@@WWW%%wwy))--( (;(;ZHH   q*---rcdd}ddl}|j|d}|t d}t |d}|t dt |j|t dS) Nzd[ Pierre/NNP Vinken/NNP ] ,/, [ 61/CD years/NNS ] old/JJ ,/, will/MD join/VB [ the/DT board/NN ] ./.rr~)rav These DT B-NP research NN I-NP protocols NNS I-NP offer VBP B-VP to TO B-PP the DT B-NP patient NN I-NP not RB O only RB O the DT B-NP very RB I-NP best JJS I-NP therapy NN I-NP which WDT B-NP we PRP B-NP have VBP B-VP established VBN I-VP today NN B-NP but CC B-NP also RB I-NP the DT B-NP hope NN I-NP of IN B-PP something NN B-NP still RB B-ADJP better JJR I-ADJP . . O )r~r)rz CoNLL output:)nltkrrpprintrrr)rrr: conll_trees rdemorRsnAKKK qd33AHHJJJ GGG A<ql;;;J / $* " ": . ./// GGGGGr__main__)r~rrNN)rr)rrF)ru nltk.metricsrr nltk.tag.mappingr nltk.tag.utilr nltk.treerrr=rrrrr rrDOTALLrrrrrror7rrrs ......$$$$$$######+++