Ng v d dl mZ d dlmZ d dlmZmZ ej Z eedddd Z de _
d Z d
Z d S ) )partial)chain)
everygrams pad_sequenceTzz)pad_leftleft_pad_symbol pad_rightright_pad_symbolzPads both ends of a sentence to length specified by ngram order.
Following convention pads the start of sentence pads its end.
c \ t t t || | S )zpHelper with some useful defaults.
Applies pad_both_ends to sentence and follows it up with everygrams.
nmax_len)r list
pad_both_ends)ordersentences Q/var/www/html/ai-engine/env/lib/python3.11/site-packages/nltk/lm/preprocessing.pypadded_everygramsr s,
d=U;;;<z,padded_everygram_pipeline..1 sB LLtDD))**E : : :LLLLLLr )r r flattenmap)r textr s ` @r padded_everygram_pipeliner " sQ %000JLLLLLtLLLJ%%&& r N)
functoolsr itertoolsr nltk.utilr r
from_iterabler r __doc__r r r r r' s . . . . . . . .
M M M r