NgdZddlZddlmZddlmZmZmZmZm Z m Z  ddlm Z dZ n #e $rdZ YnwxYw ddlmZdZn #e $rdZYnwxYw dd lmZdZn #e $rdZYnwxYw dd lmZdZn #e $rdZYnwxYwd d lmZmZeeeeee ZeddZedddddZdZ d"dedeededededededededeededed efd!ZdS)#zE Dataset Factory Hacked together by / Copyright 2021, Ross Wightman N)Optional)CIFAR100CIFAR10MNISTKMNIST FashionMNIST ImageFolder) Places365TF) INaturalist)QMNIST)ImageNet)IterableImageDataset ImageDataset)cifar10cifar100mnistkmnist fashion_mnist)traintraining)valvalid validationeval evaluationc0|dd}tj|}tj|r|Sfd}|t vr|t n|t vr|t S)N[rc|D]E}tj|}tj|r|cSFSN)ospathjoinexists)synstry_rootroots U/var/www/html/ai-engine/env/lib/python3.11/site-packages/timm/data/dataset_factory.py_tryz_search_split.._try2sP  Aw||D!,,Hw~~h''   )splitr!r"r#r$_TRAIN_SYNONYM _EVAL_SYNONYM)r(r, split_namer'r*s` r) _search_splitr0+sS!!!$Jw||D*--H w~~h ^##tN## } $ $tM"" Kr+r*RGBnamer(r, search_split class_map load_bytes is_trainingdownload batch_size num_samplesseedrepeatsinput_img_modec  vd| D} |}|dr|ddd}t d'||d| }|t vr"t |}|t v}|d'd|i|}n|dks|d krts Jd d }|d}t|d kr>|d d}t|d kr|d }|d}|t vrd}n |tvrd}td'||d|}n|dkr8ts Jd|t vrd}n |tvrd}td'd|i|}n|dkr*ts Jd|t v}td'd|i|}n|dkr,ts Jd|tvrd}t!d'd|i|}nM|dks|dkr@|r/t"j|rt)||}t+|fi| }nJd||drt-|f|||| d | }n|d!rt/|f||||||| | | | d" | }n|d#rt/|f||||||| | | | d" | }np|d$rt/|f|||||| | | | d% | }nC|r/t"j|rt)||}t-|f|||| d&| }|S)(a Dataset factory method In parentheses after each arg are the type of dataset supported for each arg, one of: * folder - default, timm folder (or tar) based ImageDataset * torch - torchvision based datasets * HFDS - Hugging Face Datasets * TFDS - Tensorflow-datasets wrapper in IterabeDataset interface via IterableImageDataset * WDS - Webdataset * all - any of the above Args: name: dataset name, empty is okay for folder based datasets root: root folder of dataset (all) split: dataset split (all) search_split: search for split specific child fold from root so one can specify `imagenet/` instead of `/imagenet/val`, etc on cmd line / config. (folder, torch/folder) class_map: specify class -> index mapping via text file or dict (folder) load_bytes: load data, return images as undecoded bytes (folder) download: download dataset if not present and supported (HFDS, TFDS, torch) is_training: create dataset in train mode, this is different from the split. For Iterable / TDFS it enables shuffle, ignored for other datasets. (TFDS, WDS) batch_size: batch size hint for (TFDS, WDS) seed: seed for iterable datasets (TFDS, WDS) repeats: dataset repeats per iteration i.e. epoch (TFDS, WDS) input_img_mode: Input image color conversion mode e.g. 'RGB', 'L' (folder, TFDS, WDS, HFDS) **kwargs: other args to pass to dataset Returns: Dataset object ci|] \}}||| Sr ).0kvs r) z"create_dataset..ms ? ? ?tq!ar+ztorch//)r(r8r inaturalistinatz@Please update to PyTorch 1.10, torchvision 0.11+ for Inaturalistfullrr_ 2021_train 2021_valid)version target_type places365zGPlease update to a newer PyTorch and torchvision for Places365 dataset.ztrain-standardrr,qmnistzDPlease update to a newer PyTorch and torchvision for QMNIST dataset.imagenetzFPlease update to a newer PyTorch and torchvision for ImageNet dataset. image_folderfolderFzUnknown torchvision dataset zhfds/)readerr,r5r=zhfids/) rUr,r5r7r8r9r:r<r;r=ztfds/zwds/) rUr,r5r7r9r:r<r;r=)rUr5r6r=r@)itemslower startswithr,dict_TORCH_BASIC_DSr-has_inaturalistlenr.r has_places365r has_qmnistr has_imagenetr r!r"isdirr0r rr)r3r(r,r4r5r6r7r8r9r:r;r<r=kwargs torch_kwargsds_class use_traindsrO split_splits r)create_datasetrg?s\@ ?v||~~ ? ? ?F ::<rps \\\\\\\\\\\\\\\\......MMMMM000000OOOOO++++++JJJJJ------LLLLL87777777$     D4000TdRVWWW ,#!! !%)#aaasmaa a  a  aaaac]aaaaaaaaas?'11>AA AAA#A,,A65A6