modelzoo.transformers.data_processing#

GenericDataProcessor

Pytorch Generic Dataloader

HDF5IterableDataProcessor

Pytorch HDF5 Dataloader

HDF5IterableDataset

PyTorch HDF5 Dataset

bert

h5_map_dataset

huggingface

qa

scripts

slimpajama

tokenizers

utils