cerebras.modelzoo.data.common#

GenericDataProcessor

Pytorch Generic Dataloader

HDF5IterableDataProcessor

Pytorch HDF5 Dataloader

HDF5IterableDataset

PyTorch HDF5 Dataset

SyntheticDataProcessor

Utilities for generating synthetic data based on some specification.

config

Config classes of T5 data Configs

h5_map_dataset

input_utils

restartable_dataloader

tensor_spec

Wrapper class used to process TensorSpecs using a custom yaml tag.