cerebras.modelzoo.data_preparation.nlp.hdf5_preprocessing.hdf5_curation_corpus_preprocessor

Classes

CurationCorpusPreprocessor

A customized version of the SummarizationPreprocessor for the Curation Corpus dataset in .jsonl format.

previous

cerebras.modelzoo.data_preparation.nlp.hdf5_preprocessing.hdf5_base_preprocessor.HDF5BasePreprocessor

next

cerebras.modelzoo.data_preparation.nlp.hdf5_preprocessing.hdf5_curation_corpus_preprocessor.CurationCorpusPreprocessor