cerebras.modelzoo.data_preparation.nlp.slimpajama.preprocessing.filter

Functions

clean
filter_dataset
get_short_documents

previous

cerebras.modelzoo.data_preparation.nlp.slimpajama.preprocessing.datasets.RedPajamaWikipediaDataset

next

cerebras.modelzoo.data_preparation.nlp.slimpajama.preprocessing.filter.clean