modelzoo.transformers.data_processing.slimpajama.preprocessing.filter

Functions

clean
filter_dataset
get_short_documents

previous

modelzoo.transformers.data_processing.slimpajama.preprocessing.datasets.RedPajamaWikipediaDataset

next

modelzoo.transformers.data_processing.slimpajama.preprocessing.filter.clean