modelzoo.transformers.data_processing.slimpajama.preprocessing.normalize_text

Script that normalizes text

Functions

normalize_text
parse_args
recreate_dataset

previous

modelzoo.transformers.data_processing.slimpajama.preprocessing.filter.get_short_documents

next

modelzoo.transformers.data_processing.slimpajama.preprocessing.normalize_text.normalize_text