modelzoo.transformers.data_processing.slimpajama.utils.utils

Functions

cycle_documents
rm_if_exists
sha256str
utf8len
write_lmd_dataset

previous

modelzoo.transformers.data_processing.slimpajama.utils

next

modelzoo.transformers.data_processing.slimpajama.utils.utils.cycle_documents