modelzoo.transformers.data_processing.utils.count_total_documents#

modelzoo.transformers.data_processing.utils.count_total_documents(metadata_files)[source]#

Counts total number of documents in metadata_files. :param str or list[str] metadata_files: Path or list of paths

to metadata files.

Returns

Number of documents whose paths are contained in the metadata files.

previous

modelzoo.transformers.data_processing.utils.convert_to_unicode

next

modelzoo.transformers.data_processing.utils.create_masked_lm_predictions