modelzoo.transformers.data_processing.scripts.chunk_preprocessing.data_reader.get_data_size#

modelzoo.transformers.data_processing.scripts.chunk_preprocessing.data_reader.get_data_size(data: Union[str, bytes]) → int[source]#

Compute the size of the given data.

Parameters

data (Union[str, bytes]) – Data whose size needs to be determined.

Returns

Size of the given data in bytes.

Return type

int

previous

modelzoo.transformers.data_processing.scripts.chunk_preprocessing.data_reader.find_last_paragraph_or_sentence_end

next

modelzoo.transformers.data_processing.scripts.chunk_preprocessing.data_reader.set_doc_idx