cerebras.modelzoo.data_preparation.nlp.chunk_data_processing.data_reader.get_data_size#

cerebras.modelzoo.data_preparation.nlp.chunk_data_processing.data_reader.get_data_size(data: Any) → int[source]#

Compute the size of the given data.

Parameters

data (Any) – Data whose size needs to be determined.

Returns

Size of the given data in bytes.

Return type

int

previous

cerebras.modelzoo.data_preparation.nlp.chunk_data_processing.data_reader.find_last_paragraph_or_sentence_end

next

cerebras.modelzoo.data_preparation.nlp.chunk_data_processing.data_reader.optional_lock