cerebras.modelzoo.data_preparation.nlp.chunk_data_processing.data_reader.find_last_paragraph_or_sentence_end#

cerebras.modelzoo.data_preparation.nlp.chunk_data_processing.data_reader.find_last_paragraph_or_sentence_end(buffer: str) → int[source]#

Find the last end of a paragraph (denoted by ‘

‘) or a sentence in the buffer.

Args:

buffer (str): The text buffer.

Returns:

int: The position of the last end of the paragraph or sentence.

previous

cerebras.modelzoo.data_preparation.nlp.chunk_data_processing.data_reader

next

cerebras.modelzoo.data_preparation.nlp.chunk_data_processing.data_reader.get_data_size