modelzoo.transformers.data_processing.bert.bertsum_data_processor.tokenize#

modelzoo.transformers.data_processing.bert.bertsum_data_processor.tokenize(params)[source]#

Split sentences and perform tokenization. Takes params.input_path, tokenize it and store it under params.output_path.

previous

modelzoo.transformers.data_processing.bert.bertsum_data_processor.create_parser

next

modelzoo.transformers.data_processing.bert.bertsum_data_processor.BertData