modelzoo.transformers.pytorch.bert.input.utils.get_whole_word_span

modelzoo.transformers.pytorch.bert.input.utils.get_whole_word_span(tokens, start_index)

Returns the whole word start and end indices.

Param

list tokens: Tokens to process.

Param

int start_index: Start index.

Returns

tuple with start and end index of the word from the token list.

previous

modelzoo.transformers.pytorch.bert.input.utils.get_meta_data

next

modelzoo.transformers.pytorch.bert.input.utils.parse_text