modelzoo.transformers.pytorch.bert.input.utils.get_whole_word_span#

modelzoo.transformers.pytorch.bert.input.utils.get_whole_word_span(tokens, start_index)[source]#

Returns the whole word start and end indices.

Param

list tokens: Tokens to process.

Param

int start_index: Start index.

Returns

tuple with start and end index of the word from the token list.