cerebras.modelzoo.data_preparation.utils.get_vocab#

cerebras.modelzoo.data_preparation.utils.get_vocab(vocab_file_path, do_lower)[source]#

Function to generate vocab from provided vocab_file_path.

Parameters

  • vocab_file_path (str) – Path to vocab file

  • do_lower (bool) – If True, convert vocab words to lower case.

Returns List[str]

list containing vocab words.

previous

cerebras.modelzoo.data_preparation.utils.get_output_type_shapes

next

cerebras.modelzoo.data_preparation.utils.pad_input_sequence