cerebras.modelzoo.common.utils.model.vocab_utils.get_vocab_size#
- cerebras.modelzoo.common.utils.model.vocab_utils.get_vocab_size(vocab_file, vocab_size=None)[source]#
Function to get vocab size and validate with vocabulary file. :params str vocab_file: Path to vocabulary file. :params int vocab_size: Size of vocabulary file. :returns integer value indicating the size of vocabulary file.