cerebras.modelzoo.data_preparation.nlp.chunk_data_processing.dpo_data_token_generator#
Classes
Initialize the DPTokenGenerator class with dataset parameters, tokenizer, and token IDs. |
Classes
Initialize the DPTokenGenerator class with dataset parameters, tokenizer, and token IDs. |