modelzoo.transformers.data_processing.huggingface.CSDataCollatorForLanguageModeling.CSDataCollatorForLanguageModeling#

class modelzoo.transformers.data_processing.huggingface.CSDataCollatorForLanguageModeling.CSDataCollatorForLanguageModeling[source]#

Bases: transformers.DataCollatorForLanguageModeling

Overrides DataCollatorForLanguageModeling from HF to shift the inputs/labels in the dataloader

Methods

torch_call

__call__(*args: Any, **kwargs: Any) Any#

Call self as a function.

__init__(*args: Any, **kwargs: Any) None#
static __new__(cls, *args: Any, **kwargs: Any) Any#