modelzoo.transformers.pytorch.gptj.fine_tuning.continuous_pretraining.data_processing.split_trc_dataset

Split input src file into multiple files. Number of examples per file is controlled by buffer_len param.

Functions

main

previous

modelzoo.transformers.pytorch.gptj.fine_tuning.continuous_pretraining.data_processing

next

modelzoo.transformers.pytorch.gptj.fine_tuning.continuous_pretraining.data_processing.split_trc_dataset.main