modelzoo.common.pytorch.model_utils#

BertPretrainModelLoss

DPOLoss

GPTLMHeadModelLoss

RotaryPositionEmbeddingHelper

T5ForConditionalGenerationLoss

activations

checkpoint_converters

convert_checkpoint

convert_config_to_mup

This script uses user provided SP GPT-2/3 yaml config to calculate and then generate a correspoding muP yaml config.

create_initializer

norms

vocab_utils

weight_initializers