modelzoo.common.pytorch.model_utils.checkpoint_converters.mup.Converter_sP_muP#

class modelzoo.common.pytorch.model_utils.checkpoint_converters.mup.Converter_sP_muP[source]#

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseDictionaryConverter

Transforms a CS muP checkpoints into a CS sP checkpoint.

muP: Maximal Update Parametrization. sP: Standard Parametrization.

Methods

convert_all_keys

convert_key

Attempts to convert the old key by matching against the list of conversion rules.

formats

get_from_index

is_mup

replaceKey

Copies value that exists at old_state_dict's old_key to new_state_dict's new_key.

scale_embedding_layernorm

scale_embeddings

scale_k_projection

scale_lm_head

supports_conversion

__init__()[source]#
convert_key(old_key: str, old_state_dict: collections.OrderedDict, new_state_dict: collections.OrderedDict, from_index: int, match_start: int = 0, prefix: str = '', action_fn_args: Optional[dict] = None, debug: bool = False) → None#

Attempts to convert the old key by matching against the list of conversion rules. The first rule to match is used for conversion (i.e. even if multiple rules would match, the latter ones are never used). Returns True if a conversion occurred.

static replaceKey(old_key: str, new_key: str, old_state_dict: collections.OrderedDict, new_state_dict: collections.OrderedDict, from_index: int, action_fn_args: Optional[dict] = None) → None#

Copies value that exists at old_state_dict’s old_key to new_state_dict’s new_key.

previous

modelzoo.common.pytorch.model_utils.checkpoint_converters.mup.ConfigConverter_sP_muP

next

modelzoo.common.pytorch.model_utils.checkpoint_converters.roberta