common.pytorch.model_utils.checkpoint_converters package

Submodules

common.pytorch.model_utils.checkpoint_converters.base_converter module

class common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter

Bases: common.pytorch.model_utils.checkpoint_converters.base_converter.BaseDictionaryConverter, abc.ABC

Converts between checkpoint state_dict formats.

__init__()

classmethod convert(checkpoint, configs, checkpoint_from_index, **kwargs)

convert_helper(checkpoint, configs: Tuple[dict, dict], from_index: int, drop_unmatched_keys: bool = False, no_progress_bar: bool = True, debug: bool = False)

Converts all keys in a checkpoint from from_index format to the other format. Conversion will fail if at least one of the keys did not match on any conversion rules and drop_unmatched_keys is not enabled. Returns the newly converted checkpoint.

abstract static file_formats() → Tuple[str, str]

abstract static get_config_converter_class() → common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

abstract classmethod load(file: str, from_index: int, **kwargs) → collections.OrderedDict

post_checkpoint_convert(checkpoint, from_index: int)

Hook executes after checkpoint conversion.

post_model_convert(old_state_dict: collections.OrderedDict, new_state_dict: collections.OrderedDict, configs: Tuple[dict, dict], from_index: int, drop_unmatched_keys: bool)

Hook executes right after model conversion.

pre_checkpoint_convert(checkpoint, configs: Tuple[dict, dict], from_index: int)

Hook executes before checkpoint conversion.

pre_model_convert(old_state_dict: collections.OrderedDict, new_state_dict: collections.OrderedDict, configs: Tuple[dict, dict], from_index: int, drop_unmatched_keys: bool)

Hook executes right before model conversion.

abstract classmethod save(file_without_ext: str, checkpoint: collections.OrderedDict, from_index: int, **kwargs) → str

class common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

Bases: common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_PT_PT

HF checkpoints contain model only while CS checkpoints package model, optimizer, and lr_scheduler into a single checkpoint. This class overrides the post_checkpoint_convert to automatically extract/package the state_dict correctly.

__init__()

static file_formats() → Tuple[str, str]

post_checkpoint_convert(checkpoint, from_index: int)

Hook executes after checkpoint conversion.

pre_checkpoint_convert(checkpoint, configs: Tuple[dict, dict], from_index: int)

Hook executes before checkpoint conversion.

class common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_PT_PT

Bases: common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter

__init__()

static file_formats() → Tuple[str, str]

classmethod load(file: str, from_index: int) → collections.OrderedDict

classmethod save(file_without_ext: str, checkpoint: collections.OrderedDict, from_index: int, export_h5_checkpoint: bool = False) → collections.OrderedDict

class common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

Bases: common.pytorch.model_utils.checkpoint_converters.base_converter.BaseDictionaryConverter, abc.ABC

__init__()

static assert_factory_fn(assert_index, assert_value)

classmethod convert(config, from_index: int, drop_unmatched_keys: bool = False, no_progress_bar: bool = True, debug: bool = False)

convert_helper(config, from_index: int, drop_unmatched_keys: bool = False, no_progress_bar: bool = True, debug: bool = False)

Converts all keys in a config from from_index format to the other format. Conversion will fail if at least one of the keys did not match on any conversion rules and drop_unmatched_keys is not enabled. Returns the newly converted config.

abstract static file_formats() → Tuple[str, str]

classmethod load(file: str, from_index: int) → dict

post_config_convert(original_config, old_config, new_config, from_index, drop_unmatched_keys)

pre_config_convert(config, from_index)

classmethod save(file_without_ext: str, config: dict, from_index: int) → str

class common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter_CS_CS

Bases: common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

CS packages model, optimizer, and lr_scheduler into a single config. This class overrides the [pre|post]_config_convert fn to automatically extract/package the model configuration correctly.

__init__()

static file_formats() → Tuple[str, str]

post_config_convert(original_config, old_config, new_config, from_index, drop_unmatched_keys)

pre_config_convert(config, from_index)

class common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter_HF_CS

Bases: common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

CS packages model, optimizer, and lr_scheduler into a single config. This class overrides the [pre|post]_config_convert fn to automatically extract/package the model configuration correctly.

__init__()

static file_formats() → Tuple[str, str]

post_config_convert(original_config, old_config, new_config, from_index, drop_unmatched_keys)

pre_config_convert(config, from_index)

class common.pytorch.model_utils.checkpoint_converters.base_converter.BaseDictionaryConverter

Bases: abc.ABC

A dictionary converter represents a pair of two dictionary formats that can be converted between each other. The converter object defines a list of conversion rules which should be applied when converting one dict format to the other (and vice-versa).

In order to make your own dictionary converter, simply: 1. Create a new converter class which inherits from BaseDictionaryConverter 2. Supply a list of conversion rules (self.rules) 3. Override the pre_model_convert or post_model_convert hooks if you

need to execute arbitrary behavior before/after the conversion.

__init__(pbar_desc=None)

convert_all_keys(old_state_dict: collections.OrderedDict, new_state_dict: collections.OrderedDict, from_index: int, action_fn_args: Optional[dict] = None, no_progress_bar: bool = True, debug: bool = False, suppress_unmatched_key_warning: bool = False)

convert_key(old_key: str, old_state_dict: collections.OrderedDict, new_state_dict: collections.OrderedDict, from_index: int, match_start: int = 0, prefix: str = '', action_fn_args: Optional[dict] = None, debug: bool = False) → None

Attempts to convert the old key by matching against the list of conversion rules. The first rule to match is used for conversion (i.e. even if multiple rules would match, the latter ones are never used). Returns True if a conversion occured.

abstract static formats() → Tuple[str, str]

classmethod get_from_index(src_fmt, tgt_fmt)

static replaceKey(old_key: str, new_key: str, old_state_dict: collections.OrderedDict, new_state_dict: collections.OrderedDict, from_index: int, action_fn_args: Optional[dict] = None) → None

Copies value that exists at old_state_dict’s old_key to new_state_dict’s new_key.

classmethod supports_conversion(src_fmt, tgt_fmt)

exception common.pytorch.model_utils.checkpoint_converters.base_converter.ConfigConversionError

Bases: Exception

Raised when a config cannot be converted

class common.pytorch.model_utils.checkpoint_converters.base_converter.ConversionRule

Bases: object

ConversionRule defines a “rule” which:

1. a key can be matched against

2. procedure for converting this old key to a new one upon a sucessful match

3. and an action to be taken once the new key is created (ex: updating the state dictionary)

A rule consists of a sequence of regex pattern (supplied as a string), EquivalentSubkey object, and (possibly) a BaseDictionaryConverter as long as this object is last in the sequence. It also contains an “exists” argument which can be set to “left”, “both”, or “right”. The “left” and “right” arguments are used to describe if a key exists in one checkpoint format but not the other and should be ignored. Without this behavior, keys that exist in one but not the other wouldn’t be matched by any conversion rules, causing a failure as drop_unmatched_keys is disabled by default.

Example: The following describes the conversion rule for mapping HF’s layer normalization key to CS layer normalization in the GPT2 model.

>>> ConversionRule(
>>>     [
>>>         EquivalentSubkey("h", "transformer_decoder.layers"),
>>>         "\.\d+\.",
>>>         EquivalentSubkey("ln_1", "norm1"),
>>>         "\.(weight|bias)",
>>>     ],
>>>     action=BaseCheckpointConverter.replaceKey,
>>> )

This should be interpreted as:

1. HF uses ‘h’ to represent the decoder name while CS uses ‘transformer_decoder.layers’

2. Both will have keys that follow with a dot, the decoder number, and then another dot

3. HF uses ‘ln_1’ for the first layer norm while CS names it ‘norm1’

4. Both will have keys that follow with a dot and then either weight or bias

This representation should make it easy to see how we can 1) build a regex which matches against old keys, and 2) use the matched result & EquivalentSubkey information to create a new key. Finally, once thhe new key is constructed the conversion rule will apply the ‘action’ described by the user in order to complete the conversion (in this case simply copying the value at old_state’s old key into the new_state at the new key).

As previously mentioned, a conversion rule object can also contain a checkpoint converter at the end of the sequence. This is used to create a new checkpoint converter which uses another converter to handle a portion of the conversion. Doing so reduces the amount of copy & pasted conversion rules. For example, many models have base model classes which are extended with additional layers for finetuning. For example, HF’s GP2Model doesn’t contain a language model head while GP2LMHeadModel does. Rather than copying the conversion rules, we could instead define a new checkpoint converter as follows:

>>> class Converter_GPT2LMHeadModel_HF_CS17(BaseDictionaryConverter):
>>>     def __init__(self):
>>>         super().__init__()
>>>         self.rules = [
>>>             ConversionRule(
>>>                 ["lm_head\.(weight|bias)"],
>>>                 action=BaseCheckpointConverter.replaceKey,
>>>             ),
>>>             ConversionRule(
>>>                 [
>>>                     EquivalentSubkey("transformer.", ""),
>>>                     Converter_GPT2Model_HF_CS17(),
>>>                 ],
>>>                 action=None,
>>>             ),
>>>         ]

The first rule simply notates that the lm_head key now exists (and is named the same in both models). The second rule notates that if the “transformer.” prefix is encountered, we should try all of the GPT2Model HF -> CS 1.7 conversion rules.

__init__(segments: List[Union[str, common.pytorch.model_utils.checkpoint_converters.base_converter.EquivalentSubkey, common.pytorch.model_utils.checkpoint_converters.base_converter.BaseDictionaryConverter]], exists: str = 'both', action: Optional[Callable[[str, collections.OrderedDict, str, collections.OrderedDict, int], None]] = None) → None

convert_key(old_key: str, old_state_dict: collections.OrderedDict, new_state_dict: collections.OrderedDict, from_index: int, match_start: int = 0, prefix: str = '', action_fn_args: Optional[dict] = None, debug: bool = False) → bool

exists_in_index(to_index: int) → bool

static segment_is_converter(elm: Union[str, common.pytorch.model_utils.checkpoint_converters.base_converter.EquivalentSubkey, common.pytorch.model_utils.checkpoint_converters.base_converter.BaseDictionaryConverter]) → bool

validate_segments()

class common.pytorch.model_utils.checkpoint_converters.base_converter.EquivalentSubkey

Bases: object

EquivalentSubkey defines the bidirectional relationship between subkeys of a model’s checkpoint. This class is simply a 2-tuple with index bounds checking.

For example if the normalization layer in one model is named “norm” and “ln” in the other, the relationship can be represented as EquivalentSubkey(“norm”, “ln”).

__init__(a: str, b: str) → None

common.pytorch.model_utils.checkpoint_converters.base_converter.converter_notes(notes)

common.pytorch.model_utils.checkpoint_converters.bert module

class common.pytorch.model_utils.checkpoint_converters.bert.ConfigConverter_Bert_CS16_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter_CS_CS

__init__()

static formats() → Tuple[str, str]

class common.pytorch.model_utils.checkpoint_converters.bert.ConfigConverter_Bert_CS16_CS18

Bases: common.pytorch.model_utils.checkpoint_converters.bert.ConfigConverter_Bert_CS16_CS17

__init__()

static formats() → Tuple[str, str]

post_config_convert(original_config, old_config, new_config, from_index, drop_unmatched_keys)

pre_config_convert(config, from_index)

class common.pytorch.model_utils.checkpoint_converters.bert.ConfigConverter_Bert_CS17_CS18

Bases: common.pytorch.model_utils.checkpoint_converters.bert.ConfigConverter_Bert_CS16_CS18

__init__()

static formats() → Tuple[str, str]

class common.pytorch.model_utils.checkpoint_converters.bert.ConfigConverter_Bert_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter_HF_CS

__init__()

assert_mlm_nonlinearity(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

convert_position_embedding_type(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

static formats() → Tuple[str, str]

post_config_convert(original_config, old_config, new_config, from_index, drop_unmatched_keys)

pre_config_convert(config, from_index)

class common.pytorch.model_utils.checkpoint_converters.bert.ConfigConverter_Bert_HF_CS18

Bases: common.pytorch.model_utils.checkpoint_converters.bert.ConfigConverter_Bert_HF_CS17

__init__()

static formats() → Tuple[str, str]

post_config_convert(original_config, old_config, new_config, from_index, drop_unmatched_keys)

pre_config_convert(config, from_index)

class common.pytorch.model_utils.checkpoint_converters.bert.Converter_BertLayerNorm_HF_CS

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__(hf_name, cs_name)

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

class common.pytorch.model_utils.checkpoint_converters.bert.Converter_BertModel_CS16_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_PT_PT

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

position_embeddings_convert(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

class common.pytorch.model_utils.checkpoint_converters.bert.Converter_BertModel_CS16_CS18

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_PT_PT

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

class common.pytorch.model_utils.checkpoint_converters.bert.Converter_BertModel_HF_CS17

Bases: common.pytorch.model_utils.checkpoint_converters.bert.Converter_BertModel_CS16_CS17, modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

post_checkpoint_convert(checkpoint, from_index: int)

pre_model_convert(old_state_dict, new_state_dict, configs, from_index, drop_unmatched_keys)

class common.pytorch.model_utils.checkpoint_converters.bert.Converter_BertModel_HF_CS18

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

class common.pytorch.model_utils.checkpoint_converters.bert.Converter_BertPretrainModel_CS16_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_PT_PT

__init__()

convert_cls_predictions_bias(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

post_checkpoint_convert(checkpoint, from_index: int)

class common.pytorch.model_utils.checkpoint_converters.bert.Converter_BertPretrainModel_CS16_CS18

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_PT_PT

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

post_checkpoint_convert(checkpoint, from_index: int)

class common.pytorch.model_utils.checkpoint_converters.bert.Converter_BertPretrainModel_HF_CS17

Bases: common.pytorch.model_utils.checkpoint_converters.bert.Converter_BertPretrainModel_CS16_CS17, modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

post_checkpoint_convert(checkpoint, from_index: int)

pre_model_convert(old_state_dict, new_state_dict, configs, from_index, drop_unmatched_keys)

class common.pytorch.model_utils.checkpoint_converters.bert.Converter_BertPretrainModel_HF_CS18

Bases: common.pytorch.model_utils.checkpoint_converters.bert.Converter_BertPretrainModel_HF_CS17

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

class common.pytorch.model_utils.checkpoint_converters.bert.Converter_Bert_CS17_CS18

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_PT_PT

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

common.pytorch.model_utils.checkpoint_converters.bert_finetune module

class common.pytorch.model_utils.checkpoint_converters.bert_finetune.ConfigConverter_BertForQuestionAnswering_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.bert.ConfigConverter_Bert_HF_CS17

__init__()

static formats() → Tuple[str, str]

post_config_convert(original_config, old_config, new_config, from_index, drop_unmatched_keys)

pre_config_convert(config, from_index)

class common.pytorch.model_utils.checkpoint_converters.bert_finetune.ConfigConverter_BertForQuestionAnswering_HF_CS18

Bases: common.pytorch.model_utils.checkpoint_converters.bert_finetune.ConfigConverter_BertForQuestionAnswering_HF_CS17, modelzoo.common.pytorch.model_utils.checkpoint_converters.bert.ConfigConverter_Bert_HF_CS18

__init__()

static formats() → Tuple[str, str]

class common.pytorch.model_utils.checkpoint_converters.bert_finetune.ConfigConverter_BertForSequenceClassification_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.bert.ConfigConverter_Bert_HF_CS17

__init__()

static formats() → Tuple[str, str]

pre_config_convert(config, from_index)

class common.pytorch.model_utils.checkpoint_converters.bert_finetune.ConfigConverter_BertForSequenceClassification_HF_CS18

Bases: common.pytorch.model_utils.checkpoint_converters.bert_finetune.ConfigConverter_BertForSequenceClassification_HF_CS17, modelzoo.common.pytorch.model_utils.checkpoint_converters.bert.ConfigConverter_Bert_HF_CS18

__init__()

static formats() → Tuple[str, str]

class common.pytorch.model_utils.checkpoint_converters.bert_finetune.ConfigConverter_BertForTokenClassification_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.bert.ConfigConverter_Bert_HF_CS17

__init__()

static formats() → Tuple[str, str]

post_config_convert(original_config, old_config, new_config, from_index, drop_unmatched_keys)

pre_config_convert(config, from_index)

class common.pytorch.model_utils.checkpoint_converters.bert_finetune.ConfigConverter_BertForTokenClassification_HF_CS18

Bases: common.pytorch.model_utils.checkpoint_converters.bert_finetune.ConfigConverter_BertForTokenClassification_HF_CS17, modelzoo.common.pytorch.model_utils.checkpoint_converters.bert.ConfigConverter_Bert_HF_CS18

__init__()

static formats() → Tuple[str, str]

class common.pytorch.model_utils.checkpoint_converters.bert_finetune.Converter_BertFinetuneModel_CS16_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_PT_PT

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

post_checkpoint_convert(checkpoint, from_index: int)

class common.pytorch.model_utils.checkpoint_converters.bert_finetune.Converter_BertFinetuneModel_CS16_CS18

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_PT_PT

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

post_checkpoint_convert(checkpoint, from_index: int)

class common.pytorch.model_utils.checkpoint_converters.bert_finetune.Converter_BertForQuestionAnswering_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

class common.pytorch.model_utils.checkpoint_converters.bert_finetune.Converter_BertForQuestionAnswering_HF_CS18

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

class common.pytorch.model_utils.checkpoint_converters.bert_finetune.Converter_BertForSequenceClassification_HF_CS17

Bases: common.pytorch.model_utils.checkpoint_converters.bert_finetune.Converter_BertFinetuneModel_CS16_CS17, modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

post_checkpoint_convert(checkpoint, from_index: int)

class common.pytorch.model_utils.checkpoint_converters.bert_finetune.Converter_BertForSequenceClassification_HF_CS18

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

class common.pytorch.model_utils.checkpoint_converters.bert_finetune.Converter_BertForTokenClassification_HF_CS17

Bases: common.pytorch.model_utils.checkpoint_converters.bert_finetune.Converter_BertFinetuneModel_CS16_CS17, modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

post_checkpoint_convert(checkpoint, from_index: int)

class common.pytorch.model_utils.checkpoint_converters.bert_finetune.Converter_BertForTokenClassification_HF_CS18

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

common.pytorch.model_utils.checkpoint_converters.gpt2_hf_cs module

class common.pytorch.model_utils.checkpoint_converters.gpt2_hf_cs.ConfigConverter_GPT2Model_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter_HF_CS

__init__()

convert_attention_type(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

static formats() → Tuple[str, str]

pre_config_convert(config, from_index)

class common.pytorch.model_utils.checkpoint_converters.gpt2_hf_cs.ConfigConverter_GPT2Model_HF_CS18

Bases: common.pytorch.model_utils.checkpoint_converters.gpt2_hf_cs.ConfigConverter_GPT2Model_HF_CS17

__init__()

static formats() → Tuple[str, str]

class common.pytorch.model_utils.checkpoint_converters.gpt2_hf_cs.Converter_GPT2LMHeadModel_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

pre_model_convert(old_state_dict, new_state_dict, configs, from_index, drop_unmatched_keys)

class common.pytorch.model_utils.checkpoint_converters.gpt2_hf_cs.Converter_GPT2LMHeadModel_HF_CS18

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

class common.pytorch.model_utils.checkpoint_converters.gpt2_hf_cs.Converter_GPT2Model_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

post_model_convert(old_state_dict, new_state_dict, configs, from_index, drop_unmatched_keys)

pre_model_convert(old_state_dict, new_state_dict, configs, from_index, drop_unmatched_keys)

replace_final_norm(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

class common.pytorch.model_utils.checkpoint_converters.gpt2_hf_cs.Converter_GPT2Model_HF_CS18

Bases: common.pytorch.model_utils.checkpoint_converters.gpt2_hf_cs.Converter_GPT2Model_HF_CS17

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

class common.pytorch.model_utils.checkpoint_converters.gpt2_hf_cs.Converter_GPT2_Attention_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

assert_already_converted(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

c_attn_converter(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

c_attn_converter_cs17_to_hf(old_key, new_key, old_state_dict, new_state_dict, action_fn_args)

c_attn_converter_hf_to_cs17(old_key, new_key, old_state_dict, new_state_dict, action_fn_args)

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

common.pytorch.model_utils.checkpoint_converters.gpt2_hf_cs.transpose_key_if_2D(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

common.pytorch.model_utils.checkpoint_converters.gpt_neox_hf_cs module

class common.pytorch.model_utils.checkpoint_converters.gpt_neox_hf_cs.ConfigConverter_GPT_Neox_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

post_config_convert(original_config, old_config, new_config, from_index, drop_unmatched_keys)

pre_config_convert(config, from_index)

rotary_dim_converter(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

class common.pytorch.model_utils.checkpoint_converters.gpt_neox_hf_cs.ConfigConverter_GPT_Neox_HF_CS18

Bases: common.pytorch.model_utils.checkpoint_converters.gpt_neox_hf_cs.ConfigConverter_GPT_Neox_HF_CS17

__init__()

static formats() → Tuple[str, str]

class common.pytorch.model_utils.checkpoint_converters.gpt_neox_hf_cs.Converter_GPT_Neox_Attention_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

assert_already_converted(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

interleave_helper(t, cs_config)

qkv_converter(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

qkv_converter_cs17_to_hf(old_key, new_key, old_state_dict, new_state_dict, action_fn_args)

qkv_converter_hf_to_cs17(old_key, new_key, old_state_dict, new_state_dict, action_fn_args)

reverse_interleave_helper(t, cs_config)

class common.pytorch.model_utils.checkpoint_converters.gpt_neox_hf_cs.Converter_GPT_Neox_Headless_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

post_model_convert(old_state_dict, new_state_dict, configs, from_index, drop_unmatched_keys)

pre_model_convert(old_state_dict, new_state_dict, configs, from_index, drop_unmatched_keys)

replace_final_norm(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

class common.pytorch.model_utils.checkpoint_converters.gpt_neox_hf_cs.Converter_GPT_Neox_Headless_HF_CS18

Bases: common.pytorch.model_utils.checkpoint_converters.gpt_neox_hf_cs.Converter_GPT_Neox_Headless_HF_CS17

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

class common.pytorch.model_utils.checkpoint_converters.gpt_neox_hf_cs.Converter_GPT_Neox_LMHeadModel_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

pre_model_convert(old_state_dict, new_state_dict, configs, from_index, drop_unmatched_keys)

class common.pytorch.model_utils.checkpoint_converters.gpt_neox_hf_cs.Converter_GPT_Neox_LMHeadModel_HF_CS18

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

common.pytorch.model_utils.checkpoint_converters.gptj_hf_cs module

class common.pytorch.model_utils.checkpoint_converters.gptj_hf_cs.ConfigConverter_GPTJModel_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter_HF_CS

__init__()

convert_attention_type(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

convert_position_embedding_type(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

static formats() → Tuple[str, str]

post_config_convert(original_config, old_config, new_config, from_index, drop_unmatched_keys)

pre_config_convert(config, from_index)

class common.pytorch.model_utils.checkpoint_converters.gptj_hf_cs.ConfigConverter_GPTJModel_HF_CS18

Bases: common.pytorch.model_utils.checkpoint_converters.gptj_hf_cs.ConfigConverter_GPTJModel_HF_CS17

__init__()

static formats() → Tuple[str, str]

class common.pytorch.model_utils.checkpoint_converters.gptj_hf_cs.Converter_GPTJ_Attention_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

replace_or_fill_masked_bias(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

class common.pytorch.model_utils.checkpoint_converters.gptj_hf_cs.Converter_GPTJ_Headless_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

post_model_convert(old_state_dict, new_state_dict, configs, from_index, drop_unmatched_keys)

pre_model_convert(old_state_dict, new_state_dict, configs, from_index, drop_unmatched_keys)

replace_final_norm(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

class common.pytorch.model_utils.checkpoint_converters.gptj_hf_cs.Converter_GPTJ_Headless_HF_CS18

Bases: common.pytorch.model_utils.checkpoint_converters.gptj_hf_cs.Converter_GPTJ_Headless_HF_CS17

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

class common.pytorch.model_utils.checkpoint_converters.gptj_hf_cs.Converter_GPTJ_LMHeadModel_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

pre_model_convert(old_state_dict, new_state_dict, configs, from_index, drop_unmatched_keys)

class common.pytorch.model_utils.checkpoint_converters.gptj_hf_cs.Converter_GPTJ_LMHeadModel_HF_CS18

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

common.pytorch.model_utils.checkpoint_converters.salesforce_codegen_hf_cs module

class common.pytorch.model_utils.checkpoint_converters.salesforce_codegen_hf_cs.ConfigConverter_Codegen_Model_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

post_config_convert(original_config, old_config, new_config, from_index, drop_unmatched_keys)

pre_config_convert(config, from_index)

rotary_dim_converter(old_key, new_key, old_state_dict, new_state_dict, from_index)

class common.pytorch.model_utils.checkpoint_converters.salesforce_codegen_hf_cs.ConfigConverter_Codegen_Model_HF_CS18

Bases: common.pytorch.model_utils.checkpoint_converters.salesforce_codegen_hf_cs.ConfigConverter_Codegen_Model_HF_CS17

__init__()

static formats() → Tuple[str, str]

class common.pytorch.model_utils.checkpoint_converters.salesforce_codegen_hf_cs.Converter_Codegen_Attention_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

assert_already_converted(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

qkv_converter(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

qkv_converter_cs17_to_hf(old_key, new_key, old_state_dict, new_state_dict, action_fn_args)

qkv_converter_hf_to_cs17(old_key, new_key, old_state_dict, new_state_dict, action_fn_args)

replace_or_fill_masked_bias(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

class common.pytorch.model_utils.checkpoint_converters.salesforce_codegen_hf_cs.Converter_Codegen_Headless_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

post_model_convert(old_state_dict, new_state_dict, configs, from_index, drop_unmatched_keys)

pre_model_convert(old_state_dict, new_state_dict, configs, from_index, drop_unmatched_keys)

replace_final_norm(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

class common.pytorch.model_utils.checkpoint_converters.salesforce_codegen_hf_cs.Converter_Codegen_Headless_HF_CS18

Bases: common.pytorch.model_utils.checkpoint_converters.salesforce_codegen_hf_cs.Converter_Codegen_Headless_HF_CS17

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

class common.pytorch.model_utils.checkpoint_converters.salesforce_codegen_hf_cs.Converter_Codegen_LMHeadModel_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

pre_model_convert(old_state_dict, new_state_dict, configs, from_index, drop_unmatched_keys)

class common.pytorch.model_utils.checkpoint_converters.salesforce_codegen_hf_cs.Converter_Codegen_LMHeadModel_HF_CS18

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

common.pytorch.model_utils.checkpoint_converters.t5 module

class common.pytorch.model_utils.checkpoint_converters.t5.ConfigConverter_T5_CS16_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter_CS_CS

__init__()

static formats() → Tuple[str, str]

class common.pytorch.model_utils.checkpoint_converters.t5.ConfigConverter_T5_CS16_CS18

Bases: common.pytorch.model_utils.checkpoint_converters.t5.ConfigConverter_T5_CS16_CS17

__init__()

static formats() → Tuple[str, str]

class common.pytorch.model_utils.checkpoint_converters.t5.ConfigConverter_T5_CS17_CS18

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter_CS_CS

__init__()

classmethod flip_use_pre_encoder_decoder_layer_norm(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

static formats() → Tuple[str, str]

class common.pytorch.model_utils.checkpoint_converters.t5.ConfigConverter_T5_HF_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter_HF_CS

__init__()

assert_decoder_nonlinearity(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

convert_nonlinearity(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

static formats() → Tuple[str, str]

post_config_convert(original_config, old_config, new_config, from_index, drop_unmatched_keys)

pre_config_convert(config, from_index)

class common.pytorch.model_utils.checkpoint_converters.t5.ConfigConverter_T5_HF_CS18

Bases: common.pytorch.model_utils.checkpoint_converters.t5.ConfigConverter_T5_HF_CS17

__init__()

static formats() → Tuple[str, str]

pre_config_convert(config, from_index)

class common.pytorch.model_utils.checkpoint_converters.t5.Converter_T5_CS16_CS17

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_PT_PT

__init__()

convert_dense_layer(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

convert_relative_attention_bias_cs16_to_cs17(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

convert_relative_attention_bias_cs17_to_cs16(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

post_checkpoint_convert(checkpoint, from_index: int)

class common.pytorch.model_utils.checkpoint_converters.t5.Converter_T5_CS16_CS18

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_PT_PT

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

post_checkpoint_convert(checkpoint, from_index: int)

class common.pytorch.model_utils.checkpoint_converters.t5.Converter_T5_CS17_CS18

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_PT_PT

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

class common.pytorch.model_utils.checkpoint_converters.t5.Converter_T5_HF_CS17

Bases: common.pytorch.model_utils.checkpoint_converters.t5.Converter_T5_CS16_CS17, modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

convert_embeddings(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

convert_relative_attention_bias_cs17_to_hf(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args)

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

post_checkpoint_convert(checkpoint, from_index: int)

pre_model_convert(old_state_dict, new_state_dict, configs, from_index, drop_unmatched_keys)

class common.pytorch.model_utils.checkpoint_converters.t5.Converter_T5_HF_CS18

Bases: modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

__init__()

static formats() → Tuple[str, str]

static get_config_converter_class() → modelzoo.common.pytorch.model_utils.checkpoint_converters.base_converter.BaseConfigConverter

Module contents

previous

common.pytorch.model_utils package

next

common.pytorch.optim package