modelzoo.transformers.pytorch.bert.bert_pretrain_models.BertMLMHeadTransform#

class modelzoo.transformers.pytorch.bert.bert_pretrain_models.BertMLMHeadTransform[source]#

Bases: torch.nn.Module

Methods

`forward`
`reset_parameters`

__call__(*args: Any, **kwargs: Any) → Any#: Call self as a function.

__init__(hidden_size=768, embedding_size=None, use_ffn_bias_in_mlm=True, activation='gelu', layer_norm_epsilon=1e-05, dropout=None, kernel_initializer='xavier_uniform')[source]#

static __new__(cls, *args: Any, **kwargs: Any) → Any#

previous

modelzoo.transformers.pytorch.bert.bert_pretrain_models.BertMLMHead

next

modelzoo.transformers.pytorch.bert.bert_pretrain_models.BertPretrainModel