cerebras.modelzoo.losses.BertPretrainModelLoss.BertPretrainModelLoss

class cerebras.modelzoo.losses.BertPretrainModelLoss.BertPretrainModelLoss

Bases: torch.nn.Module

Methods

forward

__init__(disable_nsp=False, mlm_loss_weight=1.0, label_smoothing=0.0)

__call__(*args: Any, **kwargs: Any) → Any

Call self as a function.

static __new__(cls, *args: Any, **kwargs: Any) → Any

previous

cerebras.modelzoo.losses.BertPretrainModelLoss

next

cerebras.modelzoo.losses.DPOLoss