.. _InverseSquareRootDecayLR:

InverseSquareRootDecayLR
========================

class ``modelzoo.common.pytorch.optim.lr_scheduler.InverseSquareRootDecayLR`` (optimizer: torch.optim.optimizer.Optimizer, initial_learning_rate: float, scale: float, warmup_steps: int, disable_lr_steps_reset: bool = False)

Decays the learning rate inverse-squareroot over time, as described in the following equation:
  
    .. _inverse-square-root-decay-LR.png:

    .. figure:: ../../../images/inverse-square-root-decay-LR.png
        :align: center
        :width: 250 px

Parameters:
  **optimizer** – The optimizer to schedule

  **initial_learning_rate** – The initial learning rate

  **scale** – Multiplicative factor to scale the result

  **warmup_steps** – use initial_learning_rate for the first warmup_steps