.. _supported-pytorch-optimizers:

Supported PyTorch Optimizers
============================

These are drop-in replacements from ``modelzoo.common.optim.*`` and can replace ``torch.nn.optim.*`` without any modifications.

The following optimizers work only in weight streaming mode:

`modelzoo.common.optim.Adadelta <https://pytorch.org/docs/1.11/generated/torch.optim.Adadelta.html#torch.optim.Adadelta>`_
  Implements Adadelta algorithm.

`modelzoo.common.optim.Adagrad <https://pytorch.org/docs/1.11/generated/torch.optim.Adagrad.html#torch.optim.Adagrad>`_
  Implements Adagrad algorithm.

`modelzoo.common.optim.Adam <https://pytorch.org/docs/1.11/generated/torch.optim.Adam.html#torch.optim.Adam>`_
  Implements Adam algorithm.

`modelzoo.common.optim.AdamW <https://pytorch.org/docs/1.11/generated/torch.optim.AdamW.html#torch.optim.AdamW>`_
  Implements AdamW algorithm.

`modelzoo.common.optim.Adamax <https://pytorch.org/docs/1.11/generated/torch.optim.Adamax.html#torch.optim.Adamax>`_
  Implements Adamax algorithm (a variant of Adam based on infinity norm).

`modelzoo.common.optim.ASGD <https://pytorch.org/docs/1.11/generated/torch.optim.ASGD.html#torch.optim.ASGD>`_
  Implements Averaged Stochastic Gradient Descent.

`modelzoo.common.optim.NAdam <https://pytorch.org/docs/1.11/generated/torch.optim.NAdam.html#torch.optim.NAdam>`_
  Implements NAdam algorithm.

`modelzoo.common.optim.RAdam <https://pytorch.org/docs/1.11/generated/torch.optim.RAdam.html#torch.optim.RAdam>`_
  Implements RAdam algorithm.

`modelzoo.common.optim.RMSProp <https://pytorch.org/docs/1.11/generated/torch.optim.RMSprop.html#torch.optim.RMSprop>`_
  Implements RMSprop algorithm.

`modelzoo.common.optim.Rprop <https://pytorch.org/docs/1.11/generated/torch.optim.Rprop.html#torch.optim.Rprop>`_
  Implements the resilient backpropagation algorithm.

`modelzoo.common.optim.SGD <https://pytorch.org/docs/1.11/generated/torch.optim.SGD.html#torch.optim.SGD>`_
  Implements stochastic gradient descent (optionally with momentum).