PyTorch Gradient Clipping

Gradient clipping is supported for PyTorch. Both clipping the gradient norms and gradient values are supported. For example:

torch.nn.utils.clip_grad_norm_(
    model.parameters(), max_gradient_norm,
)
## OR ##
torch.nn.utils.clip_grad_value_(
    model.parameters(), max_gradient_value,
)