cerebras.modelzoo.common.optim_utils.configure_param_groups#

cerebras.modelzoo.common.optim_utils.configure_param_groups(model, optimizer_params)[source]#

Groups the optimizer parameters into non-overlapping groups. The groups are formed along the two axis: i) if weight_decay is >0 or not for the param ii) unique adjust_learning_rate (learning_rate_scaling) for the param

Parameters

  • model (dict) – Pytorch model

  • optimizer_params (dict) – optimizer paramters

Returns

_description_

Return type

_type_

previous

cerebras.modelzoo.common.optim_utils

next

cerebras.modelzoo.common.optim_utils.flatten_optimizer_params