modelzoo.common.pytorch.model_utils.activations.gelu_new#

modelzoo.common.pytorch.model_utils.activations.gelu_new(x)[source]#

Implementation of the GELU activation function currently in Google BERT repo (identical to OpenAI GPT). Also see the Gaussian Error Linear Units paper: https://arxiv.org/abs/1606.08415

previous

modelzoo.common.pytorch.model_utils.activations.gelu_fast

next

modelzoo.common.pytorch.model_utils.activations.get_activation