.rst .pdf

tf.BahdanauAttention module

class tf.BahdanauAttention.BahdanauAttention(*args: Any, **kwargs: Any)

Bases: modelzoo.common.layers.tf.BaseLayer.BaseLayer

Bahdanau (additive) Attention with normalization.

Parameters

• hidden_size (int) – Number of units.

• initializer – Initializer.

• weight_regularizer (callable) – Weights regularizer.

• normalize (bool) – Whether to normalize the “energy” of the attention, as described in the paper: Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks.

build(input_shape)

call(q, v, mask=None)

Applies the attention mechanism to queries q and values v. Keys will be set to be same as v.

Parameters

• q (Tensor) – Queries tensor with the shape [batch_size,max_seq_len, rnn_hidden_size].

• v (Tensor) – Values tensor with the shape [batch_size,max_seq_len, rnn_hidden_size].

• mask – Attention mask tensor of shape [batch_size, max_seq_len] or [batch, q_length, length]. Padding mask is 2D. Future mask is 3D and is used to mask out the subsequent tokens inside the encoder-decoder attention module.

Returns

Bahdanau attention layer output with shape [batch_size,max_seq_len, rnn_hidden_size].

class tf.BahdanauAttention.SelfBahdanauAttention(*args: Any, **kwargs: Any)

Bases: tf.BahdanauAttention.BahdanauAttention

Bahdanau self-attention layer.

call(x, mask=None)

Applies the attention mechanism to queries q and values v. Keys will be set to be same as v.

Parameters

• q (Tensor) – Queries tensor with the shape [batch_size,max_seq_len, rnn_hidden_size].

• v (Tensor) – Values tensor with the shape [batch_size,max_seq_len, rnn_hidden_size].

• mask – Attention mask tensor of shape [batch_size, max_seq_len] or [batch, q_length, length]. Padding mask is 2D. Future mask is 3D and is used to mask out the subsequent tokens inside the encoder-decoder attention module.

Returns

Bahdanau attention layer output with shape [batch_size,max_seq_len, rnn_hidden_size].

previous

tf.AttentionLayer module

next

tf.BaseLayer module