-
Notifications
You must be signed in to change notification settings - Fork 303
Open
Labels
type:BugSomething isn't workingSomething isn't working
Description
Describe the bug
UserWarnings are presented when a mask is propagated via mask_zero=True to TransformerDecoder. Considering this is the default behavior, a warning is confusing and a nuisance.
To Reproduce
import keras_nlp
import numpy as np
seq = keras_nlp.layers.TokenAndPositionEmbedding(3,3,4,mask_zero=True)(np.ones((1,3)))
Observe:
env/keras/src/layers/layer.py:938: UserWarning: Layer 'position_embedding' (of type PositionEmbedding) was passed an input with a mask attached to it. However, this layer does not support masking and will therefore destroy the mask information. Downstream layers will not see the mask.
keras_nlp.layers.TransformerDecoder(8,1)(decoder_sequence=seq)
Observe:
env/keras/src/layers/layer.py:938: UserWarning: Layer 'query' (of type EinsumDense) was passed an input with a mask attached to it. However, this layer does not support masking and will therefore destroy the mask information. Downstream layers will not see the mask.
env/keras/src/layers/layer.py:938: UserWarning: Layer 'key' (of type EinsumDense) was passed an input with a mask attached to it. However, this layer does not support masking and will therefore destroy the mask information. Downstream layers will not see the mask.
env/keras/src/layers/layer.py:938: UserWarning: Layer 'value' (of type EinsumDense) was passed an input with a mask attached to it. However, this layer does not support masking and will therefore destroy the mask information. Downstream layers will not see the mask.
Expected behavior
These warnings should not appear when letting a mask propagate by default. I would only expect to see warnings if I'm doing something unusual or something I should probably not be doing.
Additional context
Others have also encountered this issue:
https://stackoverflow.com/q/79053957
https://stackoverflow.com/q/78868403
Metadata
Metadata
Assignees
Labels
type:BugSomething isn't workingSomething isn't working