Skip to content

Commit 38f75d5

Browse files
committed
Updated readme
1 parent 8e2f0f9 commit 38f75d5

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ import torch
3333
model = Linformer(
3434
input_size=262144, # Dimension 1 of the input
3535
channels=64, # Dimension 2 of the input
36-
dim_d=256, # The inner dimension of the attention heads
36+
dim_d=None, # Overwrites the inner dim of the attention heads. If None, sticks with the recommended channels // nhead, as in the "Attention is all you need" paper
3737
dim_k=128, # The second dimension of the P_bar matrix from the paper
3838
dim_ff=128, # Dimension in the feed forward network
3939
dropout_ff=0.15, # Dropout for feed forward network

0 commit comments

Comments
 (0)