Skip to content

Commit df75876

Browse files
committed
expose bucket sizes for researchers to play around with
1 parent 9ecb55a commit df75876

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

train.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,9 @@ def decode_tokens(tokens):
4343
depth = 8,
4444
heads = 8,
4545
causal = True,
46-
memory_efficient = True
46+
memory_efficient = True,
47+
q_bucket_size = 512,
48+
k_bucket_size = 512
4749
)
4850

4951
model = AutoregressiveWrapper(model)

0 commit comments

Comments
 (0)