Skip to content

Commit eb43056

Browse files
authored
Add benchmark for fused_linear in llama7b N4C32 dy_auto (#10647)
* Update pretrain-llama2_7b_dynamic_auto.json * Update pretrain-llama2_7b_dynamic_auto.json * Update pretrain-llama2_7b_dynamic_auto.json
1 parent f0eccbd commit eb43056

File tree

1 file changed

+4
-3
lines changed

1 file changed

+4
-3
lines changed

tests/test_tipc/static/auto_parallel/llama2/pretrain_config_llama2_7b/pretrain-llama2_7b_dynamic_auto.json

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,12 +21,13 @@
2121
"fuse_attention_qkv": true,
2222
"use_fused_rope": true,
2323
"fused_linear_param_grad_add": true,
24+
"enable_linear_fused_grad_add": true,
2425
"max_seq_length": 4096,
2526
"learning_rate": 3e-05,
2627
"min_learning_rate": 3e-06,
2728
"warmup_steps": 30,
28-
"logging_steps": 2,
29-
"max_steps": 50,
29+
"logging_steps": 10,
30+
"max_steps": 500,
3031
"save_steps": 5000,
3132
"eval_steps": 1000,
3233
"weight_decay": 0.01,
@@ -52,4 +53,4 @@
5253
"device": "gpu",
5354
"to_static": false,
5455
"enable_auto_parallel": true
55-
}
56+
}

0 commit comments

Comments
 (0)