Skip to content

Commit 66e6eab

Browse files
HERIUNqgallouedec
andauthored
✅ [doc] Update sft_trainer.md in table x->✓ (#3313)
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
1 parent 27af0aa commit 66e6eab

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

docs/source/sft_trainer.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -424,9 +424,9 @@ Below are some numbers you can get in terms of speedup and memory efficiency, us
424424

425425
| use_flash_attn_1 | model_name | max_seq_len | batch_size | time per training step |
426426
| ---------------- | ----------------- | ----------- | ---------- | ---------------------- |
427-
| x | facebook/opt-350m | 2048 | 8 | ~59.1s |
427+
| | facebook/opt-350m | 2048 | 8 | ~59.1s |
428428
| | facebook/opt-350m | 2048 | 8 | **OOM** |
429-
| x | facebook/opt-350m | 2048 | 4 | ~30.3s |
429+
| | facebook/opt-350m | 2048 | 4 | ~30.3s |
430430
| | facebook/opt-350m | 2048 | 4 | ~148.9s |
431431

432432
### Using Flash Attention-2

0 commit comments

Comments
 (0)