Roar forward stats.

[mengzhu28]

https://docs.google.com/spreadsheets/d/1lnT_-W4eZpoOwwjGgcFCOf9ReewPgrEt_rDQZRgNeYE/edit?gid=0#gid=0

Benchmark Results Table:
  method  size         MS        tag
 prefill  8192 674.950054 WARMUPSTAT
 prefill  7168 566.033595 WARMUPSTAT
 prefill  6144 470.009576 WARMUPSTAT
 prefill  5120 425.377196 WARMUPSTAT
 prefill  4096 302.061990 WARMUPSTAT
 prefill  3072 229.366926 WARMUPSTAT
 prefill  2048 146.780431 WARMUPSTAT
 prefill  1024  74.560695 WARMUPSTAT
 prefill   896  85.256646 WARMUPSTAT
 prefill   768  60.854283 WARMUPSTAT
 prefill   640  69.717665 WARMUPSTAT
 prefill   512  38.642006 WARMUPSTAT
 prefill   384  47.477070 WARMUPSTAT
 prefill   256  22.688638 WARMUPSTAT
decoding   128  21.787058 WARMUPSTAT
decoding   120  21.764661 WARMUPSTAT
decoding   112  22.144440 WARMUPSTAT
decoding   104  22.091502 WARMUPSTAT
decoding    96  22.057195 WARMUPSTAT
decoding    88  22.259244 WARMUPSTAT
decoding    80  22.165275 WARMUPSTAT
decoding    72  22.389526 WARMUPSTAT
decoding    64  19.645658 WARMUPSTAT
decoding    56  20.439255 WARMUPSTAT
decoding    48  20.169216 WARMUPSTAT
decoding    40  18.859967 WARMUPSTAT
decoding    32  18.403400 WARMUPSTAT
decoding    24  18.285265 WARMUPSTAT
decoding    16  17.784913 WARMUPSTAT
decoding     8  17.607861 WARMUPSTAT
decoding     4  17.564471 WARMUPSTAT

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀