Skip to content

Updates on vLLM benchmarking, MoE tuning, static lunch grids #27

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 64 commits into
base: main
Choose a base branch
from
Open

Updates on vLLM benchmarking, MoE tuning, static lunch grids #27

wants to merge 64 commits into from

Conversation

@bringlein
Copy link
Collaborator

No description provided.

bringlein and others added 30 commits July 16, 2025 11:09
@bringlein
rename script
bbb8610 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
getting flashinfer installed in container
d53287a 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
getting offline script to run
720dc2a 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
triton dejavu for granite 4
7e8c19c 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
adding cache to moe config script
0223476 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
moving fmwork to third_party
75f3877 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
benchmark moe fallback heuristics
da56c2e 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
implemented fused moe benchmark
fd8531c 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
first tuning result
dbddbb1 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
getting moe benchmark to run
e324ca4 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
tuning remaining parts...log lost due to restart
b7161b7 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
benchmarking fused_moe
8c7acf4
@bringlein
setup ttft sweeps
ba3ef68 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
first tuning night
2e129b8 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
adding tune log for analysis
b37a420 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
testing for longer sequences
5a31ca4 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
adding cache for baseline
fd1f12e 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
measuring full cudagraphs, adding tuned fp8 moe
600f5d8 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
itl tuning
408b462 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
improving benchmark latency script
8b0994f 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
adding simple heuristics to unit tests
023a5a5 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
making path relative
322eae9 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
also looking at decode
bbce518 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein @jvlunteren
Adding kernels with flexible tiles
cc9327d 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Jan Van Lunteren <jvl@zurich.ibm.com>
@bringlein
updating dejavu version
ad50b39 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
updating serving range script
80e9ea0 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
preparing cuda graph catpure
0b64697 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
some fixes
a2f635e 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
fixing flash_attn alternating
e31fc91 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
fix sweep script
0dda12d 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
bringlein and others added 29 commits August 12, 2025 10:05
@bringlein
starting ws experiments
af5befc 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
another schema for fp8 tuning
b2c94f1 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
first ws mb
b510a5e 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
fix tuning error
0128071 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
making it compatible with cuda graphs
836257c 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein @jvlunteren
adding kernels with static launch grid
c1df8c6 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Jan Van Lunteren <jvl@zurich.ibm.com>
@bringlein
further ws tuning for simple kernel; add tuning for new grid kernels
cce415a 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
tuning grid w/o ws
e221205 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
grid tuning with ws, preparation
19a73a8 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
tuning forcing ws (partially failing)
16d64d8 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
allowing ws or not
2057fa4 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
tuning with wrong ws config
ffe0346 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
autotuning with the right bohb spaces
d757651 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
preparing to run without bo
ebc7a41 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
switching to random search
41120d7 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
switching back to bo
adc8ccf 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
another run without ws
7eb1410 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
preparing tuning on MI300
64d6d33 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
Merge branch 'ngl_granite4_cudagraph' of github.com:foundation-model-…
f0713ca 
…stack/vllm-triton-backend into ngl_granite4_cudagraph
@bringlein
tuning 30min on H100
c68dd28 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
tuning 30min MI300
429ddd5 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
Merge branch 'ngl_granite4_cudagraph' of github.com:foundation-model-…
211e655 
…stack/vllm-triton-backend into ngl_granite4_cudagraph
@bringlein
baseline experiments
40f0dfb 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
last paper experiments
45056f5 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
updating fmwork pointer
5be31b1 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
some cleanup
0eea195 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
some cleanup
b05e851 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
@bringlein
Merge branch 'ngl_pr_2025-09-03' of github.com:foundation-model-stack…
3713282 
…/vllm-triton-backend into ngl_pr_2025-09-03
@bringlein
further cleanup
f3f623e 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
bringlein added a commit that referenced this pull request Sep 3, 2025
@bringlein
Updates on vLLM benchmarking, MoE tuning, static lunch grids (#27)
c03ee59 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@bringlein