-
Notifications
You must be signed in to change notification settings - Fork 460
[BugFix] Fix world size bug in model_runner #2915
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
rebase #2897 to test with the newest CI |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request addresses a bug in the selection of the MoE communication method within the model runner, specifically for scenarios involving expert parallelism on Ascend A2 hardware. The change correctly replaces world_size
with world_size_across_dp
in the condition to enable the 'mc2' communication method. This ensures that the decision is based on the world size relevant to expert parallelism, not the total world size which might include data parallelism dimensions. The accompanying test has been updated to reflect this logic change. The fix appears correct and is well-targeted to resolve the issue described.
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
65ec208
to
048fb5d
Compare
also merged this one #2886 |
- Fix world size bug in model_runner to make sure ep>16 runs with MC2 - enable e2e test for vl Co-Authored-By: whx-sjtu <2952154980@qq.com> Co-Authored-By: Icey <1790571317@qq.com> - vLLM version: v0.10.2 - vLLM main: vllm-project/vllm@3e903b6 Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com> Signed-off-by: offline0806 <z00858301@china.huawei.com>
- Fix world size bug in model_runner to make sure ep>16 runs with MC2 - enable e2e test for vl Co-Authored-By: whx-sjtu <2952154980@qq.com> Co-Authored-By: Icey <1790571317@qq.com> - vLLM version: v0.10.2 - vLLM main: vllm-project/vllm@3e903b6 Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-Authored-By: whx-sjtu 2952154980@qq.com
Co-Authored-By: Icey 1790571317@qq.com