Skip to content

[BugFix] Fix world size bug in model_runner. #2897

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

[BugFix] Fix world size bug in model_runner. #2897

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
37c030a
fix world size
whx-sjtu Sep 12, 2025
55a29df
fix ut bug
whx-sjtu Sep 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion vllm_ascend/worker/model_runner_v1.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1539,7 +1539,7 @@ def _select_moe_comm_method(self, num_tokens: int) -> str:
if not self.parallel_config.enable_expert_parallel:
moe_comm_method = "allgather"
elif soc_version in {AscendSocVersion.A2}:
if num_tokens <= self.mc2_tokens_capacity and self.parallel_config.world_size >= 16:
if num_tokens <= self.mc2_tokens_capacity and self.parallel_config.world_size_across_dp >= 16:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

While the change to use world_size_across_dp is correct for selecting the MoE communication method in a data-parallel setup, the corresponding unit test needs to be updated.

The test test_select_moe_comm_method in tests/ut/worker/test_model_runner_v1.py still mocks parallel_config.world_size and will likely fail or pass incorrectly after this change.

To ensure this bug fix is properly tested, please update the unit test to mock parallel_config.world_size_across_dp instead.

moe_comm_method = "mc2"
else:
moe_comm_method = "allgather"
Expand Down
Loading