Skip to content

Conversation

hust17yixuan
Copy link
Contributor

@hust17yixuan hust17yixuan commented Sep 24, 2025

What this PR does / why we need it?

the multistream moe in tochari only validate in decode, but can't be applied to chunked prefill, So add some judgments to isolate the scenario

Does this PR introduce any user-facing change?

No

How was this patch tested?

Copy link

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

  • A PR should do only one thing, smaller PRs enable faster reviews.
  • Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
  • Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request aims to fix a bug in multistream Mixture of Experts (MoE) for the DeepseekV2 model. The changes ensure that a specific communication optimization (mla_moe_communication) is only active during the decode phase, which appears to be the correct behavior. The implementation is refactored to use a centralized forward context to determine the prefill/decode state, which improves consistency. However, this refactoring has introduced a dead parameter in one of the modified functions, which poses a maintainability risk. My review includes a comment to address this issue.

Comment on lines +169 to 171
if is_force_scatter or (not forward_context.with_prefill
and output_parallel.shape[0] % self.tp_size
== 0):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using forward_context.with_prefill makes the is_prefill parameter of this forward method unused. This is misleading and can lead to future bugs if other developers rely on this parameter, expecting it to have an effect.

To improve maintainability and prevent potential issues, the is_prefill parameter should be removed from the function signature of TorchairDeepseekV2RowParallelLinearReplaceAllreduce.forward. This will require updating the call site in vllm_ascend/attention/mla_v1.py to no longer pass this argument.

@hust17yixuan hust17yixuan changed the title [bugfix]fix multistream moe [bugfix]fix multistream moe in torchair Sep 25, 2025
Signed-off-by: hust17yixuan <303660421@qq.com>
@wangxiyuan wangxiyuan added the ready read for review label Sep 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready read for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants