[New model] Qwen3-next support #2917

wangxiyuan · 2025-09-14T06:45:27Z

What this PR does / why we need it?

Add Qwen3-next support.

Does this PR introduce any user-facing change?

Yes, users can use Qwen3 next.
Related doc: #2916 the tutorial will be ready in here

How was this patch tested?

Doc CI passed

Related: #2884

Co-Authored-By: Angazenn supperccell@163.com
Co-Authored-By: zzzzwwjj 1183291235@qq.com
Co-Authored-By: MengqingCao cmq0113@163.com
Co-Authored-By: linfeng-yuan 1102311262@qq.com
Co-Authored-By: hust17yixuan 303660421@qq.com
Co-Authored-By: SunnyLee219 3294305115@qq.com
Co-Authored-By: maoxx241 maoxx241@umn.edu

vLLM version: v0.10.2
vLLM main: vllm-project/vllm@b834b4c

github-actions · 2025-09-14T06:45:36Z

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

A PR should do only one thing, smaller PRs enable faster reviews.
Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

gemini-code-assist

Code Review

This pull request adds support for the new Qwen3-next model, which is a hybrid attention model. The changes are extensive, touching attention mechanisms, model runner logic, and adding new custom operators. While the implementation is comprehensive, I've identified several critical issues related to code duplication, performance bottlenecks in the new model's prefill implementation, and potential correctness issues due to hardcoded values. There are also some high-severity issues regarding dead code and missed performance optimizations. Addressing these points will significantly improve the robustness and performance of the new model support.

vllm_ascend/models/qwen3_next.py

vllm_ascend/patch/worker/patch_common/patch_block_table.py

vllm_ascend/models/qwen3_next.py

Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

Signed-off-by: Angazenn <supperccell@163.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

Signed-off-by: Angazenn <supperccell@163.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

Signed-off-by: Your Name <you@example.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

Signed-off-by: zzzzwwjj <1183291235@qq.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

vllm_ascend/models/qwen3_next.py

Angazenn · 2025-09-14T09:34:42Z

vllm_ascend/models/qwen3_next.py

+        #     print(f"self.layer_idx: {self.layer_idx}, 111 mixed_qkv_non_spec: {mixed_qkv_non_spec}")
+
+        # 2.1: process the mutli-query part
+        # if spec_sequence_masks is not None:


spec_sequence_masks is not None is used to choose non-MTP branch. Since MTP for qwen3 next is not supported on npu now, maybe we can check it here.

vllm_ascend/worker/npu_input_batch.py

…kv cache Signed-off-by: MengqingCao <cmq0113@163.com>

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

Signed-off-by: hust17yixuan <303660421@qq.com>

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

Yikun

LGTM, some task need to be resolved later

Known issue to be addressed.
The qwen next change should mobe to ops
Remove mamba config patch after vllm merged

### What this PR does / why we need it? Add Qwen3-next support. ### Does this PR introduce _any_ user-facing change? Yes, users can use Qwen3 next. Related doc: vllm-project#2916 the tutorial will be ready in [here](https://vllm-ascend.readthedocs.io/en/latest/tutorials/multi_npu_qwen3_next.html) ### How was this patch tested? Doc CI passed Related: vllm-project#2884 Co-Authored-By: Angazenn <supperccell@163.com> Co-Authored-By: zzzzwwjj <1183291235@qq.com> Co-Authored-By: MengqingCao <cmq0113@163.com> Co-Authored-By: linfeng-yuan <1102311262@qq.com> Co-Authored-By: hust17yixuan <303660421@qq.com> Co-Authored-By: SunnyLee219 <3294305115@qq.com> Co-Authored-By: maoxx241 <maoxx241@umn.edu> - vLLM version: v0.10.2 - vLLM main: vllm-project/vllm@b834b4c --------- Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com> Signed-off-by: Angazenn <supperccell@163.com> Signed-off-by: Your Name <you@example.com> Signed-off-by: zzzzwwjj <1183291235@qq.com> Signed-off-by: linfeng-yuan <1102311262@qq.com> Signed-off-by: hust17yixuan <303660421@qq.com> Co-authored-by: MengqingCao <cmq0113@163.com> Co-authored-by: Angazenn <supperccell@163.com> Co-authored-by: Your Name <you@example.com> Co-authored-by: zzzzwwjj <1183291235@qq.com> Co-authored-by: linfeng-yuan <1102311262@qq.com> Co-authored-by: hust17yixuan <303660421@qq.com>

github-actions bot added module:ops module:core labels Sep 14, 2025

gemini-code-assist bot reviewed Sep 14, 2025

View reviewed changes

vllm_ascend/models/qwen3_next.py Show resolved Hide resolved

vllm_ascend/patch/worker/patch_common/patch_block_table.py Show resolved Hide resolved

vllm_ascend/models/qwen3_next.py Outdated Show resolved Hide resolved

wangxiyuan force-pushed the qwen3-next branch from 271794a to 73a8e05 Compare September 14, 2025 08:02

MengqingCao and others added 17 commits September 14, 2025 16:14

[hybrid kv] init support of hybrid kv

2a21f14

Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

update

90dc42f

Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

init kv cache pass

5b81e48

Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

update

c8b02fc

Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

GDN support

82c472e

Signed-off-by: Angazenn <supperccell@163.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

Update qwen3 moe

4fe8cf9

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

fix registry bugs && remove print

648a3fc

Signed-off-by: Angazenn <supperccell@163.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

repeat qk

ea89e5d

Signed-off-by: Angazenn <supperccell@163.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

fix recurrent_gated_delta_rule

9832095

Signed-off-by: Angazenn <supperccell@163.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

fix

47ef6e6

Signed-off-by: Angazenn <supperccell@163.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

fix seq_lens

fd3e0dd

Signed-off-by: Angazenn <supperccell@163.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

local

0a2beb4

Signed-off-by: Your Name <you@example.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

local 2

f62dda3

Signed-off-by: zzzzwwjj <1183291235@qq.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

local 3

98a3dbb

Signed-off-by: zzzzwwjj <1183291235@qq.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

add vllm patch

4c9d94f

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

fix may_reinitialize_input_batch bug

c0b8af7

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

revert unnecessary

c78e773

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

wangxiyuan force-pushed the qwen3-next branch from 626d6b8 to c78e773 Compare September 14, 2025 08:28

wangxiyuan added ready read for review ready-for-test start test by label for PR labels Sep 14, 2025

wangxiyuan added 2 commits September 14, 2025 17:38

fix lint

8214f81

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

fix lint

f57e114

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

vllm-project deleted a comment from gemini-code-assist bot Sep 14, 2025

Angazenn reviewed Sep 14, 2025

View reviewed changes

Yikun reviewed Sep 14, 2025

View reviewed changes

vllm_ascend/worker/npu_input_batch.py Show resolved Hide resolved

wangxiyuan added ready-for-test start test by label for PR and removed ready-for-test start test by label for PR labels Sep 15, 2025

revert changes on _convert_physical_to_logical_blocks && make use of …

0d59e18

…kv cache Signed-off-by: MengqingCao <cmq0113@163.com>

wangxiyuan removed the ready-for-test start test by label for PR label Sep 15, 2025

fix dict type

cb1b2d6

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

wangxiyuan force-pushed the qwen3-next branch from 62ed628 to cb1b2d6 Compare September 15, 2025 12:07

hust17yixuan and others added 2 commits September 15, 2025 21:10

fix pr2917

4f4d1da

Signed-off-by: hust17yixuan <303660421@qq.com>

skip eagle

60c0bb5

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

wangxiyuan added the ready-for-test start test by label for PR label Sep 15, 2025

make qwen3-test stable

cd5b26f

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

wangxiyuan added ready-for-test start test by label for PR and removed ready-for-test start test by label for PR labels Sep 15, 2025

fix prefix cache and ep

731e588

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

wangxiyuan added ready-for-test start test by label for PR and removed ready-for-test start test by label for PR labels Sep 15, 2025

wangxiyuan added 4 commits September 15, 2025 23:14

fix qwen3-next

9de86f5

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

fix lint

9f0ac45

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

fix lint

fc99a05

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

fix deepseek prefix cache

2d6582c

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

wangxiyuan added ready-for-test start test by label for PR and removed ready-for-test start test by label for PR labels Sep 15, 2025

disable prefix test

9b111ab

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

Yikun approved these changes Sep 15, 2025

View reviewed changes

Yikun merged commit c556038 into vllm-project:main Sep 15, 2025
14 checks passed

Yikun added ready-for-test start test by label for PR and removed ready-for-test start test by label for PR accuracy-test enable all accuracy test for PR labels Sep 15, 2025

Yikun mentioned this pull request Sep 20, 2025

[Bug]: Remove outofdate commits to improve perf test #3051

Open

89412825 mentioned this pull request Sep 20, 2025

[Usage]: version 0.10.2版本910B3*4 推理 Qwen3-Next-80B-A3B性能很差是什么原因？ #3061

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[New model] Qwen3-next support #2917

[New model] Qwen3-next support #2917

wangxiyuan commented Sep 14, 2025 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Sep 14, 2025

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Angazenn Sep 14, 2025

Uh oh!

Uh oh!

Yikun left a comment

Uh oh!

Uh oh!

Uh oh!

[New model] Qwen3-next support #2917

[New model] Qwen3-next support #2917

Conversation

wangxiyuan commented Sep 14, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

github-actions bot commented Sep 14, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Angazenn Sep 14, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Yikun left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

wangxiyuan commented Sep 14, 2025 •

edited by github-actions bot

Loading