[Feature] Optimize forward metadata collection across dp ranks

[jianzs]

What this PR does / why we need it?

This PR introduces two optimizations for cases where data parallel size > 1:

1. Eliminates DP communication in set_forward_context
2. Implements HCCL for DP metadata communication, resulting in significant performance improvements for large DP configurations
   • Achieves ~20ms latency reduction with DP size of 64

Does this PR introduce any user-facing change?

no

How was this patch tested?

CI passed.

• vLLM version: v0.9.2
• vLLM main: vllm-project/vllm@91b3d19

[jianzs]

@NeverRaR PTAL

[NeverRaR]

lgtm

[Copilot]

Pull Request Overview

This PR optimizes how forward-pass metadata is collected and communicated across data-parallel ranks by removing the previous all-reduce and introducing an HCCL-based all-gather approach.

• Enforce that dummy batch execution only runs under data parallelism and refactor execute_dummy_batch to use per-rank metadata.
• Replace dist.all_reduce with HCCL all_gather in _get_forward_metadata_across_dp and update callers to handle the Tensor of per-rank token counts.
• Propagate num_tokens_across_dp through dummy runs and forward contexts, masking sentinel values before the pass.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
vllm_ascend/worker/worker_v1.py	Added assertion for dp_size > 1, refactored dummy-run logic to use HCCL per-rank metadata.
vllm_ascend/worker/model_runner_v1.py	Swapped all_reduce for all_gather under get_dp_group(), changed method signature and updated callers to handle a Tensor of metadata.

Comments suppressed due to low confidence (1)

vllm_ascend/worker/model_runner_v1.py:622

• Add unit or integration tests covering the dp_size > 1 aggregation path to verify that all_gather produces the correct combined metadata and that the masked_fill_ logic correctly replaces sentinel values.

            local_forward_metadata)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 54.93%. Comparing base (c30ddb8) to head (d35cbf3).
Report is 141 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #1593       +/-   ##
===========================================
+ Coverage   27.39%   54.93%   +27.53%     
===========================================
  Files          56       80       +24     
  Lines        6191     9712     +3521     
===========================================
+ Hits         1696     5335     +3639     
+ Misses       4495     4377      -118     

Flag	Coverage Δ
unittests	54.93% <ø> (+27.53%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jianzs]

@Yikun @wangxiyuan @ApsarasX @ganyi1996ppo ready to merge.

[Angazenn]

why is it 1?

[jianzs]

If graph mode is off, a dummy run only needs to be executed; computational requirements are not a factor.

[jianzs]

@Angazenn PTAL

[jianzs]

@wangxiyuan @ganyi1996ppo @Yikun ready to merge.

[wangxiyuan]

torchair has made the code more and more complex and hard to maintain, I have a PR(#1661) to add torchair module, all torchair related code can be updated and changed there, I'll make the PR avaliable for review soon. Before that, I really don't want to merge anything about torchair change. Because it's very hard to review(I'm not sure if the change breaked anything else), Sorry.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[Yikun]

Wrong to submit the branch, pls feel free to open new PR.