[P/D] force with_prefill true after allreduce in kv producer

[liziyu179]

What this PR does / why we need it?

force with_prefill true after allreduce in kv producer

Does this PR introduce any user-facing change?

How was this patch tested?

• vLLM version: v0.11.0rc3
• vLLM main: vllm-project/vllm@c9461e0

[gemini-code-assist]

Code Review

This pull request correctly moves the logic for forcing with_prefill = True on KV producer nodes to be executed before the _sync_metadata_across_dp call. This change is crucial as it ensures that the with_prefill state is consistently synchronized across all data-parallel ranks. By doing so, it prevents potential deadlocks or correctness issues that could arise from inconsistent states during distributed dummy runs. The modification is a clean and effective fix for a critical distributed logic issue.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

[yiz-liu]

We should remove this with_prefill = True once attn_state is refactored or removed.

[linfeng-yuan]

LGTM. It doesn't affect torchair scenario since we always expect with_prefill to be True for P node.