[BugFix] Fix dummy_run memory explosion in eager mode

[dragondream-chen]

What this PR does / why we need it?

It is a quick bugfix for the memory explosion issue that requires further refactoring.
The dummy_run in eager mode may lead to OOM and the reason is that hidden_states were not released in time.
The PR temporarily resolves the issue by manually clearing the cache, and further refactoring will be conducted subsequently.

Before the modification, the dummy_run's memory showed an accumulation issue.

After modification, it can be observed that the memory is released promptly.

And it was verified that the model responded normally after a single data input.

• vLLM version: v0.10.2
• vLLM main: vllm-project/vllm@b106890

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

[gemini-code-assist]

Code Review

This pull request addresses a critical memory explosion issue in the single-operator dummy_run by explicitly deleting self.split_hidden_states after its use to release memory. The fix is applied in both FusedMoEPrepareAndFinalizeWithMC2 and FusedMoEPrepareAndFinalizeWithAll2All classes. The change is correct and effectively resolves the memory leak. My review includes a suggestion to refactor the duplicated finalize method into a common base class to improve maintainability and reduce the risk of future inconsistencies, which you can consider during the planned subsequent restructuring.

[gemini-code-assist]

While this change correctly fixes the memory leak, I noticed that the finalize method in FusedMoEPrepareAndFinalizeWithAll2All is identical to the one in FusedMoEPrepareAndFinalizeWithMC2. This code duplication, including the bug fix you are applying, increases maintenance overhead and the risk of introducing inconsistencies in the future. Since you've already noted that further restructuring is required, I recommend abstracting this common finalize logic into a shared base class as part of that effort. This will improve maintainability.

[yiz-liu]

A friendly reminder is that you can use "eager mode" instead of "single-operator" (which is not a thing, kinda like google translation lol).

[realliujiaxu]

Consider not assigning self.split_hidden_states during the prepare phase. Instead, use get_tp_group().allgather() in the finalize phase, which internally employs allgather_into_tensor to eliminate tensor move overhead.