-
Notifications
You must be signed in to change notification settings - Fork 544
[Bugfix] fix sleepmode level2 e2e test #4019
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request fixes a failing E2E test for sleep_mode_level=2 by disabling the VLLM_ASCEND_ENABLE_NZ optimization for that specific test. While this makes the test pass, it appears to be a workaround for an underlying incompatibility between sleep mode level 2's weight reloading mechanism and the FRACTAL_NZ weight format. My review points out this issue and suggests that the root cause should be addressed to ensure full test coverage for this feature combination. I have also recommended adding a code comment to explain the workaround.
| assert proc.returncode == 0 | ||
|
|
||
|
|
||
| @patch.dict(os.environ, {"VLLM_ASCEND_ENABLE_NZ": "0"}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Disabling VLLM_ASCEND_ENABLE_NZ makes this test pass, but it appears to be a workaround for a deeper issue. This change effectively prevents testing the sleep_mode_level=2 functionality when the FRACTAL_NZ weight format optimization is enabled, reducing test coverage.
The root cause seems to be that when sleep_mode_level=2 is used, the test manually reloads model weights from disk (see examples/offline_external_launcher.py, line 250). This reloading process does not account for the FRACTAL_NZ format that weights are converted to when VLLM_ASCEND_ENABLE_NZ=1. Consequently, the model receives weights in an unexpected format, leading to failure.
While this fix may be acceptable to unblock CI, the underlying incompatibility should be addressed. Possible long-term solutions include:
- Updating the weight reloading logic in the test to correctly handle the
FRACTAL_NZformat. - If
sleep_mode_level=2is fundamentally incompatible withVLLM_ASCEND_ENABLE_NZ, this should be documented and ideally enforced with a runtime check.
As an immediate step, please add a code comment above this decorator to explain why this environment variable is being disabled for this specific test. This will improve code clarity and help future developers understand the context.
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
|
It look goods to me. |
cd244da to
46f1b64
Compare
Signed-off-by: wangx700 <wangxin700@huawei.com>
What this PR does / why we need it?
enable sleepmode level2 e2e test and add the check logic to ensure the nz is not enabled.
Does this PR introduce any user-facing change?
no
How was this patch tested?
use e2e tests