Refactor e2e CI #2276

wangxiyuan · 2025-08-08T04:15:23Z

Refactor E2E CI to make it clear and faster

remove some uesless e2e test
remove some uesless function
Make sure all test runs with VLLMRunner to avoid oom error
Make sure all ops test end with torch.empty_cache to avoid oom error
run the test one by one to avoid resource limit error

vLLM version: v0.10.1.1
vLLM main: vllm-project/vllm@a344a5a

github-actions · 2025-08-08T04:16:54Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2025-08-08T04:21:40Z

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

A PR should do only one thing, smaller PRs enable faster reviews.
Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

codecov · 2025-08-08T08:45:42Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.49%. Comparing base (0df059f) to head (95b11ff).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2276   +/-   ##
=======================================
  Coverage   73.49%   73.49%           
=======================================
  Files         151      151           
  Lines       21927    21927           
=======================================
  Hits        16116    16116           
  Misses       5811     5811

Flag	Coverage Δ
unittests	`73.49% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Yikun · 2025-08-08T08:57:30Z

tests/e2e/singlecard/test_aclgraph.py

-def test_deepseek_raises_error(monkeypatch: pytest.MonkeyPatch) -> None:
-    with monkeypatch.context() as m:
-        m.setenv("VLLM_USE_MODELSCOPE", "True")
-        with pytest.raises(NotImplementedError) as excinfo:
-            VllmRunner("deepseek-ai/DeepSeek-V2-Lite-Chat",
-                       max_model_len=1024,
-                       enforce_eager=False)
-        assert "ACL Graph does not support deepseek" in str(excinfo.value)
-
-
-@pytest.mark.parametrize("model", MODELS)
-def test_ray_backend_sets_no_compilation(model: str) -> None:
-    runner = VllmRunner(model,
-                        enforce_eager=False,
-                        distributed_executor_backend="ray")
-    assert runner.model.llm_engine.vllm_config.compilation_config.level == 0


Need to move to ut

Yikun · 2025-08-08T09:00:42Z

tests/e2e/singlecard/test_chunked.py

-        temperature=0.0,
-    )
-
-    vllm_model = LLM(model, long_prefill_token_threshold=4, enforce_eager=True)


Seems need to keep: #1172

Yikun · 2025-08-08T09:02:33Z

tests/e2e/multicard/test_pyhccl_distributed.py

-    assert torch.all(tensor == pynccl_comm.world_size).cpu().item()
-
-
-def test_pyhccl():


better to keep pyhccl e2e test if no more time cost, we can remove this after pyhccl removed

tests/e2e/multicard/test_ilama_lora_tp2.py

pyproject.toml

vllm_ascend/worker/model_runner_v1.py

github-actions · 2025-08-29T14:01:26Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2025-08-30T03:08:29Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

zzhx1 · 2025-09-01T10:27:46Z

It seems that the error in this e2e, #2675 can be resolved.

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

wangxiyuan · 2025-09-02T01:02:17Z

CI is blocked for quite long time. Let's merge this first and then recover the failure test asap.

Refactor E2E CI to make it clear and faster 1. remove some uesless e2e test 2. remove some uesless function 3. Make sure all test runs with VLLMRunner to avoid oom error 4. Make sure all ops test end with torch.empty_cache to avoid oom error 5. run the test one by one to avoid resource limit error - vLLM version: v0.10.1.1 - vLLM main: vllm-project/vllm@a344a5a Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com> Signed-off-by: hfadzxy <starmoon_zhang@163.com>

Refactor E2E CI to make it clear and faster 1. remove some uesless e2e test 2. remove some uesless function 3. Make sure all test runs with VLLMRunner to avoid oom error 4. Make sure all ops test end with torch.empty_cache to avoid oom error 5. run the test one by one to avoid resource limit error - vLLM version: v0.10.1.1 - vLLM main: vllm-project/vllm@a344a5a Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com> Signed-off-by: offline0806 <z00858301@china.huawei.com>

Refactor E2E CI to make it clear and faster 1. remove some uesless e2e test 2. remove some uesless function 3. Make sure all test runs with VLLMRunner to avoid oom error 4. Make sure all ops test end with torch.empty_cache to avoid oom error 5. run the test one by one to avoid resource limit error - vLLM version: v0.10.1.1 - vLLM main: vllm-project/vllm@a344a5a Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

github-actions bot added the merge-conflicts label Aug 8, 2025

github-actions bot added documentation Improvements or additions to documentation module:tests labels Aug 8, 2025

wangxiyuan force-pushed the refactor_e2e branch from fb88b90 to 7983418 Compare August 8, 2025 06:10

github-actions bot removed merge-conflicts documentation Improvements or additions to documentation labels Aug 8, 2025

wangxiyuan force-pushed the refactor_e2e branch 2 times, most recently from 1ff0e9e to 46d8efa Compare August 8, 2025 08:15

wangxiyuan changed the title ~~[CI] Refactor e2e CI~~ [CI] [1/2] Refactor e2e CI - singlecard Aug 8, 2025

wangxiyuan force-pushed the refactor_e2e branch from 46d8efa to 9ea433c Compare August 8, 2025 08:53

Yikun approved these changes Aug 8, 2025

View reviewed changes

wangxiyuan force-pushed the refactor_e2e branch 5 times, most recently from be01f15 to 0e2b140 Compare August 10, 2025 11:02

wangxiyuan commented Aug 10, 2025

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

wangxiyuan force-pushed the refactor_e2e branch 3 times, most recently from b6a3d08 to 0a8fe5f Compare August 10, 2025 13:09

wangxiyuan commented Aug 10, 2025

View reviewed changes

vllm_ascend/worker/model_runner_v1.py Outdated Show resolved Hide resolved

wangxiyuan force-pushed the refactor_e2e branch 6 times, most recently from de360ff to 6907a94 Compare August 11, 2025 01:13

wangxiyuan force-pushed the refactor_e2e branch 4 times, most recently from e523103 to 65767d7 Compare August 29, 2025 08:57

github-actions bot added the merge-conflicts label Aug 29, 2025

wangxiyuan force-pushed the refactor_e2e branch from 65767d7 to 8a60f2b Compare August 29, 2025 14:10

github-actions bot removed the merge-conflicts label Aug 29, 2025

github-actions bot added the merge-conflicts label Aug 30, 2025

wangxiyuan force-pushed the refactor_e2e branch from 8a60f2b to 44d0d19 Compare September 1, 2025 03:27

github-actions bot removed the merge-conflicts label Sep 1, 2025

wangxiyuan force-pushed the refactor_e2e branch from 44d0d19 to c00f0da Compare September 1, 2025 06:04

wangxiyuan force-pushed the refactor_e2e branch 4 times, most recently from ba08c9e to 3045a1a Compare September 1, 2025 22:53

[CI] Refactor e2e CI

95b11ff

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

wangxiyuan force-pushed the refactor_e2e branch from 3045a1a to 95b11ff Compare September 2, 2025 01:01

wangxiyuan merged commit fef18b6 into vllm-project:main Sep 2, 2025
11 checks passed

wangxiyuan deleted the refactor_e2e branch September 4, 2025 06:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Refactor e2e CI #2276

Refactor e2e CI #2276

Uh oh!

wangxiyuan commented Aug 8, 2025 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Aug 8, 2025

Uh oh!

github-actions bot commented Aug 8, 2025

Uh oh!

codecov bot commented Aug 8, 2025 •

edited

Loading

Uh oh!

Yikun Aug 8, 2025

Uh oh!

Yikun Aug 8, 2025

Uh oh!

Yikun Aug 8, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Aug 29, 2025

Uh oh!

github-actions bot commented Aug 30, 2025

Uh oh!

zzhx1 commented Sep 1, 2025

Uh oh!

wangxiyuan commented Sep 2, 2025

Uh oh!

Uh oh!

Uh oh!

		assert torch.all(tensor == pynccl_comm.world_size).cpu().item()


		def test_pyhccl():

Refactor e2e CI #2276

Refactor e2e CI #2276

Uh oh!

Conversation

wangxiyuan commented Aug 8, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 8, 2025

Uh oh!

github-actions bot commented Aug 8, 2025

Uh oh!

codecov bot commented Aug 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Yikun Aug 8, 2025

Choose a reason for hiding this comment

Uh oh!

Yikun Aug 8, 2025

Choose a reason for hiding this comment

Uh oh!

Yikun Aug 8, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Aug 29, 2025

Uh oh!

github-actions bot commented Aug 30, 2025

Uh oh!

zzhx1 commented Sep 1, 2025

Uh oh!

wangxiyuan commented Sep 2, 2025

Uh oh!

Uh oh!

Uh oh!

wangxiyuan commented Aug 8, 2025 •

edited by github-actions bot

Loading

codecov bot commented Aug 8, 2025 •

edited

Loading