Skip to content

Commit dff6ee5

Browse files
committed
some fixes
Signed-off-by: MengqingCao <cmq0113@163.com>
1 parent 823f320 commit dff6ee5

File tree

3 files changed

+13
-13
lines changed

3 files changed

+13
-13
lines changed

docs/source/user_guide/additional_config.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ The details of each config option are as follows:
5353
| ---- | ---- | ------- | ----------- |
5454
| `enabled` | bool | `False` | Whether to enable ascend scheduler for V1 engine|
5555

56-
ascend_scheduler_config also support the options from [vllm scheduler config](https://docs.vllm.ai/en/stable/api/vllm/config.html#vllm.config.SchedulerConfig). For example, you can add `chunked_prefill_enabled: true` to ascend_scheduler_config as well.
56+
ascend_scheduler_config also support the options from [vllm scheduler config](https://docs.vllm.ai/en/stable/api/vllm/config.html#vllm.config.SchedulerConfig). For example, you can add `enable_chunked_prefill: true` to ascend_scheduler_config as well.
5757

5858
### Example
5959

@@ -62,18 +62,18 @@ A full example of additional configuration is as follows:
6262
```
6363
{
6464
"torchair_graph_config": {
65-
"enabled": true,
66-
"use_cached_graph": true,
65+
"enabled": True,
66+
"use_cached_graph": True,
6767
"graph_batch_sizes": [1, 2, 4, 8],
68-
"graph_batch_sizes_init": false,
69-
"enable_multistream_moe": false,
70-
"enable_kv_nz": false
68+
"graph_batch_sizes_init": False,
69+
"enable_multistream_moe": False,
70+
"enable_kv_nz": False
7171
},
7272
"ascend_scheduler_config": {
73-
"enabled": true,
74-
"chunked_prefill_enabled": true,
73+
"enabled": True,
74+
"enable_chunked_prefill": True,
7575
},
7676
"expert_tensor_parallel_size": 1,
77-
"refresh": false,
77+
"refresh": False,
7878
}
7979
```

docs/source/user_guide/graph_mode.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,14 +47,15 @@ from vllm import LLM
4747

4848
os.environ["VLLM_USE_V1"] = 1
4949

50-
model = LLM(model="deepseek-ai/DeepSeek-R1-0528", additional_config={"torchair_graph_config": {"enabled": True}})
50+
# TorchAirGraph is only work without chunked-prefill now
51+
model = LLM(model="deepseek-ai/DeepSeek-R1-0528", additional_config={"torchair_graph_config": {"enabled": True},"ascend_scheduler_config": {"enabled": True,}})
5152
outputs = model.generate("Hello, how are you?")
5253
```
5354

5455
online example:
5556

5657
```shell
57-
vllm serve Qwen/Qwen2-7B-Instruct --additional-config='{"torchair_graph_config": {"enabled": true}}'
58+
vllm serve Qwen/Qwen2-7B-Instruct --additional-config='{"torchair_graph_config": {"enabled": True},"ascend_scheduler_config": {"enabled": True,}}'
5859
```
5960

6061
You can find more detail about additional config [here](./additional_config.md)

tests/multicard/test_torchair_graph_mode.py

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ def test_e2e_deepseekv3_with_torchair(monkeypatch: pytest.MonkeyPatch):
4545
max_tokens = 5
4646
# torchair is only work without chunked-prefill now
4747
with VllmRunner(
48-
"vllm-ascend/DeepSeek-V3",
48+
"vllm-ascend/DeepSeek-V3-Pruning",
4949
dtype=dtype,
5050
tensor_parallel_size=4,
5151
distributed_executor_backend="mp",
@@ -58,7 +58,6 @@ def test_e2e_deepseekv3_with_torchair(monkeypatch: pytest.MonkeyPatch):
5858
},
5959
"refresh": True,
6060
},
61-
load_format="dummy",
6261
enforce_eager=False,
6362
) as vllm_model:
6463
vllm_output = vllm_model.generate_greedy(example_prompts,

0 commit comments

Comments
 (0)