[v0.8.5rc1] FAQ / Feedback | 问题/反馈

[Yikun]

Anything you want to discuss about vllm on ascend.

Anything you want to discuss about vllm on ascend.

Please use doc: https://vllm-ascend.readthedocs.io

---

请使用 https://vllm-ascend.readthedocs.io 安装

[BestKuan]

Qwen3-235B-A22B can't run successful when setting VLLM_USE_V1=1 and v0 engine is ok. #781

[gao12312]

can not run deepseek-R1 w8a8 , #813

RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.43.down_proj.,param_name is experts.w2_,name is model.layers.58.mlp.experts.w2_weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.43.down_proj.,param_name is experts.w2_,name is model.layers.58.mlp.experts.43.down_proj.weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.43.down_proj.,param_name is experts.w2_,name is model.layers.58.mlp.experts.w2_weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.43.gate_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.43.gate_proj.weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.43.gate_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.w13_weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.43.gate_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.43.gate_proj.weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.43.gate_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.w13_weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.43.up_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.43.up_proj.weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.43.up_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.w13_weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.43.up_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.43.up_proj.weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.43.up_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.w13_weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.44.down_proj.,param_name is experts.w2_,name is model.layers.58.mlp.experts.44.down_proj.weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.44.down_proj.,param_name is experts.w2_,name is model.layers.58.mlp.experts.w2_weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.44.down_proj.,param_name is experts.w2_,name is model.layers.58.mlp.experts.44.down_proj.weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.44.down_proj.,param_name is experts.w2_,name is model.layers.58.mlp.experts.w2_weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.44.gate_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.44.gate_proj.weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.44.gate_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.w13_weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.44.gate_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.44.gate_proj.weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.44.gate_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.w13_weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.44.up_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.44.up_proj.weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.44.up_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.w13_weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.44.up_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.44.up_proj.weight_scale
Loading safetensors checkpoint shards:   1% Completed | 1/157 [00:00<00:43,  3.56it/s]
Loading safetensors checkpoint shards:   1% Completed | 2/157 [00:00<00:43,  3.55it/s]
ERROR 05-13 10:04:16 [core.py:396] EngineCore failed to start.
ERROR 05-13 10:04:16 [core.py:396] Traceback (most recent call last):
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 387, in run_engine_core
ERROR 05-13 10:04:16 [core.py:396]     engine_core = EngineCoreProc(*args, **kwargs)
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 329, in __init__
ERROR 05-13 10:04:16 [core.py:396]     super().__init__(vllm_config, executor_class, log_stats,
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 64, in __init__
ERROR 05-13 10:04:16 [core.py:396]     self.model_executor = executor_class(vllm_config)
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/executor/executor_base.py", line 286, in __init__
ERROR 05-13 10:04:16 [core.py:396]     super().__init__(*args, **kwargs)
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/executor/executor_base.py", line 52, in __init__
ERROR 05-13 10:04:16 [core.py:396]     self._init_executor()
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/executor/ray_distributed_executor.py", line 114, in _init_executor
ERROR 05-13 10:04:16 [core.py:396]     self._init_workers_ray(placement_group)
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/executor/ray_distributed_executor.py", line 396, in _init_workers_ray
ERROR 05-13 10:04:16 [core.py:396]     self._run_workers("load_model",
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/executor/ray_distributed_executor.py", line 521, in _run_workers
ERROR 05-13 10:04:16 [core.py:396]     ray_worker_outputs = ray.get(ray_worker_outputs)
ERROR 05-13 10:04:16 [core.py:396]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/ray/_private/auto_init_hook.py", line 21, in auto_init_wrapper
ERROR 05-13 10:04:16 [core.py:396]     return fn(*args, **kwargs)
ERROR 05-13 10:04:16 [core.py:396]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
ERROR 05-13 10:04:16 [core.py:396]     return func(*args, **kwargs)
ERROR 05-13 10:04:16 [core.py:396]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/ray/_private/worker.py", line 2822, in get
ERROR 05-13 10:04:16 [core.py:396]     values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout)
ERROR 05-13 10:04:16 [core.py:396]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/ray/_private/worker.py", line 930, in get_objects
ERROR 05-13 10:04:16 [core.py:396]     raise value.as_instanceof_cause()
ERROR 05-13 10:04:16 [core.py:396] ray.exceptions.RayTaskError(KeyError): ray::RayWorkerWrapper.execute_method() (pid=64195, ip=10.151.18.104, actor_id=e36a0df3031e94cd0ece267115000000, repr=<vllm.executor.ray_utils.RayWorkerWrapper object at 0xfffc23b6fd00>)
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/worker/worker_base.py", line 621, in execute_method
ERROR 05-13 10:04:16 [core.py:396]     raise e
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/worker/worker_base.py", line 612, in execute_method
ERROR 05-13 10:04:16 [core.py:396]     return run_method(self, method, args, kwargs)
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/utils.py", line 2456, in run_method
ERROR 05-13 10:04:16 [core.py:396]     return func(*args, **kwargs)
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/worker_v1.py", line 178, in load_model
ERROR 05-13 10:04:16 [core.py:396]     self.model_runner.load_model()
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/model_runner_v1.py", line 939, in load_model
ERROR 05-13 10:04:16 [core.py:396]     self.model = get_model(vllm_config=self.vllm_config)
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/model_executor/model_loader/__init__.py", line 14, in get_model
ERROR 05-13 10:04:16 [core.py:396]     return loader.load_model(vllm_config=vllm_config)
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/model_executor/model_loader/loader.py", line 455, in load_model
ERROR 05-13 10:04:16 [core.py:396]     loaded_weights = model.load_weights(
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/model_executor/models/deepseek_v2.py", line 788, in load_weights
ERROR 05-13 10:04:16 [core.py:396]     param = params_dict[name]
ERROR 05-13 10:04:16 [core.py:396] KeyError: 'model.layers.57.mlp.experts.w2_weight_offset'

[wphtrying]

问下安装v0.8.5rc1 版本的vllm-ascend，一定需要gcc高于9吗？