Skip to content

Commit c0b8af7

Browse files
committed
fix may_reinitialize_input_batch bug
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
1 parent 4c9d94f commit c0b8af7

File tree

1 file changed

+1
-2
lines changed

1 file changed

+1
-2
lines changed

vllm_ascend/worker/model_runner_v1.py

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2815,8 +2815,7 @@ def may_reinitialize_input_batch(self,
28152815
# This is likely Mamba or other non-attention cache,
28162816
# no splitting.
28172817
kernel_block_sizes.append([0])
2818-
2819-
if block_sizes != [self.cache_config.block_size]:
2818+
if kernel_block_sizes != [self.cache_config.block_size]:
28202819
assert self.cache_config.cpu_offload_gb == 0, (
28212820
"Cannot re-initialize the input batch when CPU weight "
28222821
"offloading is enabled. See https://github.com/vllm-project/vllm/pull/18298 " # noqa: E501

0 commit comments

Comments
 (0)