Opensource Ascend NPU Plugin for LMCache

[matthewygf]

This PR adds Ascend NPU 910b series initial support to LMCache v0.3.3 :

• support for multi_layer and single_layer transfer for vLLM connector

Changes Made

1. Added custom CMakeBuild system support based on Ascend CANN Toolkit 8.2
2. Kernels support for multi_layer and single_layer transfer
3. Added a host register API in the pybind11 interface for Ascend due to current python API limitation
4. Extend the MixedMemory and PinnedMemory Allocator for hostregister API
5. Monkey patched vllm_adapter for init_cache_engine for config check and used the extended classes
6. Made changes to the test_*.py that uses the necessary NPU APIs to run successfully, all unit tests passed on our platform except the skipped ones, and require non-supported kernels.
7. We have included a dockerfile for our A2 series
8. Introduce a Dynamic LMCacheAscendConnector that extend the LMCacheConnector for the serve entry point and patching the necessary c_ops and functions

related issue
related pr

Pre-requisite

NPU Driver version >= 24.1

CANN Tool-kit version >= 8.2

Installed compatible version of vllm-ascend, torch-npu

We currently support the following vLLM-ascend versions:

• v0.9.2

Build Instructions

pip install --no-build-isolation -v -e .

Test Coverage

We ran the unit tests successfully for the main patched classes.

Related PR

This following PR in vllm-ascend is related to support LMCache

• support for KVConnector v1

Roadmap

We currently only support eagermode execution. We plan to have the following features in the near future:

• Cachegen
• Cacheblend
• Graph Mode with torch_npu torchair
• P/D with NPU Transport

[YaoJiayi]

lgtm