Frequently Asked Questions on Inference and Deployment of PaddleOCR-VL PaddleOCR-VL 推理部署相关高频问题回复

[Bobholamovic]

dtype Mismatch Issue When Using PaddlePaddle for Inference on GPUs with Compute Capability < 8.5 (e.g., T4, V100)

As of October 24, 2025, the default inference method of PaddleOCR-VL (using PaddlePaddle dynamic graphs) now supports GPUs with Compute Capability ≥ 7.0. Please follow the official documentation to complete the installation. If you already have PaddleOCR installed locally, you can upgrade the PaddleX version to access the latest features by running the following command:

python -m pip install -U paddlex

Is Deployment on CPU or HUAWEI NPU Supported?

The current version primarily supports inference on NVIDIA GPUs. Adaptation for x64 CPUs and Ascend 910B NPUs is currently being prioritized and is underway. Please stay tuned for subsequent updates.

What Are the Minimum Hardware Configuration Requirements?

Memory usage and inference speed may vary significantly across different hardware devices, influenced by the total GPU memory and Compute Capability. We have conducted tests on various hardware types, including several consumer-grade graphics cards. Currently, the minimum supported configuration that successfully runs is an RTX 3060 (12 GB), and the lowest supported Compute Capability is 7.0 (e.g., V100).

Occasional OOM Issues During Inference

The default PaddlePaddle dynamic graph inference mode exhibits significant fluctuations in memory usage, with peak memory consumption potentially being high when processing complex images. For more stable memory usage, it is recommended to deploy using dedicated inference acceleration frameworks such as vLLM or SGLang.

在 Compute Capability < 8.5 的 GPU（如 T4、V100）上使用 PaddlePaddle 推理时出现 dtype 不匹配的问题

目前（截至 2025 年 10 月 24 日），PaddleOCR-VL 的默认推理方式（使用 PaddlePaddle 动态图）已支持 Compute Capability ≥ 7.0 的 GPU。请按照官方文档指引完成安装。若您已在本地安装 PaddleOCR，可通过以下命令升级 PaddleX 版本，以获取最新功能支持：

python -m pip install -U paddlex

是否支持在 CPU 或华为 NPU 上部署？

当前版本主要支持在 NVIDIA GPU 上进行推理。针对 x64 CPU 及昇腾 910B NPU 的适配工作正在高优推进中，敬请关注后续更新。

最低硬件配置要求是什么？

不同硬件设备上的显存占用及推理速度可能存在较大差异，具体表现受显卡总显存及 Compute Capability 影响。我们已在多类硬件（包括多款消费级显卡）中完成测试，目前可成功运行的最小显存配置为 RTX 3060（12 GB），最低支持的 Compute Capability 为 7.0（如 V100）。

推理过程中有时出现 OOM 问题

默认的 PaddlePaddle 动态图推理方式显存占用波动较大，处理复杂图像时峰值显存使用量可能较高。如需获得更稳定的显存占用表现，建议使用 vLLM、SGLang 等专用推理加速框架进行部署。