main/version3.x/pipeline_usage/PaddleOCR-VL

[giscus[bot]]

main/version3.x/pipeline_usage/PaddleOCR-VL

Awesome multilingual OCR toolkits based on PaddlePaddle (practical ultra lightweight OCR system, support 80+ languages recognition, provide data annotation and synthesis tools, support training and deployment among server, mobile, embedded and IoT devices)

https://www.paddleocr.ai/main/version3.x/pipeline_usage/PaddleOCR-VL.html

[design365]

华为昇腾服务器arm64怎么部署？

[cuicheng01]

华为昇腾还未进行测试，欢迎帮忙测试～

[design365]

跑不起来啊，你们只有x86架构的docker镜像，自己安装的话，使用vllm-acsend又启动报错：

Traceback (most recent call last):
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch_npu/__init__.py", line 41, in <module>
    import torch_npu.npu
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch_npu/npu/__init__.py", line 130, in <module>
    from torch_npu.utils import _should_print_warning
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch_npu/utils/__init__.py", line 4, in <module>
    from torch_npu import _C
ImportError: /usr/local/python3.11.13/lib/python3.11/site-packages/torch_npu/lib/libtorch_npu.so: undefined symbol: _ZNK3c1010TensorImpl37compute_strides_like_channels_last_3dENS0_8identityIbEE

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/__init__.py", line 2821, in _import_device_backends
    entrypoint = backend_extension.load()
                 ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.13/lib/python3.11/importlib/metadata/__init__.py", line 202, in load
    module = import_module(match.group('module'))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.13/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch_npu/__init__.py", line 43, in <module>
    from torch_npu.utils._error_code import ErrCode, pta_error
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch_npu/utils/__init__.py", line 4, in <module>
    from torch_npu import _C
ImportError: /usr/local/python3.11.13/lib/python3.11/site-packages/torch_npu/lib/libtorch_npu.so: undefined symbol: _ZNK3c1010TensorImpl37compute_strides_like_channels_last_3dENS0_8identityIbEE

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/python3.11.13/bin/vllm", line 5, in <module>
    from vllm.entrypoints.cli.main import main
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/vllm/__init__.py", line 14, in <module>
    import vllm.env_override  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/vllm/env_override.py", line 5, in <module>
    import torch
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/__init__.py", line 2866, in <module>
    _import_device_backends()
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/__init__.py", line 2825, in _import_device_backends
    raise RuntimeError(
RuntimeError: Failed to load the backend extension: torch_npu. You can disable extension auto-loading with TORCH_DEVICE_BACKEND_AUTOLOAD=0.

[157677678]

Rocky9.6 3080显卡 Driver Version: 580.95.05 CUDA Version: 13.0 Docker version 28.5.1
使用3.1.1的第一条命令 报错：
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/paddlex/inference/genai/server.py", line 67, in run_genai_server
model_dir = get_model_dir(args.model_name, args.backend)
File "/usr/local/lib/python3.10/site-packages/paddlex/inference/genai/models/init.py", line 43, in get_model_dir
_check_model_name_and_backend(model_name, backend)
File "/usr/local/lib/python3.10/site-packages/paddlex/inference/genai/models/init.py", line 37, in _check_model_name_and_backend
raise ValueError(f"Unknown model: {model_name}")
ValueError: Unknown model: PaddleOCR-VL

[zhang-prog]

我们推送新的镜像解决了这个问题，如果已经pull了镜像，请更新下镜像再试试：
docker pull ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddlex-genai-vllm-server:latest

[yaoyun1]

我在vllm上部署，
启动命令：paddlex_genai_server --model_name PaddleOCR-VL-0.9B --backend vllm --port 8118 --backend_config vllm_config.yaml；

yaml文件内容:gpu-memory-utilization: 0.96
max-num-seqs: 2
tensor-parallel-size: 2
no-enable-prefix-caching: true
实例配置如下：
pipeline = PaddleOCRVL(vl_rec_backend="vllm-server", vl_rec_server_url="http://127.0.0.1:8118/v1",
use_layout_detection=False,
device = "gpu:0,1",
enable_hpi=False,
precision="fp16",
enable_mkldnn = True)
但是在测试一张身份证的耗时（不算缓存命中）需要3.2秒（推理时间）左右。这个还能提升吗，想问一下你们的测试时间是多少？
另外，enable_hpi=False,enable_mkldnn = True，这两个参数好像没作用，对时间并没有提升，是我的环境有问题吗？

[yaoyun1]

支持微调吗，有微调相关的文档吗