latest/version3.x/pipeline_usage/PaddleOCR-VL

[giscus[bot]]

latest/version3.x/pipeline_usage/PaddleOCR-VL

Awesome multilingual OCR toolkits based on PaddlePaddle (practical ultra lightweight OCR system, support 80+ languages recognition, provide data annotation and synthesis tools, support training and deployment among server, mobile, embedded and IoT devices)

https://www.paddleocr.ai/latest/version3.x/pipeline_usage/PaddleOCR-VL.html

[MrBug1024]

The recognition accuracy seems to have some hints, but there is an obvious flaw. The recognition speed seems to be much slower than PP-STuctureV3 or PP-OCRv5. When using PP-STuctureV3 or PP-OCRv5, the RTX 4090 48GB consumes only 5-7GB of video memory. However, although PaddleOCR-VL-0.9B does have some hints about accuracy and consumes the same amount of video memory resources as PP-STuctureV3, its speed has decreased by at least 20-25 times. This is a common problem with Ai models and other visual models on the market today. Overall, we are not missing a model, but a "short and efficient" model that consumes GPU. A model with fewer resources, higher recognition accuracy, and faster speed must be a model that can transcend time

[ITFisher9201]

支持的cuda版本最小是多少

[MrBug1024]

最小版本没仔细看，但是我现的版本是12.9，好像11.x的有几个版本也可以，具体的查看官网首页安装选择提供一些官方认为最好的版本支持

[luoxzhg]

使用官方镜像：ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddle:3.2.0-gpu-cuda12.6-cudnn9.5
安装： pip install paddleocr[doc-parser]
运行命令：paddleocr doc_parser -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/paddleocr_vl_demo.png

报错：
（1）ImportError: libGL.so.1: cannot open shared object file: No such file or directory
（2）ImportError: libgthread-2.0.so.0: cannot open shared object file: No such file or directory
解决之后，无法运行：

/usr/local/lib/python3.10/dist-packages/paddle/utils/decorator_utils.py:420: Warning: 
Non compatible API. Please refer to https://www.paddlepaddle.org.cn/documentation/docs/en/develop/guides/model_convert/convert_from_pytorch/api_difference/torch/torch.split.html first.
  warnings.warn(
Traceback (most recent call last):
  File "/usr/local/bin/paddleocr", line 7, in <module>
    sys.exit(console_entry())
  File "/usr/local/lib/python3.10/dist-packages/paddleocr/__main__.py", line 26, in console_entry
    main()
  File "/usr/local/lib/python3.10/dist-packages/paddleocr/_cli.py", line 192, in main
    _execute(args)
  File "/usr/local/lib/python3.10/dist-packages/paddleocr/_cli.py", line 181, in _execute
    args.executor(args)
  File "/usr/local/lib/python3.10/dist-packages/paddleocr/_pipelines/paddleocr_vl.py", line 363, in execute_with_args
    perform_simple_inference(
  File "/usr/local/lib/python3.10/dist-packages/paddleocr/_utils/cli.py", line 62, in perform_simple_inference
    wrapper = wrapper_cls(**init_params)
  File "/usr/local/lib/python3.10/dist-packages/paddleocr/_pipelines/paddleocr_vl.py", line 63, in __init__
    super().__init__(**kwargs)
  File "/usr/local/lib/python3.10/dist-packages/paddleocr/_pipelines/base.py", line 67, in __init__
    self.paddlex_pipeline = self._create_paddlex_pipeline()
  File "/usr/local/lib/python3.10/dist-packages/paddleocr/_pipelines/base.py", line 105, in _create_paddlex_pipeline
    return create_pipeline(config=self._merged_paddlex_config, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/paddlex/inference/pipelines/__init__.py", line 167, in create_pipeline
    pipeline = BasePipeline.get(pipeline_name)(
  File "/usr/local/lib/python3.10/dist-packages/paddlex/utils/deps.py", line 206, in _wrapper
    return old_init_func(self, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/paddlex/inference/pipelines/_parallel.py", line 103, in __init__
    self._pipeline = self._create_internal_pipeline(config, self.device)
  File "/usr/local/lib/python3.10/dist-packages/paddlex/inference/pipelines/_parallel.py", line 158, in _create_internal_pipeline
    return self._pipeline_cls(
  File "/usr/local/lib/python3.10/dist-packages/paddlex/inference/pipelines/paddleocr_vl/pipeline.py", line 127, in __init__
    self.vl_rec_model = self.create_model(vl_rec_config)
  File "/usr/local/lib/python3.10/dist-packages/paddlex/inference/pipelines/base.py", line 106, in create_model
    model = create_predictor(
  File "/usr/local/lib/python3.10/dist-packages/paddlex/inference/models/__init__.py", line 87, in create_predictor
    return BasePredictor.get(model_name)(
  File "/usr/local/lib/python3.10/dist-packages/paddlex/inference/models/doc_vlm/predictor.py", line 66, in __init__
    self.infer, self.processor = self._build(**kwargs)
  File "/usr/local/lib/python3.10/dist-packages/paddlex/inference/models/doc_vlm/predictor.py", line 152, in _build
    model = PaddleOCRVLForConditionalGeneration.from_pretrained(
  File "/usr/local/lib/python3.10/dist-packages/paddlex/inference/models/common/vlm/transformers/model_utils.py", line 1941, in from_pretrained
    state_dict = load_state_dict(
  File "/usr/local/lib/python3.10/dist-packages/paddlex/inference/models/common/vlm/transformers/model_utils.py", line 254, in load_state_dict
    with safe_open(checkpoint_file, framework="paddle") as f:
safetensors_rust.SafetensorError: framework paddle is invalid

[sunsssyc]

本地部署推理服务后，怎么增加前端并连通后端服务呢，有例子吗

[johnymoo]

it works for me by using client, not sure how to achieve it using http api..

pipeline = PaddleOCRVL(
            vl_rec_backend="vllm-server", 
            vl_rec_server_url="http://127.0.0.1:8118/v1"
        )
result = pipeline.predict(image_path)
res = result[0]
 save_path = "client_output_1"
            
            # 保存为 JSON 格式
            if hasattr(res, 'save_to_json'):
                res.save_to_json(save_path=save_path)
                print(f"💾 JSON 结果已保存到: {save_path}/")
            
            # 保存为 Markdown 格式
            if hasattr(res, 'save_to_markdown'):
                res.save_to_markdown(save_path=save_path)
                print(f"📝 Markdown 结果已保存到: {save_path}/")
            ```

[wanshichenguang]

客户端调用是这个意思吗？
ModuleNotFoundError: No module named 'paddle'

[coffee3699]

客户端调用需要安装paddle环境，和服务端是两套环境

[johnymoo]

need to install :
"paddleocr[doc-parser]>=3.0.0",
"paddlepaddle>=3.1.1",

[seraph900112]

这目前只支持页面作为输入？ 没有支持单个元素识别的方法吗？

[Fanzaijun]

docker run
-it
--rm
--gpus all
--network host
ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddlex-genai-vllm-server
paddlex_genai_server --model_name PaddleOCR-VL-0.9B --host 0.0.0.0 --port 8118 --backend vllm 这个部署之后，我怎么对接api，不通过客户端的方式

[HUbinand]

你不会想像VLLM部署模型一样提供一个url吧

[Fanzaijun]

这是标准的open ai，不能返回图片和文字结合md

[HUbinand]

太难部署了
1）通过docker部署官方镜像，第一官方镜像12GB为什么那么大，第二，dokcer的官方镜像需要NVIDIA Container Toolkit工具，才能让容器和宿主机互通，这个NVIDIA Container Toolkit工具又很难安装，我看官方是不支持很多操作系统。
2）通过 PaddleOCR CLI 安装的话，没有说明清楚python版本和cuda版本vllm版本，这样下来完全是盲猜，比如我的云服务器是cuda是11.8版本，但是执行paddleocr install_genai_server_deps vllm 命令，自动去下载了12.x版本cuda的且vllm版本是10.2的这样导致各种报错。非常难本地部署。。。。。。。
建议：
文档写清楚前置条件
1）GUP算力需要多少
2）python版本需要多少
3）vllm版本需要多少
4）cuda版本需要多少
5）paddlepaddle-gpu怎么来的

最好的方法就是把官方的dokcer.yaml文件发出来给大家参考一下

[neutron-L]

请问怎么用paddlex部署本地下载的PaddleOCR-VL模型，没有参数可以指定模型路径，修改配置文件好像也不行

[YJX-01]

registry.baidubce.com/paddlepaddle/paddle:3.2.0-gpu-cuda12.9-cudnn9.9 （及其他安装指南推荐）的3.2.0镜像是否支持3.3.0的pipeline模式？如果不支持能否提供适用于3.3.0的镜像？

[YangYongAn]

目前哪些云厂商支持直接使用啦