How to avoid reloading PP-StructureV3 models on every request in Flask/Celery without std::exception crashes?

[Atulok0506]

I’m using PP-StructureV3 in a Flask + Celery application.

If I reload the pipeline for each request, it works but is too slow (since it reloads ~6 models every time).

If I cache/reuse the pipeline across requests, the first request succeeds, but on the second one I get a std::exception crash from Paddle’s C++ backend.

This makes it look like Paddle’s internal state gets corrupted after reuse.

Question:
What is the recommended way to run PP-StructureV3 in a long-lived server environment (Flask/Celery)?

Is there an officially supported pattern for keeping the models loaded in memory across multiple requests?

Or is the only safe option to reload the pipeline each time or run it in short-lived subprocesses?

Do PaddleOCR/PaddleX developers recommend any workaround for avoiding std::exception while still getting reasonable performance?

[liuhongen1234567]

Hello, for deployment, you can refer to the documentation at https://www.paddleocr.ai/latest/en/version3.x/deployment/serving.html.

If you want to avoid loading the model every time, you can initialize the service with pipeline = PPStructureV3() outside or at a higher level, and then simply call output = pipeline.predict("./pp_structure_v3_demo.png") each time you want to test in Flask.

[Sanket-Navriti]

Hello, thank you for the response.

I've tried initializing pipeline = PPStructureV3() outside the request handler as recommended, but I'm running into a critical issue:

• First request: Works perfectly ✓
• Second request: Crashes with std::exception from Paddle's C++ backend ✗

The pipeline seems to have corrupted internal state after the first prediction.

My setup:

• Flask + Celery application
• PP-StructureV3 initialized once at module level
• Reusing same pipeline instance across requests

Already tried:

• Thread locks
• Garbage collection between requests
• use_gpu=False
• Single-threaded Flask

None of these resolve the std::exception on the second request.

Current situation:
The only working solution is reinitializing the pipeline each time, but this reloads ~6 models per request (5-10 seconds overhead), which defeats the purpose.
This take time to process the pdf like now it's taking 4 minutes minimum for processing.

Questions:

1. Is there a known issue with PP-StructureV3's state management in long-lived processes?
2. Are there configuration options to reset the C++ backend between predictions?
3. What's the recommended production pattern for handling multiple sequential requests?
4. Should we use subprocess/container-per-request instead?

The documentation suggests keeping the pipeline loaded, but the C++ exception indicates state isn't being properly reset between predictions. Any guidance on production-ready deployment would be helpful.

Thank you!