OCRPredictor: CUDA OutOfMemoryError with long PDF documents

[quattrobahn]

I've noticed that with the model = ocr_predictor() interface, the GPU memory use increases with the number of pages in the document. For very long docs, this results in OOM errors (depending on the amount of VRAM available). Since the pages are being processed sequentially, I'm not sure why this is occurring. Has anyone else encountered this?

from doctr.io import DocumentFile
from doctr.models import ocr_predictor

# Download a 487-page PDF
# !wget https://www.govinfo.gov/content/pkg/ERP-2024/pdf/ERP-2024.pdf
doc = DocumentFile.from_pdf("ERP-2024.pdf")
model = ocr_predictor(pretrained=True).cuda()
result = model(doc) # Results in OutOfMemoryError

[felixdittrich92]

Hi @quattrobahn 👋,

Yeah the issue here is that the whole pdf is loaded into RAM - so the gpu VRAM is mostly constant

We have already an open issue for such cases #1240

The easiest workaround atm would be to chunk the pdf before and merge the results afterwards.

[quattrobahn]

Thanks for your response! I apologize, I didn't see the existing issue when I searched for this.