How to enable GPU for ocr_predictor?

[Chin-Shan-Hong]

I am using "db_mobilenet_v3_large" and "crnn_mobilenet_v3_small" for OCR task. And I'm using CUDA 12.1 and NVIDIA GeForce RTX 3060.

model = ocr_predictor(det_arch='db_mobilenet_v3_large', reco_arch='crnn_mobilenet_v3_small', pretrained=True, assume_straight_pages=True, det_bs=8, reco_bs=1024).cuda()

I tried to add cuda() but have encountered AttributeError: 'OCRPredictor' object has no attribute 'cuda'

How can I enable GPU to speed up the OCR task?

[balochy]

you should wrap the whole predicator:

model = (ocr_predictor(det_arch='db_mobilenet_v3_large', reco_arch='crnn_mobilenet_v3_small', pretrained=True, assume_straight_pages=True, det_bs=8, reco_bs=1024)).cuda()

[LiquidGunay]

My use case is similar, where I'm doing 100s of pdfs in sequence (10000+ images total). I haven't tested with the additional flags yet, only with half precision and default settings using dbresnet50 and parseq. At some point the vram usage crosses 22 gigs and my machine crashes.

[felixdittrich92]

ok @LiquidGunay about GPU VRAM the flags should help

In the CPU RAM case @sanjay-nit you should test it (not sure what happened on Colab) but i double checked it and if you don't pass 100s of images at once to the DocumentFile the RAM usage was constant and the garbage collector did his job ^^

[sanjay-nit]

@felixdittrich92 Just FYI I tried the same what you suggested to pass the list of images rather than passing one by one. So after processing all the image it should have release the memory, right? And because it is not releasing the memory so what is happening let's say we have new documents/images coming up to process so again the memory uses will start from 36GB RAM in your case. I think it should not happen. You can see in below image after processing the 100 image it didn't release the memory so I process next 100 images so memory would increase by 2x. I hope you got my point.

[sanjay-nit]

@felixdittrich92 I tried the same thing in lambda GPU machine as well and there also I'm facing same issues.

[felixdittrich92]

@sanjay-nit Sure i got your point but unfortunately i can't reproduce it 😅