My doctR code unable to scan the image right

[nikitakor7]

I am trying to read a PDF which has fields like in the picture attached. My task is to extract fields marked with yes, no, maybe. I am using doctR to process this file. Input is a pdf file the output is incomplete with very little to no text. (I use page json and extract text)
My initial thought was to get the text and do some kind of pattern matching to extract data I need . But with no text that can be read! I am clueless on how to solve this. My ocr_predictor looks like this :

model = ocr_predictor(
                            det_arch='fast_base',
                            reco_arch='crnn_vgg16_bn',
                            pretrained=True,
                            export_as_straight_boxes=True,
                            straighten_pages=True,
                        )

Am I missing anything? Can anyone help solve this?

[felixdittrich92]

Hi @nikitakor7 👋,
Are you using v0.8.1 from pypi or the main branch ?
Because the pretrained fast models are only available on the main branch yet (as mentioned in the v0.8.1 release notes) :)

[nikitakor7]

Hi @felixdittrich92 , thanks for your quick response. I am using the pre trained fast models from the main branch, I have it git cloned on my machine

[felixT2K]

Ok 👍🏼 And it does not detect the text from your pdf ?
Would it be possible to provide an example ?

[nikitakor7]

That's correct. Let me give you an example. I created a PDF of the image from the above comments, I have attached the pdf reference. This is how the the output looks:

Document(
  (pages): [Page(
    dimensions=(630, 237)
    (blocks): [
      Block(
        (lines): [Line(
          (words): [Word(value='10000', confidence=0.59)]
        )]
        (artefacts): []
      ),
      Block(
        (lines): [Line(
          (words): [Word(value='0O0', confidence=0.19)]
        )]
        (artefacts): []
      ),
      Block(
        (lines): [Line(
          (words): [Word(value='BOEE', confidence=0.11)]
        )]
        (artefacts): []
      ),
      Block(
        (lines): [Line(
          (words): [Word(value='XX', confidence=0.54)]
        )]
        (artefacts): []
      ),
      Block(
        (lines): [Line(
          (words): [Word(value='-', confidence=0.47)]
        )]
        (artefacts): []
      ),
      Block(
        (lines): [Line(
          (words): [Word(value='0E0O', confidence=0.15)]
        )]
        (artefacts): []
      ),
      Block(
        (lines): [Line(
          (words): [
            Word(value='DOE', confidence=0.21),
            Word(value='L', confidence=0.29),
          ]
        )]
        (artefacts): []
      ),
      Block(
        (lines): [Line(
          (words): [Word(value='.', confidence=0.94)]
        )]
        (artefacts): []
      ),
    ]
  )]
)

May 20 Screenshot.pdf

As we can observe most of the word values are not captured or have very low confidence
Please let me know if you need any more details. Thank you for helping :)

[felixdittrich92]

@nikitakor7 please try it again with straighten_pages=False after testing this does still creates some artefacts but looks in general better.
The main issue here is that empty (without X) boxes are detected as "0"