Unreadable font using iTextSharp PRTokeniser

Question

I need to do a low level text extraction using PRTokeniser

For some PDF files everything is fine but for some others I get empty strings (or rather a string made up by empty boxes or empty characters).

All such files follow this pattern:

endobj
7 0 obj
<</BaseFont/RDZRPI+TimesNewRoman/FontDescriptor 8 0 R/Type/Font
/FirstChar 1/LastChar 10/Widths[ 444 500 444 500 444 333 500 500 278 250]
/Encoding 11 0 R/Subtype/TrueType>>
endobj
11 0 obj
<</Type/Encoding/BaseEncoding/WinAnsiEncoding/Differences[
1/a/b/c/d/e/f/g/h/i/space]>>
endobj 

Each character in the token stringvalue is not an ASCII or ANSI code but the index of the differences array. For instance if I have a string like "abc" I would get 1,2,3. Then iTextSharp would understand such codes as ASCII codes and would render them like empty square boxes or any other character.

Therefore I would need to get the subset array: number 1 is "a", number 2 is "b"...

The problem is that the basefont.differences array has only empty values and therefore I don't know how to rebuild the string.

On the other hand strategy.GetResultantText rendes the page content properly, but I need much more detail and that’s way I'm using PRTokeniser, although I'm stuck with this font problem.

Any ideas?

---

The PDF streams are all like this:

/FirstChar 1/LastChar 8/Widths[ 722 444 278 500 250 944 333 500]
/Encoding 11 0 R/Subtype/TrueType>>
endobj
11 0 obj
<</Type/Encoding/BaseEncoding/WinAnsiEncoding/Differences[
1/H/e/l/o/space/W/r/d]>>

In this case there is only this sentence in the PDF: Hellow Word. Therefore the chars array would be:

1 H
2 e
3 l
4 o
5 space
6 W
7 r
8 d

I need to find such an array using iTextSharp in order to decipher the string tokens.

Thanks

Answer 1

Thanks Mark for your answer:

You are awfully right. Therefore, I tried to peek under the covers at what strategy.GetResultantText as you suggested me to do, without much success. My final goal is to get each word coordinates in a pdf file. Therefore, I gave a try to the LocationTextExtractionStrategy strategy, which seems a reasonably starting point.

I can get the text rendered nicely, but I don’t know how to get the coordinates of each word or chunk. I would need to get access to something like the locationalResult array ( if such an array exists), but I don't know how.

Here is my code (VB.net):

Public Sub ParseLocation(ByVal sourcePDF As String)
    Dim reader As New iTextSharp.text.pdf.PdfReader(sourcePDF)
    Dim parser As New iTextSharp.text.pdf.parser.PdfReaderContentParser(reader)
    Dim strategy As parser.LocationTextExtractionStrategy
    Dim sResult As String

    For i As Integer = 1 To reader.NumberOfPages
        strategy = parser.ProcessContent(i, New LocationTextExtractionStrategy)
        sResult = strategy.GetResultantText
        Debug.Print(iTextSharp.text.pdf.parser.PdfTextExtractor.GetTextFromPage(reader, i, strategy))
        Debug.Print(sResult)

        strategy = parser.ProcessContent(i, New LocationTextExtractionStrategy)
        Debug.Print(strategy.GetResultantText())
    Next i
End Sub

Answer 2

11 0 obj <> isn't even valid PDF syntax... (or is it?) An empty dictionary looks like <<>>. I suspect you're missing a detail or two.

An encoding entry must be a dictionary or a string. Erk! <> is a valid empty String. Such strings are bytes encoded as hex values... <0102030304050604070308> -> "Hello World" in your sample encoding.

OTOH, an encoding entry may only be a name or dictionary (PdfName or PdfDictionary in iText-speak), so even if that really is an empty hex string, its still invalid.

At this point, I think you can cheerfully embrace Open Source and take a peek under the covers at what strategy.GetResultantText is doing.

PS: I think you're overusing PRTokenizer. Using on the stream portion of a content stream is fine and dandy, but using it to re-parse dictionaries and such is pure overhead. Those objects have already been parsed and turned into instances of PdfStream, PdfString, PdfDictionary, and so forth... using PRTokenizer.

PdfDictionary fontDict = magicallyFindFontDict();
PdfObject encodingObj = fontDict.getAsDirectObject(PdfName.ENCODING);
if (encodingObj == null) { //bail
} else if (encodingObj.isName()) {
  if (PdfName.WINANSIENCODING.equals(encodingObj)) {
    ...
  } else if (...) {}...

} else if (encodingObj.isDictionary()) {
  // details in 9.6.6 of ISO PDF Spec
  ...
}