I have a large collection of PDFs containing scanned text that I'd like to OCR.
No commercial (Abby, PhantomPDF, Acrobat Pro), service (Google Vision API), or open-source (pre-trained models using tesseract, kraken) tool has been able to OCR the text in a sufficiently accurate manner.
I have some of the PDFs in their original form (with the text intact), meaning I have a reasonable amount of exact, ground-truth training data with enormous overlap in fonts, page structure, etc.
It seems that every method to train your own OCR model requires your training data be set up line-by-line, meaning I need to cut each line of hundreds of pages in the training-PDFs into separate images (then I can simply split the text in the training-PDFs by line to create the corresponding gt.txt files for tesseract or kraken).
I've used tools to split PDFs by page and convert/save each page to an image file, but I have not been able to find a way to automate doing the same thing line-by-line. But, R's {pdftools} makes it seem like getting the y-coordinates of each line is possible...
pdftools::pdf_data(pdf_path)[[3]][1:4, ]
#> width height x y space text
#> 1 39 17 245 44 TRUE Table
#> 2 13 17 288 44 TRUE of
#> 3 61 17 305 44 FALSE Contents
#> 4 41 11 72 74 FALSE Overview
... but it's unclear to me how that can be adjusted to match the resolution scaling of any PDF-to-image routine.
All that being said...
Is there a tool out there that already does this?
If not, in what direction should I head to build my own?
It seems Magick is fully capable of this (as soon as I grok how to navigate the pixels), but that doesn't solve the question of how to translate the y-coordinates from something like {pdftools} to the pixel locations in an image generated using a DPI argument (like every? PDF-to-image conversion tool).
Edit # 1:
It turns out the coordinates are based on the PDF "object" locations, which doesn't necessarily mean that text that is supposed to be on the same line (and visually is) is always reflected as such. Text that is meant to be on the same row may be off by several pixels.
The next best thing is cropping boxes around each of the objects. In R, this does the trick:
build_training_data <- function(pdf_paths, out_path = "training-data") {
out_path_mold <- "%s/%s-%d-%d.%s"
for (pdf_path in pdf_paths) {
prefix <- sub(".pdf", "", basename(pdf_path), fixed = TRUE)
pdf_data <- pdftools::pdf_data(pdf_path)
pdf_text <- pdftools::pdf_text(pdf_path)
pdf_heights <- pdftools::pdf_pagesize(pdf_path)$height
for (i_page in seq_along(pdf_data)) {
page_text <- pdf_text[[i_page]]
line_text <- strsplit(page_text, "\n")[[1L]]
page_image <- magick::image_read_pdf(pdf_path, pages = i_page)
image_stats <- magick::image_info(page_image)
scale_by <- image_stats$height / pdf_heights[[i_page]]
page_data <- pdf_data[[i_page]]
for (j_object in seq_len(nrow(page_data))) {
cat(sprintf("\r- year: %s, page: %d, object: %d ",
prefix, i_page, j_object))
image_path <- sprintf(out_path_mold, prefix, i_page, j_object)
text_path <- sprintf(out_path_mold, prefix, i_page, j_object)
geom <- magick::geometry_area(
height = page_data$height[[j_object]] * scale_by * 1.2,
width = page_data$width[[j_object]] * scale_by * 1.1,
x_off = page_data$x[[j_object]] * scale_by,
y_off = page_data$y[[j_object]] * scale_by
)
line_image <- magick::image_crop(page_image, geom)
magick::image_write(line_image, format = "png",
path = image_path)
writeLines(page_data$text[[j_object]], text_path)
}
}
}
}
This is definitely not optimal.
