HN Explorer

Ask HN: What is the best method for turning a scanned book as a PDF into text?

206 points by resource_waste ↗ HN
I like reading philosophy, particularly from the authors rather than a secondhand account.

However I often run into that these come as scanned documents, Discourses on Livy and Politics Among Nations for example.

I would greatly benefit from turning these into text. I can snipping tool pages and put them in ChatGPT and it turns out perfect. If I used classic methods, it often screws up words. My final goal is to turn these into audiobooks, (or even just make it easier to copypaste for my personal notes)

Given the state of AI, I'm wondering what my options are. I don't mind paying.

121 comments

[ 3.4 ms ] story [ 224 ms ] thread
ssuds
Perhaps building an agent in something like gumloop that loops page by page, does AI ocr, and then exports to a Google doc? Should take like 10 minutes to set up
brudgers
https://linux.die.net/man/1/pdftotext

is the simplest thing that might work.

It is free and mature.

jbaiter
That will not work for scanned PDFs without a text layer and even if it has one, it's not guaranteed to work.
brudgers
"Might work" comes with neither express nor implied warranty.

OCR is another thing that might work which is also simpler than an LLM.

lquist
My understanding is that Gemini OCR is now considered state of the art and a material step forward in OCR accuracy
Etheryte
Is this from the article that was on the front page a few days ago? If so, it's not true. The title was intentionally misleading, they said they're the best, but if you read the article it was that they're actually the best in some subproblem, not the actual thing.
anon373839
Yes. It’s still just an LLM and that means it can alter the meaning of entire passages in ways that are difficult to detect. This technology absolutely should not be used for OCR in domains where correctness matters.
[deleted]
(comment deleted)
[deleted]
(comment deleted)
chpatrick
I think you answered it yourself, stick it into a multimodal LLM.
Larrikin
Paperless uses the latest traditional method. There are LLM enhancements you can download
maurycy
I had very good experience with `gemini-2.0-flash-exp`:

https://github.com/maurycy/gemini-json-ocr

staplung
It's hard to know what to make of this because while you've included the output JSON you haven't included the input PDF so I have no idea how to interpret what it's actually doing.
maurycy
Give it a try on any PDF! This is just 100 LOC, easy to audit.
fucalost
depending on the length of these texts — and your technical ability — you might want to check out AWS Textract

it would be easy to set up a pipeline like:

> drop pdf in s3 bucket > eventbridge triggers step function > sfn calls textract > output saved to s3 & emailed to you

srameshc
Google Cloud Document AI is amazing, I love it https://cloud.google.com/document-ai?hl=en

It can correctly read many other languages than English if that is something you need. Previously I tried others and there were many errors in conversion. This does it well.

arctangos
I’m curious about this api. I’d looked at it before but it didn’t seem like it could handle arbitrary input that didn’t fit one of the predefined schemas. I also wasn’t sure how much training data it needed. What has your experience been like?
[deleted]
(comment deleted)
pronoiac
I made a high-quality scan of PAIP (Paradigms of Artificial Intelligence Programming), and worked on OCR'ing and incorporating that into an admittedly imperfect git repo of Markdown files. I used Scantailor to deskew and do other adjustments before applying Tesseract, via OCRmyPDF. I wrote notes for some of my process over at https://github.com/norvig/paip-lisp/releases/tag/v1.2 .

I'd also tried ocrit, which uses Apple's Vision framework for OCR, with some success - https://github.com/insidegui/ocrit

It's an ongoing, iterative process. I'll watch this thread with interest.

Some recent threads that might be helpful:

* https://news.ycombinator.com/item?id=42443022 - Show HN: Adventures in OCR

* https://news.ycombinator.com/item?id=43045801 - Benchmarking vision-language models on OCR in dynamic video environments - driscoll42 posted some stats from research

* https://news.ycombinator.com/item?id=43043671 - OCR4all

(Meaning, I have these browser tabs open, I haven't fully digested them yet)

kingkongjaffa
Was technology the right approach here? Is it essentially done now? I couldn’t tell if it was completed entirely.

I can’t help but think a few amateur humans could have read the pdf with their eyes and written the markdown by hand if the OCR was a little sketchy.

pronoiac
It's still in progress! It's looong - about a thousand pages. There's an ebook, but the printed book got more editing.
tyronehed
For years I have been printing PDFs off on regular paper and then binding them into books. 1. Print it at work when no one is looking. 2. Get two rigid boards and squeeze the stack of paper together. I customarily use two wooden armrests that originally came from a garden-furniture lounger. 3. Squeeze the paper with just a 1/4-inch showing. 4. Use wood glue and with your finger working like a toothbrush, work the glue into the pages at the gluing end. 5. Get a 14-inch X 4-inch strip of canvas. I use cutoff painter's canvas. 6. Hang all this by the boards and put glue also on top of the canvas strip. 7. When it dries, remove the boards and glue down the sides. You have a strong, bound book out of those printed pages.
thinkmassive
It’s unclear how this is related to the article, but I’m intrigued by your simple DIY bookbinding process.

It seems straightforward except for the canvas strip (I assume this is part of the binding?), and whether you add thicker pages/boards on each side as covers.

Do you have any photos of the process, or at least of a finished product? Thanks!

knicholes
I'm biased as an employee, but who knows PDFs better than Adobe? Use their PDF text extraction API.
Jedd
As someone who's been using KDE's Okular PDF reader for nearly twenty years, and also has to use Adobe's products - can confidently say that at least one answer to your question is 'The developers of KDE's Okular'.
aragonite
I did this very recently for a 19th century book in German with occasionally some Greek. The method that produces the highest level of accuracy I've found is to use ImageMagick to extract each page as a image, then send each image file to Claude Sonnet (encoded as base64) with a simple user prompt like "Transcribe the complete text from this image verbatim with no additional commentary or explanations". The whole thing is completed in under an hour & the result is near perfect and certainly much better than from standard OCR softwares.
HarHarVeryFunny
Is it really necessary to split it into pages? Not so bad if you automate it I suppose, but aren't there models that will accept a large PDF directly (I know Sonnet has a 32MB limit)?
7thpower
They are limited on how much they can output and there is generally an inverse relationship between the amount of tokens you send vs quality after the first 20-30 thousand tokens.
smallnix
Are there papers on this effect? That quality of responses diminishes with very large inputs I mean. I observed the same.
HarHarVeryFunny
I think these models all "cheat" to some extent with their long context lengths.

The original transformer had dense attention where every token attends to every other token, and the computational cost therefore grew quadratically with increased context length. There are other attention patterns than can be used though, such as only attending to recent tokens (sliding window attention), or only having a few global tokens that attend to all the others, or even attending to random tokens, or using combinations of these (e.g. Google's "Big Bird" attention from their Elmo/Bert muppet era).

I don't know what types of attention the SOTA closed source models are using, and they may well be using different techniques, but it'd not be surprising if there was "less attention" to tokens far back in the context. It's not obvious why this would affect a task like doing page-by-page OCR on a long PDF though, since there it's only the most recent page that needs attending to.

Breza
I've experienced this problem but I haven't come across papers about it. For this context, it would be interesting to compare the accuracy of transcribing one page at a time to batches of n pages.
siva7
They all accept large PDFs (or any kind of input) but the quality of the output will suffer for various reasons.
therealpygon
Necessary? No. Better? Probably. Despite larger context windows, attention and hallucinations aren’t completely a thing of the past within the expanded context windows today. Splitting to individual pages likely helps ensure that you stay well within a normal context window size that seems to avoid most of these issues. Asking an LLM to maintain attention for a single page is much more achievable than an entire book.

Also, PDF size isn’t a relevant measurement of token lengths when it comes to PDFs which can range from a collection of high quality JPEG images to thousand(s) of pages of text

ant6n
I recently did some OCRing with OpenAI. I found o3-mini-hi to be imagining and changing text, whereas the older (?) o4 was more accurate. It’s a bit worrying that some of the models screw around with the text.
jazzyjackson
There’s GPT4, then GPT4o (o for Omni, as in multi modal) and then GPT o1 (chain of thought / internal reasoning) then o3 (because o2 is a stadium in London that I guess is very litigious about its trademark?), o3-mini is the latest but yes optimized to be faster and cheaper
polshaw
o2 is the UK's largest mobile network operator. They bought naming rights to what was known as the millennium dome (not even a stadium).
dotancohen
What is the o3 model good for? Is it just an evolution of o1 (chain of thought / internal reasoning)?
KTibow
Yes

(albeit I believe o3-mini isn't natively multimodal)

ant6n
Which one is the smartest, and most knowledgeable? (Like least likely to make up facts)
wrsh07
4o is going to be better for a straight up factual question

(But eg I asked it about something Martin Short / John Mulaney said on SNL and it needed 2 prompts to get the correct answer..... the first answer wasn't making anything up it was just reasonably misinterpreting something)

It also has web search which will be more accurate if the pages it reads are good (it uses bing search, so if possible provide your own links and forcibly enable web search)

Similarly the latest Anthropic Claude Sonnet model (it's the new Sonnet 3.5 as of ~Oct) is very good.

The idea behind o3 mini is that it only knows as much as 4o mini (the names suck, we know) but it will be able to consider its initial response and edit it if it doesn't meet the original prompt's criteria

woile
What about preserving the style like titles and subtitles?
aragonite
You can request Markdown output, which takes care of text styling like italics and bold. For sections and subsections, in my own case they already have numerical labels (like "3.1.4") so I didn't feel the need to add extra formatting to make them stand out. Incidentally, even if you don't specify markdown output, Claude (at least in my case) automatically uses proper Unicode superscript numbers (like ¹, ², ³) for footnotes, which I find very neat.
cxr
> a 19th century book

If you're dealing with public domain material, you can just upload to archive.org. They'll OCR the whole thing and make it available to you and everyone else. (If you got it from archive.org, check the sidebar for the existing OCR files.)

aragonite
I did try the full text OCR from archive.org, but unfortunately the error rate is too high. Here are some screenshots to show what I mean:

- Original book image: https://imgur.com/a8KxGpY

- OCR from archive.org: https://imgur.com/VUtjiON

- Output from Claude: https://imgur.com/keUyhjR

cxr
Ah, yeah, that's not uncommon. I was operating on an assumption, based on experience seeing language models make mistakes, that the two approaches would be within an acceptable range of each other for your texts, plus the idea that it's better to share the work than not.

Note if you're dealing with a work (or edition) that cannot otherwise be found on archive.org, though, then if you do upload it, you are permitted as the owner of that item to open up the OCRed version and edit it. So an alternative workflow might be better stated:

1. upload to archive.org

2. check the OCR results

3. correct a local copy by hand or use a language model to assist if the OCR error rate is too high

4. overwrite the autogenerated OCR results with the copy from step 3 in order to share with others

(For those unaware and wanting to go the collaborative route, there is also the Wikipedia-adjacent WMF project called Wikisource. It has the upside of being more open (at least in theory) than, say, a GitHub repo—since PRs are not required for others to get their changes integrated. One might find, however, it to be less open in practice, since it is inhabited by a fair few wikiassholes of the sort that folks will probably be familiar with from Wikipedia.)

joseda-hg
Maybe I've just had back luck, but their OCR butchered some of the books I've tried to get
hkonsti
Do you have a rough estimate of what the price per page was for this?
aragonite
It must have been under $3 for the 150 or so API calls, possibly even under $2, though I'm less sure about that.
Oras
Quick and easy: Gemini Flash 2

More of a system: AWS Textract or Azure Document Intelligence. This option requires some coding and the cost is higher than using a vision model.