With all the progress in machine learning it seems like there should be an amazing OCR tool that works out of the box for structured documents.
Does it exist?
I've used tesseract and its relatives, but they seem to have a hard time with any document that's not a single column. The difference between what they can achieve (which to be fair is amazing) with what I expected based on all the ML demos that only does the first 10% (numbers only, no structure), but does it in a 30-minute demo, is big. Things like affine transforms (scaling, rotation) and decorations like bold, underline, and weird fonts create even more problems.
Why isn't there a docker container with an AWS lambda function in it that takes any image format I upload (pdf, png, jpg as the most critical) and returns a UTF string of its content?
What I do for tables is use the OCR as a step 1, where I just extract the coordinates of each textbox. Then, use one of several methods to recognize columns (Hough transform, or we search for empty areas). Then, do some clustering to recognize lines.
The problem is that there's a million of edge cases that you need to handle (split boxes that span multiple columns, in the case of tables recognize cells that span multiple columns or rows, etc.). There should be a solution for all these edge cases, but it gets tricky quickly.
Having worked on some of these questions, then there comes the question of logical flow, how do you order that inset square of text at the bigger font, is it a picture caption or a header text?
Digit/letter recognition is near perfect with the latest algorithms and techniques, but organizing the output from scanning a document is still pretty hard.
I have been using it for a while to extract text from invoices/pdfs. The troble that I have to go to math the corresponding header to its value in each row is exhausting to say the least. Once Si het the text, I have to do an innumerable amount of hacks to get valid mappings. I wish we had a tool for that. ;)
(Apologies in advance if this sounds snarky, don't mean to be)
There are hundreds of posts on using Keras/TensorFlow/PyTorch/etc to do MNIST classification, and many examples on Github. All these resources are very easy to find by Googling. This post doesn't seem to have anything different to add to the conversation. So, why do such articles continue to be written, and why do they still get upvoted on HN? Is it that a lot of people want to learn these things, but haven't gotten a chance to, so they upvote with the hope of staying in touch with the topic? Is it FOMO? One might be forgiven for considering this spam.
I think this is just the nature of any open forum where everyone has an equal vote (democracy).
In any large-enough population, there will invariably be a significant non-cognoscenti segment who will upvote this type of content. And who knows, for this segment, this content may be new despite repeated iterations elsewhere.
It seems inefficient, but I'd rather have this type of system than systems that weight upvotes according to karma. This way, at least you avoid polarization by high karma users. The occasional low-quality content is not such a bad price to pay for a self-renewing system.
After the hype dies down, there will naturally be fewer MNIST articles. :)
Yes, moderators routinely downweight tutorial articles for that reason. It took us years to realize it, but tutorials aren't a good fit for Hacker News.
HN is for articles that gratify intellectual curiosity. Most tutorials don't do that; it doesn't fit with being a recipe for a task. If a recipe were to dive deeply into how and why its steps work, how they got that way, and compared other recipes that have (or don't have) similar steps, it would gratify intellectual curiosity better. But then it would be a worse recipe. It would also be more work to write.
The worst kind of tutorial from an HN point of view is the kind filled with arbitrary details that apply only to a particular program or product, like "Now put a file named foo in a directory called baz". But that is almost all of them.
“Many good ideas will not work well on MNIST (e.g. batch norm). Inversely[,] many bad ideas may work on MNIST and no[t] transfer to real [computer vision]” – a tweet by François Chollet (creator of Keras)
14 comments
[ 3.4 ms ] story [ 38.5 ms ] threadDoes it exist?
I've used tesseract and its relatives, but they seem to have a hard time with any document that's not a single column. The difference between what they can achieve (which to be fair is amazing) with what I expected based on all the ML demos that only does the first 10% (numbers only, no structure), but does it in a 30-minute demo, is big. Things like affine transforms (scaling, rotation) and decorations like bold, underline, and weird fonts create even more problems.
Why isn't there a docker container with an AWS lambda function in it that takes any image format I upload (pdf, png, jpg as the most critical) and returns a UTF string of its content?
My god, I'm spoiled by technology.
What I do for tables is use the OCR as a step 1, where I just extract the coordinates of each textbox. Then, use one of several methods to recognize columns (Hough transform, or we search for empty areas). Then, do some clustering to recognize lines.
The problem is that there's a million of edge cases that you need to handle (split boxes that span multiple columns, in the case of tables recognize cells that span multiple columns or rows, etc.). There should be a solution for all these edge cases, but it gets tricky quickly.
https://cloud.google.com/vision/
https://cloud.google.com/vision/docs/ocr
https://cloud.google.com/vision/docs/supported-files
You have to sign up for a developer account. If you are only doing low volume requests (up to 1000 a month) it is free.
There are hundreds of posts on using Keras/TensorFlow/PyTorch/etc to do MNIST classification, and many examples on Github. All these resources are very easy to find by Googling. This post doesn't seem to have anything different to add to the conversation. So, why do such articles continue to be written, and why do they still get upvoted on HN? Is it that a lot of people want to learn these things, but haven't gotten a chance to, so they upvote with the hope of staying in touch with the topic? Is it FOMO? One might be forgiven for considering this spam.
In any large-enough population, there will invariably be a significant non-cognoscenti segment who will upvote this type of content. And who knows, for this segment, this content may be new despite repeated iterations elsewhere.
It seems inefficient, but I'd rather have this type of system than systems that weight upvotes according to karma. This way, at least you avoid polarization by high karma users. The occasional low-quality content is not such a bad price to pay for a self-renewing system.
After the hype dies down, there will naturally be fewer MNIST articles. :)
HN is for articles that gratify intellectual curiosity. Most tutorials don't do that; it doesn't fit with being a recipe for a task. If a recipe were to dive deeply into how and why its steps work, how they got that way, and compared other recipes that have (or don't have) similar steps, it would gratify intellectual curiosity better. But then it would be a worse recipe. It would also be more work to write.
The worst kind of tutorial from an HN point of view is the kind filled with arbitrary details that apply only to a particular program or product, like "Now put a file named foo in a directory called baz". But that is almost all of them.
https://news.ycombinator.com/newsguidelines.html
So - please, anything harder (or rather: more relevant to deep learning). At least images in CIFAR-10: https://blog.deepsense.ai/deep-learning-hands-on-image-class...