The writing is on the wall for handwriting recognition (newsletter.dancohen.org)

Great post and amazing progress in this field! However, I have to wonder if some of these letters were part of the training data for Gemini, since they are well-known and someone has probably already done the painstaking work of transcribing them...

If I went back in time to the 90s when I was doing my PhD I would absolutely blow my mind with how well handwriting OCR works now.

My question for OCR automation is always which digits within the numbers being read are allowed to be incorrect?

Maybe for English, for the other human languages I use, it is still kind of hit and miss, just like speaking recognition, even with English it suffices to have an accent that is off the standard TV one.

Call me when it can do Russian Cursive.

Surely the true prize is to be able to ditch computers altogether and just write with pencil on paper.

> Here’s Transkribus’s best guess at George’s letter to Maryann, above:

Transkribus got a new model architecture around the corner and the results look impressive. Not only for trivial cases like text, but also for table structures and layouting.

Best of all, you can train it on your own corpus of text to support obscure languages and handwriting systems.

Really looking forward to it.

I became convinced of this after the release of KuroNet: https://arxiv.org/pdf/1910.09433 (High-quality OCR of Japanese manuscripts, which look almost impossible to read.)

The writing is on the wall for handwriting. Zoomers use speech recognition or touchscreen keyboards, millennials use keyboards. Boomers use pens

It feels unbelievable that in Europe literacy rate could be 10% of lower. Then I look at documents even as young as 150 years... fraktur, blackletter, elaborate handwritting. I guess I'm illiterate now.

Hopefully next generations will feel the same about legal contracts, law in general, and Java code bases. They're incomprehensible not because of fonts but because of unfathomable complexity.

Any self-hosted open source solution? I would like to digitize my paper notebooks but I do not want to use anything proprietary or that uses external services. What is the state of the art on the FOSS side?

Ideally something that I can train with my own handwriting. I had a look at Tesseract, wondering if there’s anything better out there.

> "transmitted": In the second line of the body, the word "transmitted" is crossed out in the original text

Am I nuts or is this wrong, not “perfect”?

It doesn’t look crossed out at all to me in the image, just some bleeding?

Still very impressive, of course

> If AI can diminish some of the monotony of research, perhaps we can spend more time thinking, writing, playing piano, and taking walks — with other people.

Whenever any progress is made, this is the logical conclusion. And yet, those who decide about how your time is being used, have an opposing view.

It's painful to see that beautiful hand-writing of the past is now pretty much extinct. For me, handwriting of a person speaks a lot about them, not just their mind, but physical state as well.

Don't worry, handwriting itself has diminished throughout the decades since the introduction of computers an especially smart phones.

Ah, maybe I'll pick up Qin seal when I retire, if I retire.

Anyone knows how the models do on Russian cursive?

Quite certain my doctor can still produce writing, that the models don't stand a chance to be able to recognize.

I thought handwriting recognition is on the wall because no one knows how to write cursive any more

I confess this largely surprises me for reasons that I think should not surprise me. I would expect current AI is largely best at guessing at what some writing was based on expectations of other things it has managed to "read." As such, I would think it is largely not going to be much better at hand writing than any other tool.

Yet, it occurs to me that that "guess and check" is exactly what I'm doing when trying to read my 6yo's writing. Often I will do a pass to detect the main sounds, but then I start thinking of what was current on his thoughts and see if I can make a match. Not surprisingly, often I do.

I can't recognize my handwriting anymore. :(

Wouldn't it be easier to train a vLLM on the handwriting style of the historical person in question? An agent graphologist if you will. Surely there is a lot of pattern matching in the way things are written.

Then again, getting this result from a heavily-generalized SOTA model is pretty incredible too.

Now, onto the next frontier: handwriting recognition for shorthand. Let's start from Orthic :)

If that's not a proof of the 10/90 rule in machine learning. The last 10% of accuracy are harder than the first 90 (and that goes recursively).

We almost solved OCR 20 years ago. Then we spent 20 years on the last percentage. We see the same in self-driving cars.