These OCR improvements will almost certainly be brought to google books, which is great. Long term it can enable compressing all non-digital rare books into a manageable size that can be stored for less than $5,000.[0] It would also be great for archive.org to move to this from Tesseract. I wonder what the cost would be, both in raw cost to run, and via a paid API, to do that.
im realizing how much of a bottleneck vision models are
im just a glorified speedreadin' promptin' QA at this point with codex
once it replaces the QA layer its truly over for software dev jobs
future would be a software genie where on aistudio you type: "go make counterstrike 1.6 clone, here is $500, you have two hours"
edit: saw the Screenspot benchmark and holy ** this is an insane jump!!! 11% to 71% even beating Opus 4.5's 50%...chatgpt is at 3.5% and it matches my experience with codex
Interesting. When i asked Gemini 3 Pro to generate a Infographic from my personal accounting sheet, it first failed to generate anything except a black background, then it generated something where it mixed different languages in a non-sensical way, with obvious typos and irrelevant information grouping. It's certainly a leap forward in OCR, rendering classic OCR useless.
The document is paints a super impressive picture, but the core constraint of “network connection to Google required so we can harvest your data” is still a big showstopper for me (and all cloud-based AI tooling, really).
I’d be curious to see how well something like this can be distilled down for isolated acceleration on SBCs or consumer kit, because that’s where the billions to be made reside (factories, remote sites, dangerous or sensitive facilities, etc).
I do some electrical drafting work for construction and throw basic tasks at LLMs.
I gave it a shitty harness and it almost 1 shotted laying out outlets in a room based on a shitty pdf. I think if I gave it better control it could do a huge portion of my coworkers jobs very soon
I just can't imagine we are close to letting LLMs do electrical work.
What I notice that I don't see talked about much is how "steerable" the output is.
I think this is a big reason 1 shots are used as examples.
Once you get past 1 shots, so much of the output is dependent on the context the previous prompts have created.
Instead of 1 shots , try something that requires 3 different prompts on a subject with uncertainty involved. Do 4 or 5 iterations and often you will get wildly different results.
It doesn't seem like we have a word for this. A "hallucination" is when we know what the output should be and it is just wrong. This is like the user steers the model towards an answer but there is a lot of uncertainty in what the right answer even would be.
To me this always comes back to the problem that the models are not grounded in reality.
Letting LLMs do electric work without grounding in reality would be insane. No pun intended.
The most promising aspect for machine learning in electrical and electronic systems is the quantity of precise and correct training data we already have, which keeps growing. This is excellent for tasks such as ASIC/FPGA/general chip design, PCB design, electrical systems design, AOI (automated optical inspection), etc.
The main task of existing tools is rule-based checks and flagging errors for attention (like a compiler), because there is simply too much for a human to think about. The rules are based on physics and manufacturing constraints--precise known quantities--leading to output accuracy which can be verified up to 100%. The output is a known-functioning solution and/or simulation (unless the tool is flawed).
Most of these design tools include auto-design (chips)/auto-routing (PCBs) features, but they are notoriously poor due to being too heavily rule-based. Similar to the Photoshop "Content Aware Fill" feature (released 15 years ago!), where the algorithm tries to fill in a selection by guessing values based on the pixels surrounding it. It can work exceptionally well, until it doesn't, due to lacking correct context, at which point the work needs to be done manually (by someone knowledgeable).
"Hallucinogenic" or diffusion-based AI (LLM) algorithms do not readily learn or repeat procedures with high accuracy, but instead look at the problem holistically, much like a human; weights of neural nets almost light up with possible solutions. Any rules are loose, context-based, interconnected, often invisible, and all based on experience.
LLM tools as features on the design-side could be very promising, as existing rule-based algorithms could be integrated in the design-loop feedback to ground them in reality and reiterate the context. Combined with the precise rule-based checking and excellent quality training data, it provides a very promising path, and more so than tasks in most fields as the final output can still be rule-checked with existing algorithms.
In the near-future I expect basic designs can be created with minimal knowledge. EEs and electrical designer "experts" will only be needed to design and manufacture the tools, to verify designs, and to implement complex/critical projects.
In a sane world, this knowledge-barrier drop should encourage and grow the entire field, as worldwide costs for new systems and upgrades decreases. It has the potential to boost global standards of living. We shouldn't have to be worrying about losing jobs, nor weighing up extortionately priced tools vs. selling our data.
And yeah just checked AI studio. 1 hour Witcher 3 blood and wine gameplay in 144p is 70MB and 300,000 tokens only. And it's pretty easy to create scene by scene description.
I'm really fascinate by the opportunities to analyze videos. The amount of tokens it compresses down to, and what you can reason across those tokens, is incredible.
It is the first model to get partial-credit on an LLM image test I have. Which is counting the legs of a dog. Specifically, a dog with 5 legs. This is a wild test, because LLMs get really pushy and insistent that the dog only has 4 legs.
In fact GPT5 wrote an edge detection script to see where "golden dog feet" met "bright green grass" to prove to me that there were only 4 legs. The script found 5, and GPT-5 then said it was a bug, and adjusted the script sensitivity so it only located 4, lol.
Anyway, Gemini 3, while still being unable to count the legs first try, did identify "male anatomy" (it's own words) also visible in the picture. The 5th leg was approximately where you could expect a well endowed dog to have a "5th leg".
That aside though, I still wouldn't call it particularly impressive.
As a note, Meta's image slicer correctly highlighted all 5 legs without a hitch. Maybe not quite a transformer, but interesting that it could properly interpret "dog leg" and ID them. Also the dog with many legs (I have a few of them) all had there extra legs added by nano-banana.
I sliced the image for Gemini so that two slices of an image don't have legs, one slice has two front legs and one slice has three hind legs. Then Gemini 3 Pro answered correctly that the dog has 5 legs. Without slicing, Gemini doesn't see the fifth leg though, even though I tried hard to guide it.
I bet if you'd show that image to a human they'd need a little time to figure out what the heck they were looking at. Humans might need additional guesses, too. Five-legged dogs aren't common, but well-endowed dogs may be.
> It is the first model to get partial-credit on an LLM image test I have. Which is counting the legs of a dog. Specifically, a dog with 5 legs. This is a wild test, because LLMs get really pushy and insistent that the dog only has 4 legs.
I wonder if “How many legs do you see?” is close enough to “How many lights do you see?” that the LLMs are responding based on the memes surrounding the Star Trek episode “Chain of Command”.
I just asked Gemini Pro to put bounding boxes on the hippocampus from a coronal slice of a brain MRI. Complete fail. There has to be thousands of pictures of coronal brain slices with hippocampal labels out there, but apparently it learned none of it...unless I am doing it wrong.
I'm playing with this and wondering if this is an actually good way to identify dominant colors and other features of a garment/product when using a photo where the item is styled and not isolated from the model or other garments
Since I think it's interesting to highlight the jagged intelligence, I have a simple word search puzzle [0] that Nano Banana Pro stills struggles to solve correctly. Gemini 3 Pro with Code Execution is able to one-shot the problem and find the positions of each word (this is super impressive! one year ago it wasn't possible), but Nano Banana Pro fails to highlight the words correctly.
Here's the output from two tests I ran:
1. Asking Nano Banana Pro to solve the word search puzzle directly [1].
2. Asking Nano Banana Pro to highlight each word on the grid, with the position of every word included as part of the prompt [2].
The fact that it gets 2 words correct demonstrates meaningful progress, and it seems like we're really close to having a model that can one-shot this problem soon.
There's actually a bit of nuance required to solve this puzzle correctly which an older Gemini model struggled to do without additional nudging. You have to convert the grid or word list to use matching casing (the grid uses uppercase, the word list uses lowercase), and you need to recognize that "soup mix" needs to have the space removed when doing the search.
52 comments
[ 4.2 ms ] story [ 54.6 ms ] thread[0] https://annas-archive.org/blog/critical-window.html
https://arxiv.org/abs/2504.07981
im just a glorified speedreadin' promptin' QA at this point with codex
once it replaces the QA layer its truly over for software dev jobs
future would be a software genie where on aistudio you type: "go make counterstrike 1.6 clone, here is $500, you have two hours"
edit: saw the Screenspot benchmark and holy ** this is an insane jump!!! 11% to 71% even beating Opus 4.5's 50%...chatgpt is at 3.5% and it matches my experience with codex
I’d be curious to see how well something like this can be distilled down for isolated acceleration on SBCs or consumer kit, because that’s where the billions to be made reside (factories, remote sites, dangerous or sensitive facilities, etc).
Prompt: "wine glass full to the brim"
Image generated: 2/3 full wine glass.
True visual and spatial reasoning denied.
I gave it a shitty harness and it almost 1 shotted laying out outlets in a room based on a shitty pdf. I think if I gave it better control it could do a huge portion of my coworkers jobs very soon
What I notice that I don't see talked about much is how "steerable" the output is.
I think this is a big reason 1 shots are used as examples.
Once you get past 1 shots, so much of the output is dependent on the context the previous prompts have created.
Instead of 1 shots , try something that requires 3 different prompts on a subject with uncertainty involved. Do 4 or 5 iterations and often you will get wildly different results.
It doesn't seem like we have a word for this. A "hallucination" is when we know what the output should be and it is just wrong. This is like the user steers the model towards an answer but there is a lot of uncertainty in what the right answer even would be.
To me this always comes back to the problem that the models are not grounded in reality.
Letting LLMs do electric work without grounding in reality would be insane. No pun intended.
The main task of existing tools is rule-based checks and flagging errors for attention (like a compiler), because there is simply too much for a human to think about. The rules are based on physics and manufacturing constraints--precise known quantities--leading to output accuracy which can be verified up to 100%. The output is a known-functioning solution and/or simulation (unless the tool is flawed).
Most of these design tools include auto-design (chips)/auto-routing (PCBs) features, but they are notoriously poor due to being too heavily rule-based. Similar to the Photoshop "Content Aware Fill" feature (released 15 years ago!), where the algorithm tries to fill in a selection by guessing values based on the pixels surrounding it. It can work exceptionally well, until it doesn't, due to lacking correct context, at which point the work needs to be done manually (by someone knowledgeable).
"Hallucinogenic" or diffusion-based AI (LLM) algorithms do not readily learn or repeat procedures with high accuracy, but instead look at the problem holistically, much like a human; weights of neural nets almost light up with possible solutions. Any rules are loose, context-based, interconnected, often invisible, and all based on experience.
LLM tools as features on the design-side could be very promising, as existing rule-based algorithms could be integrated in the design-loop feedback to ground them in reality and reiterate the context. Combined with the precise rule-based checking and excellent quality training data, it provides a very promising path, and more so than tasks in most fields as the final output can still be rule-checked with existing algorithms.
In the near-future I expect basic designs can be created with minimal knowledge. EEs and electrical designer "experts" will only be needed to design and manufacture the tools, to verify designs, and to implement complex/critical projects.
In a sane world, this knowledge-barrier drop should encourage and grow the entire field, as worldwide costs for new systems and upgrades decreases. It has the potential to boost global standards of living. We shouldn't have to be worrying about losing jobs, nor weighing up extortionately priced tools vs. selling our data.
It is the first model to get partial-credit on an LLM image test I have. Which is counting the legs of a dog. Specifically, a dog with 5 legs. This is a wild test, because LLMs get really pushy and insistent that the dog only has 4 legs.
In fact GPT5 wrote an edge detection script to see where "golden dog feet" met "bright green grass" to prove to me that there were only 4 legs. The script found 5, and GPT-5 then said it was a bug, and adjusted the script sensitivity so it only located 4, lol.
Anyway, Gemini 3, while still being unable to count the legs first try, did identify "male anatomy" (it's own words) also visible in the picture. The 5th leg was approximately where you could expect a well endowed dog to have a "5th leg".
That aside though, I still wouldn't call it particularly impressive.
As a note, Meta's image slicer correctly highlighted all 5 legs without a hitch. Maybe not quite a transformer, but interesting that it could properly interpret "dog leg" and ID them. Also the dog with many legs (I have a few of them) all had there extra legs added by nano-banana.
I wonder if “How many legs do you see?” is close enough to “How many lights do you see?” that the LLMs are responding based on the memes surrounding the Star Trek episode “Chain of Command”.
https://youtu.be/S9brF-wlja8
https://i.imgur.com/1XxYoYN.png
https://imagazine.pl/wp-content/uploads/2024/12/ugly-AI-pic-...
GPT-5 couldn't do it.
https://news.ycombinator.com/item?id=44169413
Here's the output from two tests I ran:
1. Asking Nano Banana Pro to solve the word search puzzle directly [1].
2. Asking Nano Banana Pro to highlight each word on the grid, with the position of every word included as part of the prompt [2].
The fact that it gets 2 words correct demonstrates meaningful progress, and it seems like we're really close to having a model that can one-shot this problem soon.
There's actually a bit of nuance required to solve this puzzle correctly which an older Gemini model struggled to do without additional nudging. You have to convert the grid or word list to use matching casing (the grid uses uppercase, the word list uses lowercase), and you need to recognize that "soup mix" needs to have the space removed when doing the search.
[0] https://imgur.com/ekwfHrN
[1] https://imgur.com/1nybezU
[2] https://imgur.com/18mK5i5