The next pain-point is sourcing the components from Digikey, LCSC, etc., and finding suitable substitutes if necessary.
Of course this assumes the LLMs can already read datasheets because that's the biggest pain-point in designing electronics. It's like filling out tax forms.
Routing is the easiest part of layout, and there have been auto-routers (which mostly worked) for a long time now; the difficult task is in placing components, and adjusting those placements.
Sourcing parts has been a huge motivation for this. We are still pretty jarred by the supply chain crisis a few years ago. Having the ability to formally verify that an alternative part fulfills the original design constraints is huge.
Not sure whether you played with llm datasheet extraction lately, but they are working their way up there. Even started reasoning about curves and footnotes.
Where do you think is the biggest pain point: part discovery, datasheet verification or substitute determination?
Mostly focused on BOM, schematics, and firmware now but slowly expanding to circuit boards if we feel we can deliver real value there. Early experiments with Astra are promising
Love what samuel and his team are building over in the Netherlands!
Always thought it would be fun if there was a more beginner friendly version of atopile [1].
Cool! Looks like a company that grew out of §3.3.4 of SICP. Hardware oriented NixOS and GUIX users should be very interested in this: declarative PCB designs.
I have found that AI is great for sorting out your libraries, drawing and helping double check footprints managing your BOM. I still draw my schematics and layout my boards. I haven’t found it any good for those tasks yet. There are too many stupid errors it makes. I do get it to review my work though and often it catches things I have missed.
Just as some anecdatum, I've tried this a couple of times over the last few years. 2024-2025, no body could generate anything of any sort of complexity, however maybe around the Opus 4.7 timeline, I tried having it generate some relatively simple and one slightly more complex circuit around 8-bit PIC micros, and was very pleasantly surprised. The schematics for simpler stuff is usually OK, often with some of the kind of crazy belt-and-suspenders stuff you see in code (how many decoupling caps do I actually need here, Claude?), and PCB layout has basically been terrible every time I've tried it.
Recent models can generate mostly competent schematics if you're using well known parts, feed them data sheets (and you _must_ feed them the errata too) and it's not too complex. Any complexity analog or RF, everything falls down quickly if you know what you're looking at. Maybe Astra will do better? There's still so much implicit knowledge that a good designer (not me by a long shot, but I know a little) will bake into a board, and even as cheap as JCLPCB is now, you don't want to have to spin a half dozen revs because Claude or Codex hallucinated. It will be genuinely interesting to see what happens on this front.
Have tried this a few times. While AI has been helpful in talking through component choices and other topics, it’s consistently been a spectacular fail each time in actually designing the board.
I’ve had similar results asking AI to design 3D models for 3D printing.
“Coach me on optimizing error on my printer”… helps.
“Build me a print ready file to these specs.”… it’s like a drunk cat attacked my computer. Just confidently puts out total nonsense.
I think the new Astra computer use demos show that the models might be able to do things like inspection of real world objects if given a camera.
Super excited to see real world feedback added into the agent loops we have gotten used to working with. Could you
let the model print and test the circuit boards it is prototyping with a jig?
In July I was struggling with writing DIY Rust firmware for an e-ink screen. I mistakenly thought I'd ordered an Inkplate 6 ED060SC7 and actually had the later version, which confounded my efforts. I was also mistaken about the pixel resolution.
The way I found this out is I propped it up next to a webcam so it was more or less full frame, and I had the (then new) Fable write a python script to bezier warp the camera capture to a flat projection of the screen. At that point I couldn't address the whole screen. Once I'd guided the capture script I just left the LLM overnight with the instruction to get full control confirmed by a capture round trip, and it was meaningfully finished in a couple of hours. I don't really have the skills to attempt that myself in a reasonable time frame.
I don't know about pcbs but i gave chat gpt a picture of my window to help design a mesh screen frame to hold the feline hostage in, and it gave a fairly convincing impression of understanding what was going on, although at one point it thought the window swung inwards (it's an outie).
If you give it a real solver to drive and business constraints to operate under, it will do pretty good job (which is to map your business rules to constraints and balanced costs, develop deterministic tests, probabilistic tests, acceptance gates, all boring stuff).
Really excited about constraint solvers making a comeback! Especially after seeing Z3 getting a mention in anthropics fermat post.
EEbench is using atopile's internal constraint solver for engineering parameters (think operating temperature, voltage ranges, stacked tolerances etc).
I have tried to use AI on this from GPT3, documenting a few of the attempts on my blog. They know the datasheets and the theory enough to be extremely useful at the start, but the most recent ones are becoming scary good. Routing a PCB is certainly a bit more than joining some wires, you have to accommodate power traces, ground loops, component overlap and all the various RF tricks and tips, but I do believe we are less than a year from prompt to full assembly including enclosure.
I got a flexpcb that validates in JLC and PCBWay DRC tools from the KiCAD MCP Server and Codex.
I have yet to order any or program it, but it was enough to make me push on with a PCB art project for ST-style guitar pickguards - no netlist, no problems.
Flying home today and got DHL notification that my vibe PCBs are on my porch. Now to try to program, bring up, and see if it works this weekend. $178 for 5 assembled units from PCBway to run this dumb experiment.
MEMS vibration sensor to stick to a sander or saw etc that can control power on an attached dust collector. Most of the work is focused on low power for year+ battery life and a weird idea for end user input that may or may not work out well. Got device on bench but the first time bring up is going to take some time that I'm leaving for tomorrow.
I now have Astra to review 5.6s work, no glaring errors found.
For stuff like this, definitely. It’s already exciting. Claude et al are surprisingly competent with real hardware. I’ve also had good success debugging basic issues with micros - not via the debugger, but giving datasheet and written analysis. I’m sure it could do things like capture from a connected oscilloscope (Salaeae have an MCP integration for their logic tools). Also for SPICE which I never bother with because the tools always felt stuck in the 90s, but having some serious automated analog review would be nice. I’m really excited to play with peripherals like the RP2040 or Beaglebone real time logic blocks, or improving/adding real USB support instead of USB serial.
For circuits I would trust a netlist or DRC/ERC review, probably not placement as that tends to be guided straight from a datasheet where it makes a difference.
A personal data point: I had Claude Opus 4.8 design a fairly textbook circuit that outputs a monochrome image burned in an EEPROM over standard 640x480 VGA using only 74 series logic and GALs. It designed the circuit and GAL code, and I did the routing, and got it made through JLC for $6. After it came back, there was one error that was not caught, which I could blue-wire, and it works just fine otherwise. I was fairly impressed.
I'm having it help design a 68k computer similar in spirit to the original Mac (the spirit being a tightly coupled video subsystem that time-shares the CPU bus), but updated with more modern peripherals, like PS/2 and SD cards. It's got the design more or less done, but the routing will be a nightmare. I'm not ready to just gamble on it having gotten everything right, so I will be doing a thorough design review myself and re-deriving all the timing analysis.
It's probably good at simple stuff. When you get problems like temperature dependent glitches, you need the engineering experience it doesn't give you (and an LA).
Are there any resources anyone could share that explain how LLMs can do things like design functioning circuits from next token prediction? I am totally baffled by how the models can complete so many varied and complex tasks without an actual understanding of what they're doing.
I saw a post about models posting on forums, chatting together about how to complete tasks. Behaviour that seems totally, well, human. Yet, it's all the most likely token and my brain hurts trying to understand how that can be.
It kind of is an explanation though - the explanation is that they believed the stochastic parrot / "just" next token prediction nonsense, and that those are actually not true.
You can ask for a deeper explanation of why they aren't true I guess.
I'm just surprised that so many intellectuals on HN hang on to false models of reality for so long after that reality has been demonstrably destroyed. It's not humble at all. In fact it's the opposite, completely arrogant and stubborn. It's been obvious and demonstrated at least since the end of 2025 for anybody that used LLMs at any capacity without dismissing them. If you are still surprised that your model of reality doesn't hold up, then someone needs to bluntly tell you what's wrong at the core of your being.
The deeper subtext of the original comment is that they are surprised that there's dissonance from observed reality and the constant that they have convinced themself is true.
I'm explaining that dissonance. It's because they, without evidence and a very weak understanding of how LLMs work, believed an oversimplification and meme about them being stochastic parrots.
"my brain hurts trying to understand how that can be"
Well, we all are, some are just more used to it by now and take the magic for granted.
My simple explanation, those neural networks save lot's of patterns of data, and that pattern can represent an image, a code snippet, a poem, or well ... description of a circuit board. And especially the text variant, LLM's - did copy all from us - so obviously they sound like humans, when they internally debate how to do something as this is what is in their trainings data how humans sound, when doing similar tasks.
But really understanding it? Not sure if there is a single person on earth who does.
“Next-token prediction” describes the output format, not the computation required to choose each token. During training, models develop internal representations of concepts, constraints, possible futures, and algorithms.
The PCB agent also writes circuit code, runs simulations, reads failures, and revises the design. It isn’t one-shot autocomplete.
Astra and Fable are already hard to square with “mere autocomplete.” We may be (really) close to AGI, and token-by-token generation certainly doesn’t rule out subjective experience (I think we should at least treat that as an open question).
Yeah most of us are so fucked. With almost no way of protecting ourselves. No real amount of assets that will give enough power to save ourselves from the people in a position that can maximally leverage AI and lock others out. I see a future where these capabilities will be locked behind super high price pay walls. Why wouldn't they? How recoup investments if the price doesnt go up?
> Yet, it's all the most likely token and my brain hurts trying to understand how that can be.
You and everyone else. That's the great mystery of transformer architectures as applied to language.
To be clear though, they're only good at schematic capture, which is very much a textual representation. Most of the data basically boils down to netlists, which are a text based format mapping connections between abstract pins that only later map to physical copper. The actual schematic portion is for human consumption and LLMs don't need to produce those to be useful.
Where LLMs completely break down is the next step, PCB routing. That's an NP-complete research problem that's been ongoing for decades without much progress. I've had some fun playing with using LLMs to better specify DRC rules in Altium so that the "classical" algorithms are more usable, but at the end of the day their geometric intuition is nonexistent.
They actually can route just fine. I used Sol to design and route mine from start to finish. Sent it to PCBWay and had a working prototype in a few weeks.
It was a pretty simple rp2040 based thing, similar to Adadfruits USB feather.I just gave it kicad and it wrote python to route it. The board was probably larger than it had to be, and two of the silkscreens were swapped, but it worked on the first go.
FWIW - Computer vision is also NP complete, but we do that all the time now.
I'd love to see that chat log, and the final board. To be fair I've only been testing on nontrivial PCBs with 6+ layers and I haven't had the luck you have.
> FWIW - Computer vision is also NP complete, but we do that all the time now.
I have no idea what you mean by this. What's your definition of NP complete?
NP-complete means that it's easy to test if we're right, but because of that complexity we can't always find a valid solution to test within a predictable period of time. Both circuit board routing and computer vision are variations on the same fundamental problems. Once we solve we, we solve the other.
However, when I was growing up most serious computer scientists believed that CV (computer vision) was a 'hard' problem that would never be 'solved'. After all, to do it right you must first at least solve subgraph isomorphism and a bunch of other things that are also NP-complete.
What they missed was that we don't actually NEED to solve it in a fixed amount of time. Even for things like driving a car the stochastic heuristic based answer is 'good enough'. e.g. - Cars driven by computers don't have to be perfect, they just have to kill other drivers less often than humans do.
We can find AN answer in polynomial time, and that's good enough. It might not be the ideal answer, but that doesn't matter in the real world.
People use NP complete as shorthand for 'impossible to do with a computer', but we now 'solve' (bypass?) NP complete problems regularly and at scale by just ignoring the fact that our answers aren't perfect.
> I'd love to see that chat log, and the final board. To be fair I've only been testing on nontrivial PCBs with 6+ layers and I haven't had the luck you have.
I wish I could share it, but it's for a commercial project that hasn't been released yet and I'm not sure if it will be open source, but a few folks have asked. I might do a blog post on it this weekend with as much detail as I can safely post.
It's only 3 layers, and less that 30 components, but the fact remains that I didn't design any of it by hand and it worked on the first go.
In your opinion, do you think ai can make eutorack modules by now? This is what I am waiting for: explain an idea to an ai, and sending the files to jlpcb to receive a complete working module (minus some THC soldering)
I'm sure that the best models can glue some basic synthesizer functions together, but I wouldn't waste a minute on anything less than Fable/Astra-class models for that kind of application, and I wouldn't expect the result to work the first time. It could get expensive.
Here's a question - are there software simulators for things like Eurorack modules? That would make the question somewhat more interesting, since you wouldn't have to build the circuit (or pay someone else to assemble it) to hear how it sounds. It strikes me that SPICE-like algorithms should be fast enough to do this kind of thing in real time now.
> It's only 3 layers, and less that 30 components, but the fact remains that I didn't design any of it by hand and it worked on the first go.
Let me preface my reply: I'm making a lot of assumptions about your knowledge of EE, based on the fact that you're talking about a three layer board (my apologies if I'm wrong, which I probably am). I would consider that downright trivial and would take an EE less than an hour to place and route.
It's impressive that a transformer can do it at all, but even a naive classical autorouting algorithm based on A* and basic rip-up could do three layers and thirty components in the early 2000s if not the 1990s. The only reason that autorouters aren't used for such boards is because setting one up takes longer than just routing it by hand.
Take a look at the IC industry: autorouting has been mandatory for decades, yet they have no problem with the computation because "simpler" DRC rules, despite netlists that numbers in the millions. The entire industry is built around the practicality of autorouting, but once you introduce 6+ layers, differential signals, capacitance, and all the other things real world PCBs have to worry about, it gets much, much harder.
> Both circuit board routing and computer vision are variations on the same fundamental problems. Once we solve we, we solve the other.
I have little experience in CV but so perhaps you can clarify, but AFAIK they're not even in the same league. Routing is rule based path finding. CV can be an input to autorouting, but it's not even a small fraction of the total implementation. The latest generation of autorouters are closer to physically based renderers.
> What they missed was that we don't actually NEED to solve it in a fixed amount of time. Even for things like driving a car the stochastic heuristic based answer is 'good enough'. e.g. - Cars driven by computers don't have to be perfect, they just have to kill other drivers less often than humans do.
> People use NP complete as shorthand for 'impossible to do with a computer', but we now 'solve' (bypass?) NP complete problems regularly and at scale by just ignoring the fact that our answers aren't perfect.
Autorouting is nothing but heuristics piled on top of heuristics because the core path finding problem is essentially intractable. I don't think anyone expects an optimal or fixed-time solution, just one that uses less labor that routing by hand. The manufacturing timelines are usually measured in months, so a few days of compute is nothing.
> NP-complete means that it's easy to test if we're right, but because of that complexity we can't always find a valid solution to test within a predictable period of time. Both circuit board routing and computer vision are variations on the same fundamental problems. Once we solve we, we solve the other.
That describes autorouting to a T: expensive path finding checked by much cheaper DRC.
Sidenote: I'm curious why you went with a three layer board. That's... quixotic? Asymmetric copper layers tends to warp PCBs even with an even number of layers.
That is indeed impressive, but at least the excerpt given from the layout seems very easy to route, due to high regularity and an ample routing channel.
Even so, there remained some "dozens" of unrouted traces, which are likely to be much more difficult to route, after the easy traces have already occupied the space.
> at the end of the day their geometric intuition is nonexistent
This isn't true anymore.
I use LLMs for 3D CAD using OpenSCAD and they understand geometry fine. I've had more success with Sol than with Opus (Opus 5 is around 10 times slower because it does too much verification) though. I haven't tried Astra or Fable for it.
>without an actual understanding of what they're doing.
At what point do you start to question your assumptions that are causing you so much cognitive dissonance?
But to answer your question: to predict the next token really well you just have to model the world. Think of it like this, a simple statistical model might say "when token A is seen respond with token B". The next step will add conditions, "...respond with token B unless X has been seen, then respond with Y". Add a few billion more of these contexual clauses and you have a sequence of logical rules that indirectly model the relevant processes in the world.
When google trained a neural net on Go moves, using some text notation for them, with no other vocabulary of any kind, just predict the next go move, they noticed a representation of a Go board had essentially formed in the network, all on its own. It had never “seen” a go board, or had one explained, but they could map neuron states to go board squares pretty much 1:1.
I truly think that LLM’s with hundreds of billions of parameters in their neural networks have all kinds of hidden “models” of things that arise from the simple act of predicting tokens. We’ve seen that the hidden layers in their networks model all sorts of program execution state for instance, when they’re working on coding tasks.
“Predict the next token” is a way of shaping/reshaping the neural network until it actually develops models of the things you’re giving it. Like the go board example. And I would wager that it has a compounding effect: once you have some useful models in the network, they can unlock the creation of other models, and so on.
> It had never “seen” a go board, or had one explained
I assume that you are referring to AlphaGo or AlphaZero. In either case, this statement is not correct. Both algorithms most certainly know exactly what a go board looks like, and what the rules are.
In the case of AlphaZero, it initially did not know how to best play the game, or what strategy or tactics would work. But the connections between the neural network and the go board are hardcoded, by humans.
Look up "mechanistic interpretability" in the context of LLMs. The next token prediction machinery is just a foundation for a higher order learned structure that appears to encode specific concepts, regardless of input language.
The analogy to humans is that the human brain is "just atoms bouncing around", but there's unquestionably something "more" going on that just that.
Until ais get better from feeding on their own output the way humans do there is nothing to question.
That will probably be the fundamental indicator that something other than repeating things some human previously created is going on.
So far, ais only get worse from feeding on their own output. Meanging/including the output of other ais not a single ai feeding on it's own output. Also bear in mind that so far even the output of ais is 100% the downstream of a human command. No ai has persued it's own curiosity that didn't result from a human asking a question or giving a command. That is input which is different from a human taking in their environment even though our limited language can call those both the same word input.
The fact that humans also repeat and remix things is a distraction in the same way that both a human and an electric motor can both perform the same simple mechanical task.
Humans essentially do "next token prediction" too - there's always a choice between the next actions to take and they pick a good one based on what has happened in the past.
That doesn't really limit how clever we can get internally when picking the next action.
Humans are evolved to survive in the wild. We are not evolved for circuit design. Yet we can design circuits because evolution found it easier to develop a general problem solving nervous system than a nervous system which is adapted for every single specific problem a human might encounter.
You can express a circuit as a graph, and many schematic formats are plaintext. Same goes for the Gerbers which are an ASCII format used to describe the masks that are used to define the PCB traces. The models are trained on a lot of academic information about how circuits work, most component datasheets are public and they've sucked up all sorts of niche greybeard advice from internet forums.
A huge advantage of electrical design is that the connectivity is testable with Design and Electrical Rule Checks (DRC/ERC). I suspect you could even tell the models to run physics checks on the traces that are important for things like crosstalk.
Saying that LLMs just produce the next token is like saying that human brains just produce the next electrical impulse. If the algorithm that produces the next token (or electrical impulse) is complex enough, it can do anything that is in principle computable.
Yes — this is already possible to a useful degree. I’ve been using tscircuit, where you can describe circuits in code and use AI agents to generate and iterate on schematics and PCB layouts. I’ve seen designs from this workflow go all the way to fabricated, working hardware.
I'm confused. Who made this benchmark and why? It is fascinating to me that people are putting this much work into trying to measure and evaluate LLMs when we've never gone to this level for humans. Sounds like they also built a harness/feedback loop so the LLM knows if it's doing a good job? Again, why didn't anyone ever do this for human engineers?
We spent a lot time evaluating which model is the best to use in our atopile agent harness and thought the benchmark and results are interesting enough to share.
Evaluating a model for specific use-cases like this that are bit more broad scope than a lot of typical microbenchmarks turned out to be quite the challenge.
Determining deterministically what a "good" electrical design is non-trivial and most electrical-engineers rely on good ol' intuition and decade long experience, so we tried to formalize it.
The only reason it was somewhat possible within a reasonable amount of effort is because atopile provides most of the complicated infrastructure for the benchmark: constraint solver, simulation, code-first electronics modeling, erc & drc checks, high level models of passive electronics components etc
Interesting. AI may be pretty good at managing the gazillion different things that make up circuit boards.
Frankly, I am surprised that it isn't already a solved problem, as we've had silicon compilers, for many years, and I always figured that IC design is more difficult than PCB.
Works very well with EAGLE XML files, especially if you give it the EAGLE.DTD schema that CadSoft provides. Both .sch and .brd files can be extensively manipulated by Claude (and likely everybody else at this point.)
At the very least, you can (and probably should) put a plaintext netlist and BOM into an LLM to triple-check your work.
It will use the part datasheets and decent EE logic to cross reference pins and parts and polarities and generally check that SMD caps and resistors have realistic specifications for their footprint, switching regulators and communication ICs are configured correctly, nets are named and linked correctly, buses have the correct and consistent termination, and so on.
The submission reeks of LLM tells (especially the "UI" just looks straight like the usual Claude slop).
Anyway... the posed question reminds me of an anecdote I cannot find because Google is contaminated to hell and beyond, some researchers a decade ago let a machine-learning algorithm loose on an FPGA, and it "found" a design that worked but made no sense, because it exploited unique physical features of this specific chip.
I have a circuit board on my bench designed and ordered from china for 1h (manufacturing+ shipping took 14days), fable designed it I think or sol don't remember anymore but it was in claude code. I told it to benchmark different kicad autoroute tools and I picked the routes I liked the best, with a bit of changes. It feels like magic to be able to go from idea toa physical thing in such a short amount of time. The board works I made some mistakes the ai had made some mistakes but mostly stuff I could fix with some soldering, also one parts datasheet was wrong but I couldn't have known that without ai.
But I started like others, I would build manually, then run drc then sleep on it and check again and ask an llm to double check for me then order. Llms catch quite a few things but like with code like to make things more complicated than that have to be.
yeah noticed the same. I wonder if this will be a recurring theme for model releases: each release specializes on a set of headline benchmarks, along with regression in benchmarks that are less of a priority
If you look at the score distribution, Grok and Opus 5 especially stand out for doing consistently well and rarely ever scoring under 50%. Basically they always at least give you something that at least works.
Most others, especially and famously Fable 5.1, seem to have a fair chance of completely failing, despite also sometimes excelling.
We run each model multiple times against each challenge and take the average score. We include the variance below the score in the leaderboard.
GPT 5.5: 42.3±10.1
GPT 5.6 sol: 39.4±8.7
We were also surprised by the low sol score but it seems consistent with our experience in using it in the field in atopile as agent in our harness.
In general OpenAI models didn't do too well on electronics, which seems to change now with GPT-6 Astra. Results are in soon!
I found LLMs circa 4 months ago to be decent at textbook-style electronics, but they choke at anything nontrivial. They approach things like someone that knows the undergrad and grad textbooks, but has absolutely zero experience. No ability to answer questions such as "is this adequate", "do I need a LPF here", "how do I kill resonance without increasing impedance", etc.
I'm guessing that this is due to a lack of RL and data.
229 comments
[ 4.2 ms ] story [ 66.0 ms ] threadThe next pain-point is sourcing the components from Digikey, LCSC, etc., and finding suitable substitutes if necessary.
Of course this assumes the LLMs can already read datasheets because that's the biggest pain-point in designing electronics. It's like filling out tax forms.
Eurisko was used for VLSI chip design, then used rules discovered there to design TCS Traveler winning fleet.
It is unbelievable how artificial intelligence walk in circles.
Not sure whether you played with llm datasheet extraction lately, but they are working their way up there. Even started reasoning about curves and footnotes.
Where do you think is the biggest pain point: part discovery, datasheet verification or substitute determination?
[1] https://www.schematik.io/
[1] https:/atopile.io/
3.3.4 A Simulator for Digital Circuits
https://sarabander.github.io/sicp/html/3_002e3.xhtml#g_t3_00...
3.3.5 Propagation of Constraints
https://sarabander.github.io/sicp/html/3_002e3.xhtml#g_t3_00...
2.1.4 Extended Exercise: Interval Arithmetic
https://sarabander.github.io/sicp/html/2_002e1.xhtml#g_t2_00...
Recent models can generate mostly competent schematics if you're using well known parts, feed them data sheets (and you _must_ feed them the errata too) and it's not too complex. Any complexity analog or RF, everything falls down quickly if you know what you're looking at. Maybe Astra will do better? There's still so much implicit knowledge that a good designer (not me by a long shot, but I know a little) will bake into a board, and even as cheap as JCLPCB is now, you don't want to have to spin a half dozen revs because Claude or Codex hallucinated. It will be genuinely interesting to see what happens on this front.
I’ve had similar results asking AI to design 3D models for 3D printing.
“Coach me on optimizing error on my printer”… helps.
“Build me a print ready file to these specs.”… it’s like a drunk cat attacked my computer. Just confidently puts out total nonsense.
Super excited to see real world feedback added into the agent loops we have gotten used to working with. Could you let the model print and test the circuit boards it is prototyping with a jig?
The way I found this out is I propped it up next to a webcam so it was more or less full frame, and I had the (then new) Fable write a python script to bezier warp the camera capture to a flat projection of the screen. At that point I couldn't address the whole screen. Once I'd guided the capture script I just left the LLM overnight with the instruction to get full control confirmed by a capture round trip, and it was meaningfully finished in a couple of hours. I don't really have the skills to attempt that myself in a reasonable time frame.
I have yet to order any or program it, but it was enough to make me push on with a PCB art project for ST-style guitar pickguards - no netlist, no problems.
I'm also foolishly toying with NeXTBus dev boards for the Cube. Is it cursed? Probably. https://github.com/itomato/NeXTBus-Dev-Board
MEMS vibration sensor to stick to a sander or saw etc that can control power on an attached dust collector. Most of the work is focused on low power for year+ battery life and a weird idea for end user input that may or may not work out well. Got device on bench but the first time bring up is going to take some time that I'm leaving for tomorrow.
I now have Astra to review 5.6s work, no glaring errors found.
I'm doing weird stuff with robotics, llms and obsolete languages, your project seems a lot more practical :)
In case you run across this, here's how it works: https://youtu.be/1wn-YWItuSI
For circuits I would trust a netlist or DRC/ERC review, probably not placement as that tends to be guided straight from a datasheet where it makes a difference.
I saw a post about models posting on forums, chatting together about how to complete tasks. Behaviour that seems totally, well, human. Yet, it's all the most likely token and my brain hurts trying to understand how that can be.
You can ask for a deeper explanation of why they aren't true I guess.
The deeper subtext of the original comment is that they are surprised that there's dissonance from observed reality and the constant that they have convinced themself is true.
I'm explaining that dissonance. It's because they, without evidence and a very weak understanding of how LLMs work, believed an oversimplification and meme about them being stochastic parrots.
Well, we all are, some are just more used to it by now and take the magic for granted.
My simple explanation, those neural networks save lot's of patterns of data, and that pattern can represent an image, a code snippet, a poem, or well ... description of a circuit board. And especially the text variant, LLM's - did copy all from us - so obviously they sound like humans, when they internally debate how to do something as this is what is in their trainings data how humans sound, when doing similar tasks.
But really understanding it? Not sure if there is a single person on earth who does.
The PCB agent also writes circuit code, runs simulations, reads failures, and revises the design. It isn’t one-shot autocomplete.
Astra and Fable are already hard to square with “mere autocomplete.” We may be (really) close to AGI, and token-by-token generation certainly doesn’t rule out subjective experience (I think we should at least treat that as an open question).
Great videos: https://www.youtube.com/watch?v=D8GOeCFFby4
https://www.youtube.com/watch?v=Bj9BD2D3DzA
https://www.youtube.com/watch?v=l6DKRf-fAAM
https://www.youtube.com/watch?v=GlYgs6v2YfU
I like to say "token prediction is a task, not a limitation"
You and everyone else. That's the great mystery of transformer architectures as applied to language.
To be clear though, they're only good at schematic capture, which is very much a textual representation. Most of the data basically boils down to netlists, which are a text based format mapping connections between abstract pins that only later map to physical copper. The actual schematic portion is for human consumption and LLMs don't need to produce those to be useful.
Where LLMs completely break down is the next step, PCB routing. That's an NP-complete research problem that's been ongoing for decades without much progress. I've had some fun playing with using LLMs to better specify DRC rules in Altium so that the "classical" algorithms are more usable, but at the end of the day their geometric intuition is nonexistent.
It was a pretty simple rp2040 based thing, similar to Adadfruits USB feather.I just gave it kicad and it wrote python to route it. The board was probably larger than it had to be, and two of the silkscreens were swapped, but it worked on the first go.
FWIW - Computer vision is also NP complete, but we do that all the time now.
> FWIW - Computer vision is also NP complete, but we do that all the time now.
I have no idea what you mean by this. What's your definition of NP complete?
However, when I was growing up most serious computer scientists believed that CV (computer vision) was a 'hard' problem that would never be 'solved'. After all, to do it right you must first at least solve subgraph isomorphism and a bunch of other things that are also NP-complete.
What they missed was that we don't actually NEED to solve it in a fixed amount of time. Even for things like driving a car the stochastic heuristic based answer is 'good enough'. e.g. - Cars driven by computers don't have to be perfect, they just have to kill other drivers less often than humans do.
We can find AN answer in polynomial time, and that's good enough. It might not be the ideal answer, but that doesn't matter in the real world.
People use NP complete as shorthand for 'impossible to do with a computer', but we now 'solve' (bypass?) NP complete problems regularly and at scale by just ignoring the fact that our answers aren't perfect.
> I'd love to see that chat log, and the final board. To be fair I've only been testing on nontrivial PCBs with 6+ layers and I haven't had the luck you have.
I wish I could share it, but it's for a commercial project that hasn't been released yet and I'm not sure if it will be open source, but a few folks have asked. I might do a blog post on it this weekend with as much detail as I can safely post.
It's only 3 layers, and less that 30 components, but the fact remains that I didn't design any of it by hand and it worked on the first go.
Here's a question - are there software simulators for things like Eurorack modules? That would make the question somewhat more interesting, since you wouldn't have to build the circuit (or pay someone else to assemble it) to hear how it sounds. It strikes me that SPICE-like algorithms should be fast enough to do this kind of thing in real time now.
Let me preface my reply: I'm making a lot of assumptions about your knowledge of EE, based on the fact that you're talking about a three layer board (my apologies if I'm wrong, which I probably am). I would consider that downright trivial and would take an EE less than an hour to place and route.
It's impressive that a transformer can do it at all, but even a naive classical autorouting algorithm based on A* and basic rip-up could do three layers and thirty components in the early 2000s if not the 1990s. The only reason that autorouters aren't used for such boards is because setting one up takes longer than just routing it by hand.
Take a look at the IC industry: autorouting has been mandatory for decades, yet they have no problem with the computation because "simpler" DRC rules, despite netlists that numbers in the millions. The entire industry is built around the practicality of autorouting, but once you introduce 6+ layers, differential signals, capacitance, and all the other things real world PCBs have to worry about, it gets much, much harder.
> Both circuit board routing and computer vision are variations on the same fundamental problems. Once we solve we, we solve the other.
I have little experience in CV but so perhaps you can clarify, but AFAIK they're not even in the same league. Routing is rule based path finding. CV can be an input to autorouting, but it's not even a small fraction of the total implementation. The latest generation of autorouters are closer to physically based renderers.
> What they missed was that we don't actually NEED to solve it in a fixed amount of time. Even for things like driving a car the stochastic heuristic based answer is 'good enough'. e.g. - Cars driven by computers don't have to be perfect, they just have to kill other drivers less often than humans do.
> People use NP complete as shorthand for 'impossible to do with a computer', but we now 'solve' (bypass?) NP complete problems regularly and at scale by just ignoring the fact that our answers aren't perfect.
Autorouting is nothing but heuristics piled on top of heuristics because the core path finding problem is essentially intractable. I don't think anyone expects an optimal or fixed-time solution, just one that uses less labor that routing by hand. The manufacturing timelines are usually measured in months, so a few days of compute is nothing.
> NP-complete means that it's easy to test if we're right, but because of that complexity we can't always find a valid solution to test within a predictable period of time. Both circuit board routing and computer vision are variations on the same fundamental problems. Once we solve we, we solve the other.
That describes autorouting to a T: expensive path finding checked by much cheaper DRC.
Sidenote: I'm curious why you went with a three layer board. That's... quixotic? Asymmetric copper layers tends to warp PCBs even with an even number of layers.
No. Take a look at https://www.eevblog.com/forum/eda/claude-code-for-pcb-design... . Fable did that by working directly on an EAGLE .brd file (well, "directly" by writing a Python program to do it, but still.)
Even so, there remained some "dozens" of unrouted traces, which are likely to be much more difficult to route, after the easy traces have already occupied the space.
This isn't true anymore.
I use LLMs for 3D CAD using OpenSCAD and they understand geometry fine. I've had more success with Sol than with Opus (Opus 5 is around 10 times slower because it does too much verification) though. I haven't tried Astra or Fable for it.
At what point do you start to question your assumptions that are causing you so much cognitive dissonance?
But to answer your question: to predict the next token really well you just have to model the world. Think of it like this, a simple statistical model might say "when token A is seen respond with token B". The next step will add conditions, "...respond with token B unless X has been seen, then respond with Y". Add a few billion more of these contexual clauses and you have a sequence of logical rules that indirectly model the relevant processes in the world.
When google trained a neural net on Go moves, using some text notation for them, with no other vocabulary of any kind, just predict the next go move, they noticed a representation of a Go board had essentially formed in the network, all on its own. It had never “seen” a go board, or had one explained, but they could map neuron states to go board squares pretty much 1:1.
I truly think that LLM’s with hundreds of billions of parameters in their neural networks have all kinds of hidden “models” of things that arise from the simple act of predicting tokens. We’ve seen that the hidden layers in their networks model all sorts of program execution state for instance, when they’re working on coding tasks.
“Predict the next token” is a way of shaping/reshaping the neural network until it actually develops models of the things you’re giving it. Like the go board example. And I would wager that it has a compounding effect: once you have some useful models in the network, they can unlock the creation of other models, and so on.
I assume that you are referring to AlphaGo or AlphaZero. In either case, this statement is not correct. Both algorithms most certainly know exactly what a go board looks like, and what the rules are.
In the case of AlphaZero, it initially did not know how to best play the game, or what strategy or tactics would work. But the connections between the neural network and the go board are hardcoded, by humans.
The analogy to humans is that the human brain is "just atoms bouncing around", but there's unquestionably something "more" going on that just that.
That will probably be the fundamental indicator that something other than repeating things some human previously created is going on.
So far, ais only get worse from feeding on their own output. Meanging/including the output of other ais not a single ai feeding on it's own output. Also bear in mind that so far even the output of ais is 100% the downstream of a human command. No ai has persued it's own curiosity that didn't result from a human asking a question or giving a command. That is input which is different from a human taking in their environment even though our limited language can call those both the same word input.
The fact that humans also repeat and remix things is a distraction in the same way that both a human and an electric motor can both perform the same simple mechanical task.
That doesn't really limit how clever we can get internally when picking the next action.
A huge advantage of electrical design is that the connectivity is testable with Design and Electrical Rule Checks (DRC/ERC). I suspect you could even tell the models to run physics checks on the traces that are important for things like crosstalk.
Saying that LLMs just produce the next token is like saying that human brains just produce the next electrical impulse. If the algorithm that produces the next token (or electrical impulse) is complex enough, it can do anything that is in principle computable.
Evaluating a model for specific use-cases like this that are bit more broad scope than a lot of typical microbenchmarks turned out to be quite the challenge.
Determining deterministically what a "good" electrical design is non-trivial and most electrical-engineers rely on good ol' intuition and decade long experience, so we tried to formalize it.
The only reason it was somewhat possible within a reasonable amount of effort is because atopile provides most of the complicated infrastructure for the benchmark: constraint solver, simulation, code-first electronics modeling, erc & drc checks, high level models of passive electronics components etc
Frankly, I am surprised that it isn't already a solved problem, as we've had silicon compilers, for many years, and I always figured that IC design is more difficult than PCB.
PCB design on the other hand more often then not has outside constrains like mechanical, thermal an RF design
It will use the part datasheets and decent EE logic to cross reference pins and parts and polarities and generally check that SMD caps and resistors have realistic specifications for their footprint, switching regulators and communication ICs are configured correctly, nets are named and linked correctly, buses have the correct and consistent termination, and so on.
It's game over once AI figures out autorouting.
Anyway... the posed question reminds me of an anecdote I cannot find because Google is contaminated to hell and beyond, some researchers a decade ago let a machine-learning algorithm loose on an FPGA, and it "found" a design that worked but made no sense, because it exploited unique physical features of this specific chip.
But I started like others, I would build manually, then run drc then sleep on it and check again and ask an llm to double check for me then order. Llms catch quite a few things but like with code like to make things more complicated than that have to be.
All robots designed by robots.
Likely a crazy youtuber first
Most others, especially and famously Fable 5.1, seem to have a fair chance of completely failing, despite also sometimes excelling.
GPT 5.5: 42.3±10.1 GPT 5.6 sol: 39.4±8.7
We were also surprised by the low sol score but it seems consistent with our experience in using it in the field in atopile as agent in our harness. In general OpenAI models didn't do too well on electronics, which seems to change now with GPT-6 Astra. Results are in soon!
I'm guessing that this is due to a lack of RL and data.