I'm working on a procedural design & animation tool for WebGL ( https://polygonjs.com ), and I've recently added some early CAD modeling capabilities ( https://www.youtube.com/watch?v=6AoL2xRLiOs&t=1s ). One of my next steps is now to add natural language to setup those nodes.
So it won't be just for CAD, but more general 3D (material creation, physics, particles, etc). And it is very early in my research, so I'm still unsure how far things can go, or how usable could a scene created by this technique be. But it would at the very least help discover features, and try ideas even faster.
Thank you! And yes, 3D apps are already full of performance challenges, it's even more so when made in javascript. I probably spend 30% of my time doing optimizations (while trying to avoid premature optimizations!).
The biggest win really is to avoid allocating memory when the scene plays. So as you set up your scene, I have all sorts of tricks to pre-evaluate what the scene will need, and allocate that once only. That's probably what made the biggest difference to keep a smooth FPS. But that's only on the javascript side, there is more pitfalls when generating shaders.
And for the CAD part, it's powered by opencascade/wasm, so that's reasonably fast for now. But I'm sure this will get battle tested soon, with larger models and nodes setups.
If I were gonna work on a tool in that space, I'd try to get it running with eg FreeCAD macros or OpenSCAD before trying to get it to work with something like Solidworks.
For anything sufficiently complex, a natural language interface is going to be hard though. How do you get it to understand you want it toleranced in Y way vs X way? How do you get it to fix the non-manifold mess it created by destroying all the constraints when you asked it to fillet something? Not impossible certainly, but a lot of work for the library.
7 comments
[ 2.8 ms ] story [ 28.5 ms ] threadSo it won't be just for CAD, but more general 3D (material creation, physics, particles, etc). And it is very early in my research, so I'm still unsure how far things can go, or how usable could a scene created by this technique be. But it would at the very least help discover features, and try ideas even faster.
The biggest win really is to avoid allocating memory when the scene plays. So as you set up your scene, I have all sorts of tricks to pre-evaluate what the scene will need, and allocate that once only. That's probably what made the biggest difference to keep a smooth FPS. But that's only on the javascript side, there is more pitfalls when generating shaders.
And for the CAD part, it's powered by opencascade/wasm, so that's reasonably fast for now. But I'm sure this will get battle tested soon, with larger models and nodes setups.
https://github.com/gd3kr/BlenderGPT - actually an attempt to make something useful, though its working with meshes and not step files.
https://github.com/JennyList/Breakfast-by-ChatGPT - project to get GPT to output OpenSCAD. Mostly just messing around but interesting attempt nonetheless.
If I were gonna work on a tool in that space, I'd try to get it running with eg FreeCAD macros or OpenSCAD before trying to get it to work with something like Solidworks.
For anything sufficiently complex, a natural language interface is going to be hard though. How do you get it to understand you want it toleranced in Y way vs X way? How do you get it to fix the non-manifold mess it created by destroying all the constraints when you asked it to fillet something? Not impossible certainly, but a lot of work for the library.
https://spline.design/ai
I suspect someone else is working on the accuracy necessary for actual CAD work.
It's not using natural language, but now it could easily be modified to support it.