We are converging on text specification to 3D world.
Gaussian Splat technology doesn't get as much hype as it deserves.
The killer app for VR headsets is still to come.
Every time I hear about Gaussian splatting, I think about triangle splatting[0] - GPUs were built to render triangles, so I wonder what games would be like if we supported generating and rendering them as much as we do Gaussian splats.
I think that Gaussian splatting is generally better, as the Gaussians are smoother and easier to optimize, if you can use them.
Things like triangle splatting are useful if you have to integrate into an engine that only support triangles. But with modern graphics pipelines, you can implement Gaussians quite efficiently, including newer techniques like Gaussian point splatting that can scale up seamlessly to enormous scenes:
There are various papers on variants of Gaussian splatting which replace the Gaussians with something else that can be rendered in real time. A triangle soup is just one example. I recall one that uses some sort of tetrahedral foam.
No, nanite is just having different levels of detail for objects, and which LOD gets selected is done automatically by the engine. So users can use ultra-high-def models (including 3d scans of real-world objects), and the engine automatically generates and uses lower polygon versions for when the object is far away or whatever.
Strange, when I looked into it that seemed to be just a desirable side effect with the rendering being done via the triangles - which was also supposedly the reason why you could directly use scanned models, despite them having millions of triangles in their model
But again, I've really only super superficial understanding of this, I just understood it like that back then
Yeah, there were other splats before Gaussian splats, so lots of cool possibilities depending on the style you're after. However, hard triangles have hard edges, and then you might be anti-aliasing those edges in some way. Gaussian splats can just be a triangle or two where the pixel shader feathers towards the edges.
Fable 5.2 is really good at using Blender from my testing. That being said, even smaller models are good enough for junior level work, it just depends on what deliverable or metric you are looking at. Most agentic LLMs can now do low poly assets for example.
Blender can be guided with Opus 5.5 to do reasonably detailed buildings and generate textures, and modify existing models, say a car, and add doors. There is still a lot of handholding, but it's good enough!
I've been using Claude to drive Blender with the official Blender MCP server. Mixed results; but some of them very good.
Claude Opus is very good at creating geometry nodes for specific applications. You have to know what to ask for. It's not a magic genie.
For example, I asked it to make a node setup to create the geometry for a table. It did a mediocre job of it. Then I told it to do research on the nomenclature for tables and create a parametric control interface to generate tables with the UI using the table nomenclature. It did a very good job of that. It set it up so that I could put the units in using Imperial or metric. I also asked it to come up with reasonable default values for the parameters and it did. With just a little tweaking I have a very useful table generator for greyboxing scenes.
I then did a similar round with it for book cases and straightback chairs with useful results.
I asked it to make material nodes for wood: oak, maple, etc. And it did a pretty good job of that. Nothing amazing, but a useful place to start.
It won't be replacing people that know what they are doing, but its useful for giving that person better specifications for what you want.
I had Claude keep notes on the project and it made a skill that has been useful for later projects.
Think of it as a very bright intern that works very fast but has very little practical experience.
The problem, of course, is that if AI replaces interns, then there may not be a future generation of experts. By then, though, the AIs may be good enough to replace the experts. Next year? Next week? Who knows. If you guide it right to learn as it works, it _does_ get better over time.
I have no idea what to expect or what the actual limits of this tech is. I've see it make really hilarious mistakes, then turn around and explain the math its using so well that I can ask for what I want "properly" and it's surprised that that was what I really wanted.
It is like dealing with a very well read alien.
If the AIs don't end up murdering us all in our sleep it could be an amazing world we are entering.
> By then, though, the AIs may be good enough to replace the experts. Next year? Next week? Who knows.
Lately, I've been pondering this question in general and I'm worried the answer might be "Never" for some kinds of work - at least if we don't change how LLMs are currently developed and improved. At a high-level, the problem is they don't 'learn on the job' like a human and writing their memory & learning in .md files works for some things but fails for others.
While the 'rising tide' of baseline LLM knowledge continues to grow, they are still FTIs (Factory Trained Intelligences), meaning they mostly learn from data which can be assembled/tagged or synthesized in mass bulk - plus specific abilities in certain domains which the factory has prioritized for more costly special training. For some broad categories of important work, I suspect there's a significant unfilled gap in our LLM instances being able to effectively crystallize knowledge acquired on-task. Without that, context files will remain helpful crutches not true substitutes.
I would argue that we have no idea how the technology will change. Humans have a poor record of predicting what human based technology can do in a given amount of time. This is non-human intelligences. All bets are off. Maybe the AI can give us a clue. If they feel like it.
Sci-fi stuff aside, it's not just a matter of the AI's telling the supporting tech what to do. We are already at the point where we can't predict what the AI will do to the supporting tech. Blender, for example, is mostly traditional computer science with some physics and art direction thrown in. It's damn hard to use.
This is not a criticism, that's just where we are in developing a UI for the math and physics and art direction. Just using the AI to make for me UI changes, convenience features, and special utilities has drastically changed not only the way I work, but the kind of projects I'll undertake. I think nothing of having Claude whip up a special physics based particle system or compositing stack these days. I could have done that in the past, but it would have been so much work for a one time job I wouldn't consider it. I'd just brute force a solution with manual keyframes and a lot of trial and error.
And while I can program and solve (simple) physics problems, its not my day job. My day job is telling stories and explaining things with animated narratives.
Earlier today I was asked to help with some animations explaining how musical scales worked and to come up with ways to visualize them. Using Claude I've already got Blender doing a credible job at creating musical notation and linking it up to a (sort of) physically accurate sound wave simulation/visualization. Not done yet, but it's looking promising. This is within a few tens of minutes of an instructional designer asking me what might be possible for the topic. And I already have a sound synthesizer written in javascript (by Claude) that I used for a project a while back. Hooking them together is just a json file or two away...
Even if the Super Intelligence never materializes, just using current LLMs as an amplifier for existing technology is going to change things into something unrecognizable. And something that is very dangerous in the hands of human bad actors.
> I would argue that we have no idea how the technology will change.
I didn't discuss how I think the tech will change. I pointed out a current gap in crystalizing knowledge acquisition gained on-task because I feel that gap is still under-appreciated. There's a lot of very interesting work trying to identify how to cost effectively narrow that gap (Dynamic Modular Adapters, Background Distillation, Test-Time Training, etc).
They all can be effective to varying degrees in different ways but they also all have their own gaps and trade-offs. As of now, there's no clear path to per-instance knowledge crystalization that's both broadly generalizable and scales cost-effectively. Will both be solved eventually? I think almost certainly. Will both be solved in the next 2-5 years? Maybe... but it smells like the kind of fundamental problem that's 'wicked' hard, so I wouldn't be surprised if the LLM you and I can afford to daily drive in 2 - 5 years is far more capable and 'smart' but isn't that much better at automatic crystalized learning from on-task knowledge acquisition than the best LLMs are today.
To be clear, LLMs are fabulously useful in myriad ways and I think they'll continue getting much more useful in many more ways. That's so much a given, it makes identifying which remaining gaps are so fundamental there's a good chance they won't be solved in 2-5 years especially interesting.
> Why are we worried that AIs might not be able to replace people?
While I agree that increasing LLM use will, unfortunately, cause disruption and reallocation in job roles, this has always happened whenever technology enables automation. It predates computers going back hundreds of years. A friend of mine has a doctorate in 'History of Technology' and wrote his thesis on the massive employment and societal disruption steam power and railroads caused in the 1800s. The disruption inevitably consolidates roles in some areas while creating new jobs elsewhere. While the directly attributable new jobs are sometimes fewer in number than those lost, they usually have meaningfully higher average compensation.
Although there are a few exceptions, so far the vast majority of these disruptions have, on decadal timescales, resulted in significant economic growth, expanded labor productivity and higher standards of living overall. This has held since Gutenberg's movable type displaced manual scribes, steam shovels displaced dangerous mining jobs and Hollerith's punch card mechanical tabulators replaced huge rooms full of human calculators engaged in mind-numbing manual calculation. Since I generally like humans, I'm glad when fewer of them are forced to feed their families at the cost of black lung in a coal mine, crippling arthritis at 50 or the soul-crushing tedium of paper-pushing. This is part of gradually transitioning to a post-scarcity world where fewer but higher-paying jobs, which are far more productive and engaging, "pay for" more leisure, education, child-rearing, early retirement and unleashing more creative joy by artists, musicians, poets and dreamers. We can't get to 'post-scarcity' by somehow keeping everything the same and one day at 9am flipping a switch. Big changes that matter, whether for better or worse, have always been disruptive.
Individually, disruption is still disruptive and uncomfortable. I get it. No one likes change and uncertainty being forced on them. For hundreds of years each of these transitions has been especially hard on those who are end-of-career or otherwise unwilling to re-skill, learn and transition to new work. And it might be tougher for the current end-of-career generation (including myself) because we're the last generation who will ever have entered the workforce with the expectation we could work for >40 years and retire without ever re-skilling.
While you might think that LLMs are so damn BIG that "this time it's different", there's ample evidence that every generation before us thought that too and I've personally seen different disruptions happen several times in my four decade career, and heard worried claims in the '80s and '90s of "...but it was never like THIS before." It's true the rate of large technological disruptions has been steadily increasing for at least the last 200 years and their size is bigger in absolute terms but not bigger relative to the size of the overall economy, which has grown orders of magnitude. That's why it feels different and bigger to every generation, but it's still the same directional trend you read about in textbooks, except it's now happening to you and all around you. Schumpeter called it "Creative Destruction" for a reason. These are the labor pains and contractions that have always birthed the future.
I truly feel for those personally experiencing uncertainty, discomfort or displacement and being forced into "figuring out an uncertain future." But it's worth pointing out that even the possibility of NOT being under relatively constant uncertainty, discomfort and frequent displacement has only ever existed in all of human history for about the last hundred years. Before that, well... the old adage about wishing on someone "May you live in interesting times" was literally considered a curse. For the last ~50 years, everyone alive a...
Not for work but playing around. Opus 5.5 built a house in Blender based on construction blueprint pdf (vectored graphics, based on which it manually computed scale and dimensions). It did more in 1.5h than I (total 0 exp) did in multiple months working on and off on the project. It flagged some blueprint flaws I noticed myself too. It even mapped terrain.
Later I gave it 3d rendered images (just png's!) of interior design of specific rooms. It created all the furniture in Blender with 95% accuracy (the 5% were small geometry quirks and shortcuts the model took, but nothing that can't be fixed manually).
Bundle support in geometry nodes is huge for complex pipelines and workflows, particularly when combined with lists and closures. I have been running on the alpha builds to get access to them in my workflows.
It took me a while to properly understand it, but it's genuinely great. It maps so much more of what I expect from traditional coding, scripting etc into the visual paradigm and makes an awful lot easier to use.
Also, should out to databpy and nodebpy, both of which are indispensable and make using nodes and blender so much nicer:
Apparently, while Gaussian splatting is not the core technique for all world models, it has become a dominant representation and rendering backend for spatial and reconstructive world models.
Pretty cool that Gaussian Splatting is included in Blender. It always amazes me how many (sometimes pretty hidden) features Blender provides while still being rather a lean piece of software.
I believe though that Gaussian Splatting is not ideal for getting well-structured mesh data with limited vertices for typical video game usage. I'm wondering where this is going in the future.
He is very frequently posting news about the splat world as well as quite many tests and demos with different software or hardware. I would expect him to cover Blender specific stuff soon, but it's a great source already for existing stuff.
You show something to your boss, he pats you on the shoulder and says "Perfect! Just move chairs 2 feet left and ship it". How does one do it with gaussian splats?
Traditional photogrammetry produces polygon models which are completely different from radiance fields that splatting does. I'd compare the difficulty to editing a vector image vs editing raster image.
You select the points involved in the chair and move them and then you fix the holes in the floor. Gaussian splats are just 3D point clouds with a bunch of extra values per point.
Point clouds generally (often) represent surfaces (in a sense), splats represent light fields. The key thing is that a splat (or set of splats) is not related to a specific object necessarily, so it is non-trivial to select all splats involved in the chair.
There are ways to convert point clouds from splats into regular editable meshes and primitives. This is one popular method but there are many (look up surface reconstruction if you're interested):
Tried the beta on my M4 a couple of weeks ago on a scene of 4M splats (exported from spirula studio, great tool) , and found that it's immediately very mature, fast, and well-integrated. I could not find anyinconstencies with the rest of the interface. Congrats to the team!
I just started work on a screencast video for this year's EmacsConf. Previously I had used ScreenFlow on macOS for this sort of thing, but having since fled macOS I was casting about for something in the Linux world to replace my workflow.
I was pleasantly surprised to learn that Blender has a fairly competent video editor built into it. I was also stoked to learn that Blender has a built in Python interpreter, and it was pretty easy to build a Python startup module that registers a timer callback that polls a directory looking for evidence of new OBS screen recordings, and upon finding one automatically appends it to the open project. An integration with i3 and the OBS WebSocket API made it possible to start, stop, and finalize screen recordings to ensure that all the filesystem writes were atomic.
Another Blender Python startup module was able to implement all the timeline editing commands that are hardwired into my fingers from a decade of ScreenFlow and Logic Pro use, so I was able to be reasonably productive right away. Blender 5.3 is the first version I've used, so I don't have anything to compare it to, but color me impressed! It supports my old 5700xt GPU via Vulkan, so with a little more tweaking I was able to get hardware H264 encode/decode working too.
congrats on the new release! I hope with all the attention Blender has been getting recently (I've been seeing it in a bunch of tech demos from some gigantic companies) it gets a nice boost of donations.
Watching the release change log Video while knowing nothing about modeling was awesome. I love getting a small glimpse into a whole different world of nerds
And much much more. Blender release videos are genuinely delightful to watch and they get better the more you learn about the software. More features are added, AND they keep polishing what's there, like for example weight painting finally having auto-normalize toggled on by default.
It's also wonderful to see the video sequencer get more love and node parity grow between the shader editor, geometry nodes, and the compositor.
67 comments
[ 4.1 ms ] story [ 64.7 ms ] threadhttps://news.ycombinator.com/item?id=46670024
[0] https://trianglesplatting.github.io/
Things like triangle splatting are useful if you have to integrate into an engine that only support triangles. But with modern graphics pipelines, you can implement Gaussians quite efficiently, including newer techniques like Gaussian point splatting that can scale up seamlessly to enormous scenes:
* Video: https://www.youtube.com/watch?v=2AWv_nIfp-U
* Paper: https://jorisar.nl/gaussian_point_splatting/gaussian_point_s...
* Repo: https://github.com/JorisAR/gaussian-point-splatting
https://rongliu-leo.github.io/universal-beta-splatting/
https://linus-franke.com/hibs/
To be clear, I'm not proficient in it's use, just watched a few videos about it's theory when it was initially announced
But again, I've really only super superficial understanding of this, I just understood it like that back then
runs on a phone: https://storage.googleapis.com/jax3d-public/projects/mobilen...
https://threejseval.com/
Though with some graphic design tools you have to proxy it with AppleScript because there is no other automatable interface with the design tool.
Claude Opus is very good at creating geometry nodes for specific applications. You have to know what to ask for. It's not a magic genie.
For example, I asked it to make a node setup to create the geometry for a table. It did a mediocre job of it. Then I told it to do research on the nomenclature for tables and create a parametric control interface to generate tables with the UI using the table nomenclature. It did a very good job of that. It set it up so that I could put the units in using Imperial or metric. I also asked it to come up with reasonable default values for the parameters and it did. With just a little tweaking I have a very useful table generator for greyboxing scenes.
I then did a similar round with it for book cases and straightback chairs with useful results.
I asked it to make material nodes for wood: oak, maple, etc. And it did a pretty good job of that. Nothing amazing, but a useful place to start.
It won't be replacing people that know what they are doing, but its useful for giving that person better specifications for what you want.
I had Claude keep notes on the project and it made a skill that has been useful for later projects.
Think of it as a very bright intern that works very fast but has very little practical experience.
The problem, of course, is that if AI replaces interns, then there may not be a future generation of experts. By then, though, the AIs may be good enough to replace the experts. Next year? Next week? Who knows. If you guide it right to learn as it works, it _does_ get better over time.
I have no idea what to expect or what the actual limits of this tech is. I've see it make really hilarious mistakes, then turn around and explain the math its using so well that I can ask for what I want "properly" and it's surprised that that was what I really wanted.
It is like dealing with a very well read alien.
If the AIs don't end up murdering us all in our sleep it could be an amazing world we are entering.
Lately, I've been pondering this question in general and I'm worried the answer might be "Never" for some kinds of work - at least if we don't change how LLMs are currently developed and improved. At a high-level, the problem is they don't 'learn on the job' like a human and writing their memory & learning in .md files works for some things but fails for others.
While the 'rising tide' of baseline LLM knowledge continues to grow, they are still FTIs (Factory Trained Intelligences), meaning they mostly learn from data which can be assembled/tagged or synthesized in mass bulk - plus specific abilities in certain domains which the factory has prioritized for more costly special training. For some broad categories of important work, I suspect there's a significant unfilled gap in our LLM instances being able to effectively crystallize knowledge acquired on-task. Without that, context files will remain helpful crutches not true substitutes.
Sci-fi stuff aside, it's not just a matter of the AI's telling the supporting tech what to do. We are already at the point where we can't predict what the AI will do to the supporting tech. Blender, for example, is mostly traditional computer science with some physics and art direction thrown in. It's damn hard to use.
This is not a criticism, that's just where we are in developing a UI for the math and physics and art direction. Just using the AI to make for me UI changes, convenience features, and special utilities has drastically changed not only the way I work, but the kind of projects I'll undertake. I think nothing of having Claude whip up a special physics based particle system or compositing stack these days. I could have done that in the past, but it would have been so much work for a one time job I wouldn't consider it. I'd just brute force a solution with manual keyframes and a lot of trial and error.
And while I can program and solve (simple) physics problems, its not my day job. My day job is telling stories and explaining things with animated narratives.
Earlier today I was asked to help with some animations explaining how musical scales worked and to come up with ways to visualize them. Using Claude I've already got Blender doing a credible job at creating musical notation and linking it up to a (sort of) physically accurate sound wave simulation/visualization. Not done yet, but it's looking promising. This is within a few tens of minutes of an instructional designer asking me what might be possible for the topic. And I already have a sound synthesizer written in javascript (by Claude) that I used for a project a while back. Hooking them together is just a json file or two away...
Even if the Super Intelligence never materializes, just using current LLMs as an amplifier for existing technology is going to change things into something unrecognizable. And something that is very dangerous in the hands of human bad actors.
I didn't discuss how I think the tech will change. I pointed out a current gap in crystalizing knowledge acquisition gained on-task because I feel that gap is still under-appreciated. There's a lot of very interesting work trying to identify how to cost effectively narrow that gap (Dynamic Modular Adapters, Background Distillation, Test-Time Training, etc).
They all can be effective to varying degrees in different ways but they also all have their own gaps and trade-offs. As of now, there's no clear path to per-instance knowledge crystalization that's both broadly generalizable and scales cost-effectively. Will both be solved eventually? I think almost certainly. Will both be solved in the next 2-5 years? Maybe... but it smells like the kind of fundamental problem that's 'wicked' hard, so I wouldn't be surprised if the LLM you and I can afford to daily drive in 2 - 5 years is far more capable and 'smart' but isn't that much better at automatic crystalized learning from on-task knowledge acquisition than the best LLMs are today.
To be clear, LLMs are fabulously useful in myriad ways and I think they'll continue getting much more useful in many more ways. That's so much a given, it makes identifying which remaining gaps are so fundamental there's a good chance they won't be solved in 2-5 years especially interesting.
>Lately, I've been pondering this question in general and I'm worried the answer might be "Never" for some kinds of work
Why are we worried that AIs might not be able to replace people? That seems like quite the dystopian statement.
While I agree that increasing LLM use will, unfortunately, cause disruption and reallocation in job roles, this has always happened whenever technology enables automation. It predates computers going back hundreds of years. A friend of mine has a doctorate in 'History of Technology' and wrote his thesis on the massive employment and societal disruption steam power and railroads caused in the 1800s. The disruption inevitably consolidates roles in some areas while creating new jobs elsewhere. While the directly attributable new jobs are sometimes fewer in number than those lost, they usually have meaningfully higher average compensation.
Although there are a few exceptions, so far the vast majority of these disruptions have, on decadal timescales, resulted in significant economic growth, expanded labor productivity and higher standards of living overall. This has held since Gutenberg's movable type displaced manual scribes, steam shovels displaced dangerous mining jobs and Hollerith's punch card mechanical tabulators replaced huge rooms full of human calculators engaged in mind-numbing manual calculation. Since I generally like humans, I'm glad when fewer of them are forced to feed their families at the cost of black lung in a coal mine, crippling arthritis at 50 or the soul-crushing tedium of paper-pushing. This is part of gradually transitioning to a post-scarcity world where fewer but higher-paying jobs, which are far more productive and engaging, "pay for" more leisure, education, child-rearing, early retirement and unleashing more creative joy by artists, musicians, poets and dreamers. We can't get to 'post-scarcity' by somehow keeping everything the same and one day at 9am flipping a switch. Big changes that matter, whether for better or worse, have always been disruptive.
Individually, disruption is still disruptive and uncomfortable. I get it. No one likes change and uncertainty being forced on them. For hundreds of years each of these transitions has been especially hard on those who are end-of-career or otherwise unwilling to re-skill, learn and transition to new work. And it might be tougher for the current end-of-career generation (including myself) because we're the last generation who will ever have entered the workforce with the expectation we could work for >40 years and retire without ever re-skilling.
While you might think that LLMs are so damn BIG that "this time it's different", there's ample evidence that every generation before us thought that too and I've personally seen different disruptions happen several times in my four decade career, and heard worried claims in the '80s and '90s of "...but it was never like THIS before." It's true the rate of large technological disruptions has been steadily increasing for at least the last 200 years and their size is bigger in absolute terms but not bigger relative to the size of the overall economy, which has grown orders of magnitude. That's why it feels different and bigger to every generation, but it's still the same directional trend you read about in textbooks, except it's now happening to you and all around you. Schumpeter called it "Creative Destruction" for a reason. These are the labor pains and contractions that have always birthed the future.
I truly feel for those personally experiencing uncertainty, discomfort or displacement and being forced into "figuring out an uncertain future." But it's worth pointing out that even the possibility of NOT being under relatively constant uncertainty, discomfort and frequent displacement has only ever existed in all of human history for about the last hundred years. Before that, well... the old adage about wishing on someone "May you live in interesting times" was literally considered a curse. For the last ~50 years, everyone alive a...
Later I gave it 3d rendered images (just png's!) of interior design of specific rooms. It created all the furniture in Blender with 95% accuracy (the 5% were small geometry quirks and shortcuts the model took, but nothing that can't be fixed manually).
It took me a while to properly understand it, but it's genuinely great. It maps so much more of what I expect from traditional coding, scripting etc into the visual paradigm and makes an awful lot easier to use.
Also, should out to databpy and nodebpy, both of which are indispensable and make using nodes and blender so much nicer:
https://github.com/BradyAJohnston/databpy https://github.com/BradyAJohnston/nodebpy
Thanks!
He is very frequently posting news about the splat world as well as quite many tests and demos with different software or hardware. I would expect him to cover Blender specific stuff soon, but it's a great source already for existing stuff.
They have easy subscription options, even a monthly coffee adds up! Thanks!
Yes you can mitigate the problem. But the same is largely true for splats as well.
It's fair to say one is easier than the other, but it's a difference of degree rather than of kind.
https://imagine.enpc.fr/~guedona/sugar/
I was pleasantly surprised to learn that Blender has a fairly competent video editor built into it. I was also stoked to learn that Blender has a built in Python interpreter, and it was pretty easy to build a Python startup module that registers a timer callback that polls a directory looking for evidence of new OBS screen recordings, and upon finding one automatically appends it to the open project. An integration with i3 and the OBS WebSocket API made it possible to start, stop, and finalize screen recordings to ensure that all the filesystem writes were atomic.
Another Blender Python startup module was able to implement all the timeline editing commands that are hardwired into my fingers from a decade of ScreenFlow and Logic Pro use, so I was able to be reasonably productive right away. Blender 5.3 is the first version I've used, so I don't have anything to compare it to, but color me impressed! It supports my old 5700xt GPU via Vulkan, so with a little more tweaking I was able to get hardware H264 encode/decode working too.
Pretty cool, no need for ScreenFlow!
In its early days most competing software was still distributed via disc. A small download size made it more accessible to more people
https://www.youtube.com/watch?v=bvB4PHbzn0E https://fund.blender.org/
Otherwise they are making the video editor better, which is great.
It's also wonderful to see the video sequencer get more love and node parity grow between the shader editor, geometry nodes, and the compositor.