> Artificial Intelligence researchers have been trying to get [X] to [Y] for over 65 years
For 10,000 different problems. A great many of which have been solved in recent years.
Robotics is improving at a very fast clip, relative to most tech. I am unaware of any barrier, or any reason to infer there is one, for dextrous robots.
I think the primary difference between AI software models and services, and robotic AI, is economics.
The cost per task for AI software is .... very small. And the cost per task for a robot with AI is ... many orders of magnitude over that.
The marginal costs of serving one more customer are completely incomparable.
It's just a push of a button to replace the "fleet" of chatbots a million customers are using. Something unthinkable in the hardware world.
The seemingly lower level of effort and progress is because hardware that could operate in our real world with the same dexterity that ChatGPT/Claude can converse online, will be extremely expensive at first.
Robotics companies are not just focused on dexterity. They are focused on improvements to dexterity that stay within a very tight economic envelope. Inexpensive dexterity is going to take a while.
> I am unaware of any barrier, or any reason to infer there is one, for dextrous robots.
I don't think there's a fundamental barrier to building a humanoid robot but the cost will be an extremely high barrier to adoption.
A human is nature's ultimate robot: hundreds of servos, millions of sensors, self-assembling from a bag of rice, self-repairing for minor damage. You just can't beat that, not for a very long time.
Very interesting point that while we've figured out how to digitize images, text and sounds we haven't digitized touch. At best we can describe in words what a touch sensation was like. Smell is in a similar situation. We haven't digitized it at all.
Touch is a 2D field of 3D vectors. Easily stored and transmitted as images, and easily processed by neural nets. You could add temperature and pain/damage channels if you want, though they don't seem essential for most manipulation tasks. (Actually I don't believe touch is as essential as he argues anyway. Of course someone who learned a task with touch will struggle without it, but they can still do it and would quickly change strategies and improve.)
The problem with touch is making sensors that are cheap and durable and light and thin and repairable and sensitive and shape-conforming. Representation is trivial in comparison.
> Before too long (and we already start to see this) humanoid robots will get wheels for feet, at first two, and later maybe more, with nothing that any longer really resembles human legs in gross form. But they will still be called humanoid robots.
Totally agree. Wheels are cheaper, more durable and more effective than legs.
Human would have wheels if there was an evolution pathway to wheels.
It's hard to see one. Even a nice flat world with ample incentive and taking good "bearings" for granted, how can you evolve a wheel-organ that maintains a biological connection as well as being able to rotate an indefinite number of time?
A few difficult and grotesque endpoints:
* The wheel only rotates a fixed number of times before the creature must pivot and "unwind" in the opposite direction. This one seems most plausible, but it's not a real wheel.
* The main body builds replacement wheels internally (like tooth enamel) and periodically ejects a "dead" wheel which can be placed onto a spoke. This option would make it easier to generate very tough rim materials though.
* A biological quick-release/quick-connect system, where the wheel-organ disconnects to move, but then reconnects to flush waste and get more nutrients.
* A communal organism, where wheel-creatures are alive and semi-autonomous, with their own own way to acquire nutrients. Perhaps they would, er... suckle. Eeugh.
In any case, it seems like a "simple" problem to solve. An accelerometer chip costs a few cents and the data rates can be handled by a very light wiring harness, ex I2C.
So embedding such a sensor in every rigid component, wiring a single data line to all of them (using the chassis as electrical ground) and feeding the data back to the model seems a trivial way to work around this problem without any kind of real pressure sensitivity. The model knows the inputs it gives to the actuators/servos, so it will quickly learn to predict the free mechanical behavior of the body, and use any deviation to derive data equivalent to pressure and force feedback.
Another possible source of data is the driving current of the motors/actuators which is proportional to the mechanical resistance the limb encounters. All sorts of garbage sources of data that were almost useless noise in the classical approach become valuable with a model large enough.
> Another possible source of data is the driving current of the motors/actuators which is proportional to the mechanical resistance the limb encounters.
The problem is precisely the actuators. A lot of a human's muscles actually come in pairs - agonist and antagonist muscles [1], and it's hard to match the way human muscles work and their relatively tiny size in a non-biological actuator.
Just take your elbow and angle it to 90 degrees, then rapidly close it so your upper and lower arm are (almost) in parallel. An absolutely easy, trivial task to do for your pair of muscles controlling the tendons. But now, try to replicate even this small feat in a motor based actuator. You either use some worm gear to prevent the limb from going in the wrong direction but lose speed, or you use some sort of stepper motor that's very hard to control and takes up a lot of space.
There's a number of "robotics and embodied AI" ETFs out there that should show up with a quick search. I don't have an opinion as to their quality so you'd have to do your own research.
It's been tried a number of times already though, robotics companies have been around for decades, Sony, Boston Dynamics, Hyundai and many others are already in the space (and some of those are on the stonks market). I don't think it'll become any bigger than what it is. Also since many have tried to make it hype already, Tesla being the latest.
If you asked someone 300 years ago what an automated dishwashing machine would've looked like, it would be a lot more like a person than the wet cupboard we have now. I'm assuming many tasks will be like that -- it's more of a lack of imagination for why we say we need a humanoid robot to solve that task. I'm assuming it'll be the minority of tasks where it make it makes sense for that
Right but that's very task specific, and what many people want is a single robot which can do many different tasks, and do so without modifying the existing environment. I would love a robot which could cook and clean and do laundry (including folding) but I still need to live in the same space it would use. The most obvious way to do that is a humanoid robot, which is why nanny companies are working on it, and here he's arguing that's not going to work.
A robot that isn't stationary, in a home or in a factory, wants legs. Wheels are fine for cars but not great for stepping over things (like on cluttered floors) and stairs. So legs, assuming we've got the compute and algorithms to get them to work well, only make sense. The rest allows for application of creativity. As a human, have a head, my brain is in it, as are my eyes. Humanoid robot doesn't need a head, and can have cameras in its chest and on its back, and then also have its brain in the chest. Depending on what's useful, it doesn't need to be limited to two arms. It could have one centrally mounted in its chest, with two cheaper ones on both sides. Or four, two on each side. I've wished for three hands before. The problem though is that they look weird. Any non-traditional design is going to fall into the uncanny valley, so that no matter how much better your non-traditionally armed robot is technically, it's just not gonna sell to the mass market. We only have to look at weird cars/vehicles which have a history of being boondoggles. So it's not a failure of imagination, and more a matter of practicality.
> No sense of touch. Human hands are packed absolutely full of sensors. [...] We store energy in our tendons and reuse it on the next step
Side-rant: As cool as some cyberpunk/sci-fi ideas are, I can't imagine a widespread elective mechanical limb replacement within the lifetime of anyone here. We dramatically under-estimate how amazing our normal limbs are. I mean, they're literally swarms of nanobots beyond human comprehension. To recycle an old comment against mechanical limbs:
________
[...] just remember that you're sacrificing raw force/speed for a system with a great deal of other trade-offs which would be difficult for modern science to replicate.
1. Supports a very large number of individual movements and articulations
2. Meets certain weight-restrictions (overall system must be near-buoyant in water)
3. Supports a wide variety of automatic self-repair techniques, many of which can occur without ceasing operation
4. Is entirely produced and usually maintained by unskilled (unconscious?) labor from common raw materials
5. Contains a comprehensive suite of sensors
6. Not too brittle, flexes to store and release mechanical energy from certain impacts
7. Selectively reinforces itself when strain is detected
8. Has areas for the storage of long-term energy reserves, which double as an impact cushion
9. Houses small fabricators to replenish some of its own operating fluids
10. Subsystems for thermal management (evaporative cooling, automatic micro-activation)
_______________
I predict the closest thing we might see instead will be just growing replacement biological limbs, followed by waldoes where you remotely control an arm without losing your own.
I'd add an 11th point to expand on #1: supports a very wide range of movement speeds, movement force/torque and movement precision.
Take the elbow joint and the muscles it's connected to. It supports very fine precision, slow speed operations as well as high speed but at the same time the same operation at high speeds - say, lifting yourself up on a horizontal bar, assuming adequate strength you can either do a slow or a fast lift, and both at enough precision and torque to prevent your body mass from impacting to the bar which is another feat in itself.
Now try to replicate that with a classic mechanical mechanism, you'll always lose either precision, speed or torque.
Brooks describes how speech preprocessed by chopping it up into short time segments and converting the segments to the frequency domain. He then bemoans the fact that there's no similar preprocessing for touch data. OK.
But then he goes on to vision, where the form that goes into vision processing today is an array of pixels. That's not much preprocessing. That's pretty much what existed at the image sensor. Older approaches to vision processing had feature extractors, with various human-defined feature sets. That was a dead end. Today's neural nets find their own features to extract.
Touch sensing suffers from sensor problems. A few high-detail skin-like sensors have been built. Ruggedness and wear are a big problem.
Consider, though, a rigid tool such as an end wrench. Humans can feel out the position of a bolt with an end wrench, get the wrench around the bolt, and apply pressure to tighten or loosen a nut. Yet the total information available is position plus six degrees of freedom of force. If the business end of your tool is rigid, the amount of info you can get from it is quite limited. That doesn't mean you can't get a lot done. (I fooled around with this idea pre-LLM era, but didn't get very far.) That's at least a way to get warmed up on the problem.
Here's a video of a surgeon practicing by folding paper cranes with small surgical tools.[1] These are rigid tools, so the amount of touch information available is limited. That's a good problem to work on.
Not sure which lab (I think google?) it was, but there was a recent demo of a ML-model driven robot that folded paper in that style as one of the tasks.
> When an instability is detected while walking and the robot stabilizes after pumping energy into the system all is good, as that excess energy is taken out of the system by counter movements of the legs pushing against the ground over the next few hundred milliseconds. But if the robot happens to fall, the legs have a lot of free kinetic energy, rapidly accelerating them, often in free space. If there is anything in the way it gets a really solid whack of metal against it. And if that anything happens to be a living creature it will often be injured, perhaps severely.
True, most of the tasks can be done with off-the-shelf hardware already. But single task robotics is already a solved problem, what the humanoid robots are about is multi-task, aimed at replacing the tasks that still require human hands / legs / eyes / brains / etc.
But I think most of those can be replaced by existing robotics as well anyway. I mean take car manufacturing, over time more and more humans were replaced by robots, and nowadays the newest car factories are mostly automated (see lights-out manufacturing: https://en.wikipedia.org/wiki/Lights_out_(manufacturing)). Interestingly a Japanese robot factory has been lights-out since 2001, where they can run for 30 days on end without any lights, humans, heating or cooling.
I misread the title and I thought it was about humans.
And I could see it. With prevalence of screens kids already don't learn a lot of dexterity that previous generations have learned. Their grip strength is weak and capacity for fine 3d motions is probably underdeveloped as well.
Last week I've seen an intelligent and normally developing 7 year old kid asking mum to operate a small screwdriver to get to the battery compartment of a toy because that apparently was beyond his competence.
Now with recent developments in robotics, fully neural controllers and training in simulated environments there could be that modern babies will have very little tasks requiring dexterity left when they grow up.
> because that apparently was beyond his competence.
This has almost nothing to do with nature (barring a development issue).
This has to do with nurture. Every time they went to do something with a tool a helicopter gunship of a parent showed up to tell them no. Now they have a learned helplessness when it comes to these things.
But that's not really any different then when I was a kid so very long ago. At 4 or 5 I was given a stack of old broken radios and took them to the garage for a rip and tear session. I got to look at all their pretty electronic guts that fascinated me. There were plenty of other parents of that time that would have been horrified to see their kids do something similar.
Isn't another hardware problem being ignored here? Pound-for-pound muscle fibers are just superior to what you can achieve with electric motors or pneumatics.
Take size, strength, precision, longevity, and speed. It's not hard to match or beat organic muscle fibers on one or two of these dimensions with an electrically driven system, but if it does, it's going to neglect other dimensions to such a degree as to put building a humanoid robot that achieves parity with a human completely out of reach.
You can slather as much AI as you want on top of inadequate hardware - it's not going to help.
I don't follow (possibly through my own limitations) the main argument.
> The center piece of my argument is that the brute force learning approaches that everyone rightfully touts as great achievements relied on case-specific very carefully engineered front-ends to extract the right data from the cacophony of raw signals that the real-world presents.
In nearly each of the preceding examples, isn't the argument really about the boundaries that define the learning machine? Just because data preparation / formatting / sampling / serialization is more cost-effective to do externally from the learning machine, doesn't mean that boundary is necessary. One could build all of this directly inside the boundary of the learning machine and feed it the raw, messy, real world signals.
Also, humans having plentiful learning aids doing "tokenization", as anyone who helped a child learn to count has experienced first hand.
I think he meant to write "Prologue" instead of "PROLOG".
I spent a few minutes excitedly trying to figure out how one of my favourite declarative programming languages was used to solve modern robotic sensing problems, only to realise it was probably just a misspelling ... :(
Try threading a nut onto a bolt. Pay attention to how your fingers feel when the threads engage properly and you aren't cross-threading it.
Next, insert a Standard screwdriver into a screw head, set the screw in place, and screw it in. In order to make it work, you have to push and torque it at the same time, and not let the blade slip out of the hole or damage the screw head.
If you think this is easy, try to teach a kid to do it. Watch them struggle to control the nut and the screwdriver.
Our hands are really, really good at both major motor control and very fine motor control.
We said the same thing about language to be honest- the nuances of words and concepts are too hard for a word generator to correctly put together and now we have LLMs. We said the same thing about video generation where the nuances of light and shadow and micro expressions would be hard to replicate and LLMs are doing a pretty good job with that. We’re just waiting for physical LLMs, it will happen at some point.
Who is kidding who?
Just watch a film of a single cell critter, approach something and one , go yum! and engulf it or two go ahhhhhhhh!, and run away
I believe the full technical explanation for that goes, mumble,mumble,chemical receptors something, mumble mumble.Humans are sensitive to certain chemicals in the parts per billion, and your finger can detect surface roughness down to 1/1000'th of an inch, thats the standard issue, exceptional indivuals with training will perform significantly better.
Confusing title because of the choice for the word "humanoid". When I see that, I expect we're talking about a creature shaped like a human. The word for human-shaped robots has always been "android". Can we please just continue using that?
50 comments
[ 4.2 ms ] story [ 57.2 ms ] threadI can put on a thick glove (losing touch and pressure sensitivity all together) and grab a fragile glass without breaking it.
For 10,000 different problems. A great many of which have been solved in recent years.
Robotics is improving at a very fast clip, relative to most tech. I am unaware of any barrier, or any reason to infer there is one, for dextrous robots.
I think the primary difference between AI software models and services, and robotic AI, is economics.
The cost per task for AI software is .... very small. And the cost per task for a robot with AI is ... many orders of magnitude over that.
The marginal costs of serving one more customer are completely incomparable.
It's just a push of a button to replace the "fleet" of chatbots a million customers are using. Something unthinkable in the hardware world.
The seemingly lower level of effort and progress is because hardware that could operate in our real world with the same dexterity that ChatGPT/Claude can converse online, will be extremely expensive at first.
Robotics companies are not just focused on dexterity. They are focused on improvements to dexterity that stay within a very tight economic envelope. Inexpensive dexterity is going to take a while.
I don't think there's a fundamental barrier to building a humanoid robot but the cost will be an extremely high barrier to adoption.
A human is nature's ultimate robot: hundreds of servos, millions of sensors, self-assembling from a bag of rice, self-repairing for minor damage. You just can't beat that, not for a very long time.
The problem with touch is making sensors that are cheap and durable and light and thin and repairable and sensitive and shape-conforming. Representation is trivial in comparison.
(I'm not disagreeing with the author, just sharing an article that is interesting/relevant.)
Totally agree. Wheels are cheaper, more durable and more effective than legs.
Human would have wheels if there was an evolution pathway to wheels.
It's hard to see one. Even a nice flat world with ample incentive and taking good "bearings" for granted, how can you evolve a wheel-organ that maintains a biological connection as well as being able to rotate an indefinite number of time?
A few difficult and grotesque endpoints:
* The wheel only rotates a fixed number of times before the creature must pivot and "unwind" in the opposite direction. This one seems most plausible, but it's not a real wheel.
* The main body builds replacement wheels internally (like tooth enamel) and periodically ejects a "dead" wheel which can be placed onto a spoke. This option would make it easier to generate very tough rim materials though.
* A biological quick-release/quick-connect system, where the wheel-organ disconnects to move, but then reconnects to flush waste and get more nutrients.
* A communal organism, where wheel-creatures are alive and semi-autonomous, with their own own way to acquire nutrients. Perhaps they would, er... suckle. Eeugh.
Maybe in your living room. But step into a dense forest (which is what we are made for) and that statement will be far away from reality.
So embedding such a sensor in every rigid component, wiring a single data line to all of them (using the chassis as electrical ground) and feeding the data back to the model seems a trivial way to work around this problem without any kind of real pressure sensitivity. The model knows the inputs it gives to the actuators/servos, so it will quickly learn to predict the free mechanical behavior of the body, and use any deviation to derive data equivalent to pressure and force feedback.
Another possible source of data is the driving current of the motors/actuators which is proportional to the mechanical resistance the limb encounters. All sorts of garbage sources of data that were almost useless noise in the classical approach become valuable with a model large enough.
The problem is precisely the actuators. A lot of a human's muscles actually come in pairs - agonist and antagonist muscles [1], and it's hard to match the way human muscles work and their relatively tiny size in a non-biological actuator.
Just take your elbow and angle it to 90 degrees, then rapidly close it so your upper and lower arm are (almost) in parallel. An absolutely easy, trivial task to do for your pair of muscles controlling the tendons. But now, try to replicate even this small feat in a motor based actuator. You either use some worm gear to prevent the limb from going in the wrong direction but lose speed, or you use some sort of stepper motor that's very hard to control and takes up a lot of space.
[1] https://en.wikipedia.org/wiki/Anatomical_terms_of_muscle
Side-rant: As cool as some cyberpunk/sci-fi ideas are, I can't imagine a widespread elective mechanical limb replacement within the lifetime of anyone here. We dramatically under-estimate how amazing our normal limbs are. I mean, they're literally swarms of nanobots beyond human comprehension. To recycle an old comment against mechanical limbs:
________
[...] just remember that you're sacrificing raw force/speed for a system with a great deal of other trade-offs which would be difficult for modern science to replicate.
1. Supports a very large number of individual movements and articulations
2. Meets certain weight-restrictions (overall system must be near-buoyant in water)
3. Supports a wide variety of automatic self-repair techniques, many of which can occur without ceasing operation
4. Is entirely produced and usually maintained by unskilled (unconscious?) labor from common raw materials
5. Contains a comprehensive suite of sensors
6. Not too brittle, flexes to store and release mechanical energy from certain impacts
7. Selectively reinforces itself when strain is detected
8. Has areas for the storage of long-term energy reserves, which double as an impact cushion
9. Houses small fabricators to replenish some of its own operating fluids
10. Subsystems for thermal management (evaporative cooling, automatic micro-activation)
_______________
I predict the closest thing we might see instead will be just growing replacement biological limbs, followed by waldoes where you remotely control an arm without losing your own.
Take the elbow joint and the muscles it's connected to. It supports very fine precision, slow speed operations as well as high speed but at the same time the same operation at high speeds - say, lifting yourself up on a horizontal bar, assuming adequate strength you can either do a slow or a fast lift, and both at enough precision and torque to prevent your body mass from impacting to the bar which is another feat in itself.
Now try to replicate that with a classic mechanical mechanism, you'll always lose either precision, speed or torque.
But then he goes on to vision, where the form that goes into vision processing today is an array of pixels. That's not much preprocessing. That's pretty much what existed at the image sensor. Older approaches to vision processing had feature extractors, with various human-defined feature sets. That was a dead end. Today's neural nets find their own features to extract.
Touch sensing suffers from sensor problems. A few high-detail skin-like sensors have been built. Ruggedness and wear are a big problem.
Consider, though, a rigid tool such as an end wrench. Humans can feel out the position of a bolt with an end wrench, get the wrench around the bolt, and apply pressure to tighten or loosen a nut. Yet the total information available is position plus six degrees of freedom of force. If the business end of your tool is rigid, the amount of info you can get from it is quite limited. That doesn't mean you can't get a lot done. (I fooled around with this idea pre-LLM era, but didn't get very far.) That's at least a way to get warmed up on the problem.
Here's a video of a surgeon practicing by folding paper cranes with small surgical tools.[1] These are rigid tools, so the amount of touch information available is limited. That's a good problem to work on.
[1] https://www.youtube.com/watch?v=5q-HHoqzQi0
Not sure which lab (I think google?) it was, but there was a recent demo of a ML-model driven robot that folded paper in that style as one of the tasks.
But I think most of those can be replaced by existing robotics as well anyway. I mean take car manufacturing, over time more and more humans were replaced by robots, and nowadays the newest car factories are mostly automated (see lights-out manufacturing: https://en.wikipedia.org/wiki/Lights_out_(manufacturing)). Interestingly a Japanese robot factory has been lights-out since 2001, where they can run for 30 days on end without any lights, humans, heating or cooling.
And I could see it. With prevalence of screens kids already don't learn a lot of dexterity that previous generations have learned. Their grip strength is weak and capacity for fine 3d motions is probably underdeveloped as well.
Last week I've seen an intelligent and normally developing 7 year old kid asking mum to operate a small screwdriver to get to the battery compartment of a toy because that apparently was beyond his competence.
Now with recent developments in robotics, fully neural controllers and training in simulated environments there could be that modern babies will have very little tasks requiring dexterity left when they grow up.
This has almost nothing to do with nature (barring a development issue).
This has to do with nurture. Every time they went to do something with a tool a helicopter gunship of a parent showed up to tell them no. Now they have a learned helplessness when it comes to these things.
But that's not really any different then when I was a kid so very long ago. At 4 or 5 I was given a stack of old broken radios and took them to the garage for a rip and tear session. I got to look at all their pretty electronic guts that fascinated me. There were plenty of other parents of that time that would have been horrified to see their kids do something similar.
Take size, strength, precision, longevity, and speed. It's not hard to match or beat organic muscle fibers on one or two of these dimensions with an electrically driven system, but if it does, it's going to neglect other dimensions to such a degree as to put building a humanoid robot that achieves parity with a human completely out of reach.
You can slather as much AI as you want on top of inadequate hardware - it's not going to help.
> The center piece of my argument is that the brute force learning approaches that everyone rightfully touts as great achievements relied on case-specific very carefully engineered front-ends to extract the right data from the cacophony of raw signals that the real-world presents.
In nearly each of the preceding examples, isn't the argument really about the boundaries that define the learning machine? Just because data preparation / formatting / sampling / serialization is more cost-effective to do externally from the learning machine, doesn't mean that boundary is necessary. One could build all of this directly inside the boundary of the learning machine and feed it the raw, messy, real world signals.
Also, humans having plentiful learning aids doing "tokenization", as anyone who helped a child learn to count has experienced first hand.
I spent a few minutes excitedly trying to figure out how one of my favourite declarative programming languages was used to solve modern robotic sensing problems, only to realise it was probably just a misspelling ... :(
Next, insert a Standard screwdriver into a screw head, set the screw in place, and screw it in. In order to make it work, you have to push and torque it at the same time, and not let the blade slip out of the hole or damage the screw head.
If you think this is easy, try to teach a kid to do it. Watch them struggle to control the nut and the screwdriver.
Our hands are really, really good at both major motor control and very fine motor control.
Rodney Brook's achievements include Lucid, the Roomba and Baxter.
Letting AIs play games to learn, but by using physical controllers, etc.