Would be great if you expanded on what should be touted as such, then?
My take is that you should read "holy grail" as a directional claim. That is, not something that can ever be obtained, and as we get closer to the current holder of that vision, expect/demand that it will change.
I spent a number of years working on software at Amazon in the Picking and storage space. This truly is the Holy Grail in the sense that I'm not actually sure what they want is possible/real.
There is a trade-off between storage density and ease of picking. This is true for humans as well as robots. An example: Open a drawer in your kitchen. If it's full as possible in an unorganized fashion, you'll often struggle to even locate the item you're looking for. And once you find it, you need to carefully move other items around to get it out.
Now, empty that drawer and put back 1/3 of the items, carefully organized. Now you can find and grab anything quickly! Even a robot could! ... But you need 3 times as many drawers in your kitchen.
Your messy kitchen drawer is a pretty close parallel to a typical Amazon storage bin.
Set aside the question of robot speed (they're slow), the real question is how much less storage density is needed for the robot to function. Giving up storage density means a smaller selection in each warehouse, which increases shipping costs- the item you want isn't as near to you anymore- or it demands a lot more warehouses, which are expensive to build and operate.
Shipping costs dominate their overall cost to get you what you ordered. Fully automating picking at the cost of reduced storage density is a net loss.
Surely there's capacity to double or even triple the number of storage bins in existing warehouses by expanding vertically. Humans are limited to 6 ft but robots have no vertical limit. In the future, warehouses could be as tall as skyscrapers with full vertical pickability.
For sure. You can also cut heat and AC a lot in a robot warehouse. Meaning even if it was 50% more square feet for the same stuff, it should be a lot cheaper to operate,
Expanding vertically? Nope. The storage part of the warehouse is typically 4 levels with ten foot ceilings. They have always been under these space constraints and have absolutely already optimized for space vertically.
You could build a taller warehouse, but there's a standard height and there are, I believe, often building code things to prevent taller towers like that. And again, if it was reasonable to do it already, they'd have gone taller already.
For the limited subset of a parking garage, which generally has much lower ceilings than is normal for other uses, the cost of one underground parking space is around $30,000.
I work on projects T&L space and this does not fit my experience, in fact large warehouses and DCs are getting taller and most currently are much taller than 10’. Order pickers routinely pick or store items from the 5th and higher levels. Maybe you have in mind a local warehouse?
I don't know for sure but I think building, OSHA, and various other regulations would probably make problems for getting a highly unique building like this built.
Plus the reason most warehouses don't build up is building up is expensive. Most warehouses are in cheap land areas so building out is more cost effective.
I've worked in an Amazon Warehouse doing picking on two occasions --- towards the end of the first (1 year, part-time) stint, I walked up to a bin full of non-descript cardboard boxes, grabbed one, spun it in my hands to do a six-sided box check, then scanned it to verify it was the correct box --- a trainer w/ a group of trainees then prevailed on me to explain how I'd done it. It took a couple of minutes to explain why based on the product description the other boxes in the bin could not have been the item in question, and the minor clues which led me to my conclusion.
So you learned a lot of subtle features that helped you pick items more efficiently?
But robots can have senses that humans can't. Why don't they just embed an RFID chip in every box, give the robot sensors to triangulate the position of the RFID code matching the desired product, and grab that?
Yeah, I used to joke that I wasn't being paid enough for the brain cells I was using to memorize such minutiae.
RFID would certainly be one workable approach.
I'm still surprised that Amazon hasn't just chosen to pre-package everything in their warehouses, then do the stacking in such a way that a picker (or robot) is told which row/column a given product of a box is in.
But, I guess that gets back to OP's observation on the expense of warehouse space/real estate.
> Why don't they just embed an RFID chip in every box, give the robot sensors to triangulate the position of the RFID code matching the desired product, and grab that?
Whenever you start a sentence with "why don't they just", remember that there are literally hundreds of people whose entire job it is to solve these problems, and there's probably very good reasons.
In this case, the answer is "noise".
A fellow I knew worked on a project at Amazon fulfillment to use RFID enabled shipping labels so that ideally they could load the truck and know which packages were on it.
It was a total failure.
If the sensors are high enough to read every package, then you'll read half the packages 20 feet away as well. If it's low enough to only detect the ones you want, it won't detect all of them.
"Triangulate the position of the RFID code" is just not really a thing. Maybe in the future, but today RFID is just not that reliable.
Also the labor cost to add an RFID tag to every item is prohibitively high.
I wonder how this will change when warehouses are designed around robots. Right now robots are working in warehouses that were built to be optimally efficient for human workers, but if they're built from the ground up for robots, you're probably going to be able to find space efficiencies that aren't there today.
And then of course there's the Optimus-style humanoid robot that one can imagine will exist one day - if you're replacing humans with them (and they have comparable capabilities, which it's tough to imagine they won't eventually), then you'd get at least the full storage capacity that you have in today's human-optimized warehouse.
Maybe it's not possible with picking, but having worked at Amazon IXD, I think it's certainly possible with most items (>70%) in the receiving, sorting, and preparation processes at the cross docks.
I'm not sure about Amazon Logistics in the US, but at least in Europe I got the feeling that management just doesn't care about automating processes because humans are still cheaper in most cases.
Amazon Logistics uses some brilliant tactics in Germany to basically get workers for 50% off while maximizing productivity; all subsidized by taxpayers:
In West Germany we have countless unemployed people after the migration wave of 2015/2016, and those who were able to stay and gotten asylum are often "unskilled" workers who have trouble finding a job. These people go to the employment office (Arbeitsamt), and the Arbeitsamt's "Jobcenter" then sends them to Amazon. If people don't comply, the Arbeitsamt cuts their unemployment benefits, which would lead to even worse living conditions for those who are already living in poverty. So, there is a big incentive to take these very bad jobs at Amazon.
The worst thing about this is that in Germany the government pays employers an "integration subsidy" (Eingliederungszuschuss). This is a percentage of wages paid by the state.
It's up to 50% for 12 months for everyone who joined Amazon via the Jobcenter; and for people who were unemployed for >2 years, it's 75%(!) the first year and 50% the second year.
Also, after two years, an employee is entitled to a permanent contract. Everybody wants to have this unlimited contract, but only a small percentage gets it. Therefore, workers with fixed-term contracts try as hard as they can to get this offer at the end of their term.
However, Amazon knows exactly that after the workers get a permanent contract, the productivity decreases. There are internal statistics for this (this is also the reason why only such a small percentage gets this offer in the first place).
But the smart thing now is that each warehouse is a separate legal entity. This means that in reality, workers are terminated after 2 years, but are offered a new contract by the warehouse that is right next to it. There they then have to work extremely hard again for 2 years, hoping for a permanent contract. HAt this state most people give up and leave Amazon.
Anyway, back to automation:
In Europe, Amazon has always just bought extremely expensive instead of investing in research themselves. We then bought overpriced machines like OPEX Sure Sort. The sorting system itself works quite well, but it's just not integrated into the process at all. This means that at the end of the day, people have to fill the machine manually, and the bottom line is that only two workers per machine (which costs a quarter million euros, so you need 4 years for break-even) are eliminated.
The only in-house product we had from Amazon Robotics / Ex-Kiva was a robot for palletizing EPAL pallets. This thing is actually great and really helped to maximize throughput.
I don't think Amazon will ever achieve an all lights off IXD with the current governance.
> I'm not sure about Amazon Logistics in the US, but at least in Europe I got the feeling that management just doesn't care about automating processes because humans are still cheaper in most cases.
Historically this has been the problem with automation. Automation tends to occur when the labor supply tightens and/or the credit supply increases (i.e. the cost of borrowing to purchase new capital drops substantially). Interest rates being what they are right now I doubt anyone wants to think to hard about automating, especially when the labor market is flooded with people who've lost their jobs.
> I don't think Amazon will ever achieve an all lights off IXD with the current governance.
I mean, there doesn't really seem to be a strong push from up top for it, or at least no more than there is to fix Mentor for drivers.
Well, a system can have "perfect" knowledge of where in that drawer things are, a human can't, but nevertheless you need a minimum amount of space to move things around.
Is Amazon retail still basically making 1-2% profit and AWS makes all the rest? I've seen these Amazon stories for a decade, and I guess there probably is a large amount of reinvestment, but isn't Walmart more profitable?
The perfect warehouse robot based on organic life would be a spider?
Interestingly they said the computer vision only matches 65% of Amazons product inventory (at the moment).
I wonder if they can still deploy it on certain lines where the order’s products 100% fall into the 65% bucket and route the rest to human lines. Or let the robots sort first before passing only unfinished ones to humans?
I’m also curious which products it can’t recognize. Probably a lot of small loose items.
Oh, that's how Sparrow works. The end effector is 7 vacuum cups arranged in a filled hexagon, each individually retractable. So they can use one for a small object, three or five for a linear object, and all seven for a big object. That's novel. It's also a simple mechanism that can probably run for a long time without problems. It's limited to things you can grab with vacuum, of course.
Amazon used to have a "robot picking challenge". Here's the MIT team's video from 2017.[1] That's able to deal with a more confused input bin. They had both a vacuum picker and a gripper. The gripper looks too fragile for a production environment.
Unstructured bin picking remains a very hard problem. I first saw a demo in the early 1980s, where a robot at a Detroit automation show was picking Magic Markers from a bin to give to visitors. Improvement since then is far less than you'd expect. Here's a reasonably good demo from 2019.[2]
As I point out occasionally, we're still having trouble getting to squirrel-level AI.
I'm curious why it's designed to hold the object in front of a rather distant camera for several seconds in order to identify it, instead of mounting a camera on the vacuum arm itself so that it can observe the object as it approaches. The face it grabs tends to be the biggest and contain the product label so you'd also be able to get a better look at the object.
Note that this is just Amazon demonstrating a prototype in their Innovation Center. I take it to be a mostly PR move showing “thought leadership.” Replicating the functionality of the human hand is notoriously hard, although it has been tried many times before by many companies, AFAIK there has been no great success.
The push for automation currently is not about replacing workers and cutting costs (although, of course, those are desirable): there’s tremendous churn in all warehouse roles so much so that most companies are hard pressed to find workers. Many new hires quit during the two week training programs.
Couldn’t care less. When they have a robot that will clean my house, do my dishes and laundry, mow my lawn and remove snow, call me. I would spend thousands for a robot like that.
It starts with a spouse to create children who do those tasks. The challenge is to space them appropriately to maximize your return on investment. It starts at thousands a year and eventually consumes at least half of your net worth
39 comments
[ 752 ms ] story [ 1004 ms ] threadOr is it just taking existing tech and applying it in a real warehouse.
How Amazon is using this? Nah.
My take is that you should read "holy grail" as a directional claim. That is, not something that can ever be obtained, and as we get closer to the current holder of that vision, expect/demand that it will change.
There is a trade-off between storage density and ease of picking. This is true for humans as well as robots. An example: Open a drawer in your kitchen. If it's full as possible in an unorganized fashion, you'll often struggle to even locate the item you're looking for. And once you find it, you need to carefully move other items around to get it out.
Now, empty that drawer and put back 1/3 of the items, carefully organized. Now you can find and grab anything quickly! Even a robot could! ... But you need 3 times as many drawers in your kitchen.
Your messy kitchen drawer is a pretty close parallel to a typical Amazon storage bin.
Set aside the question of robot speed (they're slow), the real question is how much less storage density is needed for the robot to function. Giving up storage density means a smaller selection in each warehouse, which increases shipping costs- the item you want isn't as near to you anymore- or it demands a lot more warehouses, which are expensive to build and operate.
Shipping costs dominate their overall cost to get you what you ordered. Fully automating picking at the cost of reduced storage density is a net loss.
Surely there's capacity to double or even triple the number of storage bins in existing warehouses by expanding vertically. Humans are limited to 6 ft but robots have no vertical limit. In the future, warehouses could be as tall as skyscrapers with full vertical pickability.
You could build a taller warehouse, but there's a standard height and there are, I believe, often building code things to prevent taller towers like that. And again, if it was reasonable to do it already, they'd have gone taller already.
For the limited subset of a parking garage, which generally has much lower ceilings than is normal for other uses, the cost of one underground parking space is around $30,000.
Plus the reason most warehouses don't build up is building up is expensive. Most warehouses are in cheap land areas so building out is more cost effective.
But robots can have senses that humans can't. Why don't they just embed an RFID chip in every box, give the robot sensors to triangulate the position of the RFID code matching the desired product, and grab that?
RFID would certainly be one workable approach.
I'm still surprised that Amazon hasn't just chosen to pre-package everything in their warehouses, then do the stacking in such a way that a picker (or robot) is told which row/column a given product of a box is in.
But, I guess that gets back to OP's observation on the expense of warehouse space/real estate.
When Amazon puts two items into the same box, they massively reduce shipping costs. And shipping costs are the biggest cost they have.
Whenever you start a sentence with "why don't they just", remember that there are literally hundreds of people whose entire job it is to solve these problems, and there's probably very good reasons.
In this case, the answer is "noise".
A fellow I knew worked on a project at Amazon fulfillment to use RFID enabled shipping labels so that ideally they could load the truck and know which packages were on it.
It was a total failure.
If the sensors are high enough to read every package, then you'll read half the packages 20 feet away as well. If it's low enough to only detect the ones you want, it won't detect all of them.
"Triangulate the position of the RFID code" is just not really a thing. Maybe in the future, but today RFID is just not that reliable.
Also the labor cost to add an RFID tag to every item is prohibitively high.
And then of course there's the Optimus-style humanoid robot that one can imagine will exist one day - if you're replacing humans with them (and they have comparable capabilities, which it's tough to imagine they won't eventually), then you'd get at least the full storage capacity that you have in today's human-optimized warehouse.
I'm not sure about Amazon Logistics in the US, but at least in Europe I got the feeling that management just doesn't care about automating processes because humans are still cheaper in most cases.
Amazon Logistics uses some brilliant tactics in Germany to basically get workers for 50% off while maximizing productivity; all subsidized by taxpayers:
In West Germany we have countless unemployed people after the migration wave of 2015/2016, and those who were able to stay and gotten asylum are often "unskilled" workers who have trouble finding a job. These people go to the employment office (Arbeitsamt), and the Arbeitsamt's "Jobcenter" then sends them to Amazon. If people don't comply, the Arbeitsamt cuts their unemployment benefits, which would lead to even worse living conditions for those who are already living in poverty. So, there is a big incentive to take these very bad jobs at Amazon.
The worst thing about this is that in Germany the government pays employers an "integration subsidy" (Eingliederungszuschuss). This is a percentage of wages paid by the state. It's up to 50% for 12 months for everyone who joined Amazon via the Jobcenter; and for people who were unemployed for >2 years, it's 75%(!) the first year and 50% the second year.
Also, after two years, an employee is entitled to a permanent contract. Everybody wants to have this unlimited contract, but only a small percentage gets it. Therefore, workers with fixed-term contracts try as hard as they can to get this offer at the end of their term.
However, Amazon knows exactly that after the workers get a permanent contract, the productivity decreases. There are internal statistics for this (this is also the reason why only such a small percentage gets this offer in the first place).
But the smart thing now is that each warehouse is a separate legal entity. This means that in reality, workers are terminated after 2 years, but are offered a new contract by the warehouse that is right next to it. There they then have to work extremely hard again for 2 years, hoping for a permanent contract. HAt this state most people give up and leave Amazon.
Anyway, back to automation:
In Europe, Amazon has always just bought extremely expensive instead of investing in research themselves. We then bought overpriced machines like OPEX Sure Sort. The sorting system itself works quite well, but it's just not integrated into the process at all. This means that at the end of the day, people have to fill the machine manually, and the bottom line is that only two workers per machine (which costs a quarter million euros, so you need 4 years for break-even) are eliminated.
The only in-house product we had from Amazon Robotics / Ex-Kiva was a robot for palletizing EPAL pallets. This thing is actually great and really helped to maximize throughput.
I don't think Amazon will ever achieve an all lights off IXD with the current governance.
Historically this has been the problem with automation. Automation tends to occur when the labor supply tightens and/or the credit supply increases (i.e. the cost of borrowing to purchase new capital drops substantially). Interest rates being what they are right now I doubt anyone wants to think to hard about automating, especially when the labor market is flooded with people who've lost their jobs.
> I don't think Amazon will ever achieve an all lights off IXD with the current governance.
I mean, there doesn't really seem to be a strong push from up top for it, or at least no more than there is to fix Mentor for drivers.
Is Amazon retail still basically making 1-2% profit and AWS makes all the rest? I've seen these Amazon stories for a decade, and I guess there probably is a large amount of reinvestment, but isn't Walmart more profitable?
The perfect warehouse robot based on organic life would be a spider?
https://youtu.be/lqxFvYgo5S8
Interestingly they said the computer vision only matches 65% of Amazons product inventory (at the moment).
I wonder if they can still deploy it on certain lines where the order’s products 100% fall into the 65% bucket and route the rest to human lines. Or let the robots sort first before passing only unfinished ones to humans?
I’m also curious which products it can’t recognize. Probably a lot of small loose items.
Amazon used to have a "robot picking challenge". Here's the MIT team's video from 2017.[1] That's able to deal with a more confused input bin. They had both a vacuum picker and a gripper. The gripper looks too fragile for a production environment.
Unstructured bin picking remains a very hard problem. I first saw a demo in the early 1980s, where a robot at a Detroit automation show was picking Magic Markers from a bin to give to visitors. Improvement since then is far less than you'd expect. Here's a reasonably good demo from 2019.[2]
As I point out occasionally, we're still having trouble getting to squirrel-level AI.
[1] https://www.youtube.com/watch?v=6fG7zwGfIkI
[2] https://www.youtube.com/watch?v=Pxd5-ZD9XcQ
The push for automation currently is not about replacing workers and cutting costs (although, of course, those are desirable): there’s tremendous churn in all warehouse roles so much so that most companies are hard pressed to find workers. Many new hires quit during the two week training programs.
[1] https://www.pcmag.com/picks/the-best-robot-lawn-mowers
[2] https://www.geeky-gadgets.com/robot-snow-blower-24-10-2019/