How does your system handle the massive variance in sensor quality (accelerometer, gyro, WiFi radio) between a high-end iPhone and a budget Android device? Does the 1m accuracy hold up across the board, or does it degrade gracefully? Getting this right seems critical for scaling to a 'billion people'
Peripheral chips aren't differentiated by user values like the end products that use them. You don't get more preciser sensors in high end phones. Everyone gets the same thing. You pay more only for more materials.
Sensors that are actually a lot better than standard offerings would also be subject to and/ofs of ITAR or EAR or MTCR or local equivalents thereof, so progress in IMU appears to have been stagnating a bit due to that issue. Sony Semiconductor Solutions had a Arduino IDE compatible clustered IMU board that they say you can see rotation of Earth in data, they ended up selling it with scary warnings and without any of the cool stuffs.
There's a fair bit of quality difference between different chips and better chips have gotten cheaper. More importantly, default filtering quality has improved with more powerful uCs on the IMU package, which is what most cheap phone vendors are probably using.
The ITAR stuff is way more fun though. It's great to read between the lines for the intended customer in the datasheet.
Very, very impressive, if (intentionally?) a little vague on how you got to 1m accuracy given you WiFi only gives you 3m.
I guess you must use the constraints on the directions a person can walk imposed by the shelves and other structures to give you orientation of the accelerometers. Which in turn means the person doing the ground truth mapping must walk down every aisle, and into every gap. That's not so difficult if your staff are doing it, but I bet you have trouble training the store staff gathering that data to do it well.
Thanks for the feedback. I wanted to keep it balanced to be accessible but also insightful.
To answer your point: we have the digital map, can use that to understand obstacles etc in the space. In some of those larger stores you see in the visual, we typically survey the entire store within 2-3 hours, it's low-effort work, not a blocker.
Many times these stores have floor cleaning machines - either robotic or driven by a human. An employee could zip-tie their sensor to it, let it do its cleaning trip around the store, and return to collect the data later.
This would allow an employee to do several stores in a town in a single day. And potentially less chance of a workers-compensation claim being filed if they fall down while walking around looking at their device.
In my view, you need to 'streetview' the world's shopping malls and make this a free app, the Hyper Browser. Add product search & reviews and drive traffic away from non-customer stores.
> They wanted to bring indoor maps and navigation to their retail stores... It turns out that this doesn’t just apply to retail. Every office, university campus, events venue, hotel, airport, warehouse, factory — basically everywhere indoors have some need to navigate people around, provide relevant information, and improve efficiency.
You'd think they would add this information to openstreetmap then or at least put a map on their website (and put it in the public domain so others like OSM can add it to their maps). Or put it in the store so people can take a picture. I go into target and there are posters saying to install an app for maps. Put the map on the poster!
> and they could pop up relevant promotions along the way
Oh, right, they don't want to provide information. They want to track people and spam them.
Honestly, I've wanted a system for a while where shops can provide a map, and I can search for an item and it will show me where in the shop that item is. I don't think I've ever been in a shop big enough where I'd need satnav to tell me how to get there though, is that an American thing?
Do you think the basic physics and sensor tolerances would let you go to 10^-2 meters if the environment (e.g. wifi station placements, location of RF-interfering elements) was designed by you?
Interestingly, a version of this technology was built by a government contractor some years ago, because they too wanted to track people indoors. I don't think it used a smartphone though...
I think an acquisition would be unfortunate. This could become really huge / useful to the world without being locked up as a private company's IP. Personally I would license it rather than sell it, as well as offering offline apps and limited SaaS. You don't need an established enterprise to sell it commercially; you just need a sexy product and some industry vets with contacts. If you do end up going that way, and need someone in IT Ops / DevOps / SysEng stuff to work on the "going enterprise with a billion users", give me a shout.
This is very, very impressive. I have three questions. How do you know where you are within the store? Do you manually upload the floorplan? How does it know where all the items in the store are?
Really interesting. Feels like it would be a natural addition to a company like Brain (https://www.braincorp.com/) who is already using robots to regularly perform those kinds of survey missions and have an overlapping customer base.
This is a very cool project / startup! I'm curious, how do you get the ground truth data? Is that just you marking down where you are as you walk through the store?
Thank you! I tried to keep it interesting but not get lost in rabbit holes.
We have around 300 hours of ground-truth data now, in 1-second intervals, which we use for algorithm training and refinement. The same as performing a survey, our team marks their location on the map, then walks to the next location, and through post-processing we can correct any errors and interpolate the locations in between. You can see this process in the diagram with the large black dots, where the user marked their location.
Love reading about these types of R&D efforts, especially when they are successful. I started in robotics in 01 and was present at some of the first commercial vSLAM efforts and materially contributed to a hardware solution utilizing IR beacons on the ceiling. We also looked at RF radio mapping at the time, but the computational power wasn't there. Great to see how far it has come and LOVE that it doesn't require infrastructure!
I am curious how deeply you have had to study the impact of how busy the store is with your signal error? Considering that humans are bags of water which is quite detrimental to RF signals, my guess is that your error increases along with the density of people.
The few examples they show do look pretty good for a wifi-based method, although who knows how cherry-picked they are. I wonder how much the "SLAM" part is contributing and how sensitive that is to the sensor quality on the phone. I would've assumed that they'd be using vision, which seems to be the method of choice for other companies like niantic. The ground-truth data part for vision would certainly be more onerous, though.
He explains it fairly well if you understand how you'd go from wifi accuracy to SLAM. THE WIFI was providing 3m accuracy and the SLAM down to 1M. how much it provides is those two numbers. I'm sure the algorithms are complex but he points out that SLAM is corrected by the actual maps made by the self service app. So it's fairly easy to understand: the map provides a probability space, the wifi puts you within 3m and the SLAM is use to fill in the blanks with help from the probability space.
How fast do the WiFi signal strength maps get out of date?
Just someone changing the angle of an antenna or shifting a pile of stock near the router has to have a pretty big impact on signal strength.
And obviously a WiFi system upgrade where all the Mac addresses change must be a fairly big change and effectively gives a full service outage for all the users till you remap.
We've had the solution live in stores for years without any issue. Our algorithms can clean up and filter out unreliable data, based on all of the other data we have, from other APs, from previous data etc..
If they re-fit the WiFi, which they might spend a month doing once every 5-10 years, it would need a re-survey, which could take 1-2 hours. But thankfully no new infrastructure.
Awesome breakdown of the challenges of indoor navigation! One thing I was curious about... given that many modern phones now have UWB radios built in, was UWB ever considered as part of the solution stack? From what I understand UWB can get to sub meter accuracy, and I know it's used in several sports applications where precise tracking is really critical.
Is the constraint more about infrastructure (installing anchors, device compatibility, power) or something else that made you lean towards WiFi + SLAM fusion instead?
Huge respect to anybody who ships a product, even software. I have yet to use any indoor navigation or even have that offered, but it would be nice. Question is, is SLAM the correct term? I thought mapping meant like the device doesn't have any prior knowledge of the environment and is, well, mapping it. This more feels like sensor fusion with a kalman filter.
40 comments
[ 3.3 ms ] story [ 78.1 ms ] threadSensors that are actually a lot better than standard offerings would also be subject to and/ofs of ITAR or EAR or MTCR or local equivalents thereof, so progress in IMU appears to have been stagnating a bit due to that issue. Sony Semiconductor Solutions had a Arduino IDE compatible clustered IMU board that they say you can see rotation of Earth in data, they ended up selling it with scary warnings and without any of the cool stuffs.
The ITAR stuff is way more fun though. It's great to read between the lines for the intended customer in the datasheet.
https://en.wikipedia.org/wiki/Kalman_filter
It can combine several inaccurate sources and output a result that is more accurate than any one of them.
I was at an Amazon Fresh grocery store, and saw squares in the ceiling that look like QR codes. I guess that's how they are mapping the store.
I guess you must use the constraints on the directions a person can walk imposed by the shelves and other structures to give you orientation of the accelerometers. Which in turn means the person doing the ground truth mapping must walk down every aisle, and into every gap. That's not so difficult if your staff are doing it, but I bet you have trouble training the store staff gathering that data to do it well.
Best of British.
To answer your point: we have the digital map, can use that to understand obstacles etc in the space. In some of those larger stores you see in the visual, we typically survey the entire store within 2-3 hours, it's low-effort work, not a blocker.
This would allow an employee to do several stores in a town in a single day. And potentially less chance of a workers-compensation claim being filed if they fall down while walking around looking at their device.
It’s the Built for simple campus navigation video.
I’d recommend converting it to SDR.
You'd think they would add this information to openstreetmap then or at least put a map on their website (and put it in the public domain so others like OSM can add it to their maps). Or put it in the store so people can take a picture. I go into target and there are posters saying to install an app for maps. Put the map on the poster!
> and they could pop up relevant promotions along the way
Oh, right, they don't want to provide information. They want to track people and spam them.
I think an acquisition would be unfortunate. This could become really huge / useful to the world without being locked up as a private company's IP. Personally I would license it rather than sell it, as well as offering offline apps and limited SaaS. You don't need an established enterprise to sell it commercially; you just need a sexy product and some industry vets with contacts. If you do end up going that way, and need someone in IT Ops / DevOps / SysEng stuff to work on the "going enterprise with a billion users", give me a shout.
We have around 300 hours of ground-truth data now, in 1-second intervals, which we use for algorithm training and refinement. The same as performing a survey, our team marks their location on the map, then walks to the next location, and through post-processing we can correct any errors and interpolate the locations in between. You can see this process in the diagram with the large black dots, where the user marked their location.
I am curious how deeply you have had to study the impact of how busy the store is with your signal error? Considering that humans are bags of water which is quite detrimental to RF signals, my guess is that your error increases along with the density of people.
Just someone changing the angle of an antenna or shifting a pile of stock near the router has to have a pretty big impact on signal strength.
And obviously a WiFi system upgrade where all the Mac addresses change must be a fairly big change and effectively gives a full service outage for all the users till you remap.
If they re-fit the WiFi, which they might spend a month doing once every 5-10 years, it would need a re-survey, which could take 1-2 hours. But thankfully no new infrastructure.
It tells me how many are in stock, aisle number, and bay number. No need for an app or walking advertisements.
Is the constraint more about infrastructure (installing anchors, device compatibility, power) or something else that made you lean towards WiFi + SLAM fusion instead?