You can always count on HNers to point out that a neutral network designed to run realtime on devices that have a 1 watt-hour battery definitely "isn't going to be used primarily in smartwatches". The US gov't can't possibly afford to do offline pose estimation on a real GPU, am I right?
I could see this being used as something like an adjunct to cop body cameras, and likewise military applications.
Maybe also a requirement for CDL drivers. From there it extends to car insurance companies offering consumers a "discount" if you give em your data, similar to that ODB thing they currently do.
You mean the guy whose grandfather helped build the internet and who supplies the majority of US intel agencies with cloud computing, storage and access might be testing government technology in the public sector to not raise suspicion?
The camera shots that are shown in that paper are really restricted in what they can see. How can the model possibly estimate the position of an arm that is not visible?
Probably by learning how humans have to balance their extremities to remain standing upright. The camera angle with respect to the enviroment is likely a factor too, since errors got larger when outdoors.
Errors are also not exactly small, ~6cm average, although it's more like 1-2 or 5-12, depending on body part. I think this would very likely be noticeable in VR applications, but it is still very impressive accuracy overall.
The question also nicely highlights the disadvantages of trained algorithms: No one knows for sure, and it certainly isn't obvious by looking at the network weights...
On the other hand, when looking at the pictures, I felt like the authors: There should be enough information in there to get at least a good estimate. And it is extremely useful that one can nowadays "just" train a model to confirm such theories.
Has the result been reproduced? I could quite easily move my forearm to several distinct positions not visible from that view that would have no impact on the rest of my posture. I feel like there's a lot of ML out there making big claims and not distributing the models for verification.
This is an interesting approach, but full body tracking with a few sensors is already much better than this at the $150 price point.[1] Because it will suffer badly from occlusion, it will probably do well in expected situations and terribly in unexpected and occluded situations.
The press release contains a few errors and imprecisions. The abstract from the scientific article is more informative:
In this paper, we present BodyTrak, an intelligent sensing technology that can estimate full body poses on a wristband. It only requires one miniature RGB camera to capture the body silhouettes, which are learned by a customized deep learning model to estimate the 3D positions of 14 joints on arms, legs, torso, and head. We conducted a user study with 9 participants in which each participant performed 12 daily activities such as walking, sitting, or exercising, in varying scenarios (wearing different clothes, outdoors/indoors) with a different number of camera settings on the wrist. The results show that our system can infer the full body pose (3D positions of 14 joints) with an average error of 6.9 cm using only one miniature RGB camera (11.5mm x 9.5mm) on the wrist pointing towards the body [...].
I think we can expect full body tracking to be added to the Quest Pro and Quest 2 as a software update if you buy the new controllers since these already have cameras on them. Meta already published a paper about doing pose prediction a while back.
17 comments
[ 5.3 ms ] story [ 40.9 ms ] threadName checks out. This kind of tech isn't going to be used primarily in smartwatches it's going to be integrated into state-based surveillance.
Maybe also a requirement for CDL drivers. From there it extends to car insurance companies offering consumers a "discount" if you give em your data, similar to that ODB thing they currently do.
>Soldier tracking >Prisoner tracking >Student tracking >Smartwatches
The camera shots that are shown in that paper are really restricted in what they can see. How can the model possibly estimate the position of an arm that is not visible?
Errors are also not exactly small, ~6cm average, although it's more like 1-2 or 5-12, depending on body part. I think this would very likely be noticeable in VR applications, but it is still very impressive accuracy overall.
The question also nicely highlights the disadvantages of trained algorithms: No one knows for sure, and it certainly isn't obvious by looking at the network weights...
On the other hand, when looking at the pictures, I felt like the authors: There should be enough information in there to get at least a good estimate. And it is extremely useful that one can nowadays "just" train a model to confirm such theories.
This is an interesting approach, but full body tracking with a few sensors is already much better than this at the $150 price point.[1] Because it will suffer badly from occlusion, it will probably do well in expected situations and terribly in unexpected and occluded situations.
[1] https://www.youtube.com/watch?v=ImEKHrUp4QM
In this paper, we present BodyTrak, an intelligent sensing technology that can estimate full body poses on a wristband. It only requires one miniature RGB camera to capture the body silhouettes, which are learned by a customized deep learning model to estimate the 3D positions of 14 joints on arms, legs, torso, and head. We conducted a user study with 9 participants in which each participant performed 12 daily activities such as walking, sitting, or exercising, in varying scenarios (wearing different clothes, outdoors/indoors) with a different number of camera settings on the wrist. The results show that our system can infer the full body pose (3D positions of 14 joints) with an average error of 6.9 cm using only one miniature RGB camera (11.5mm x 9.5mm) on the wrist pointing towards the body [...].