HN Explorer

Ask HN: Tracking People Positions in a Room

4 points by atoav ↗ HN
I was asked by an artist friend whether or how it would be possible to build a technological solution that e.g. returns a list of people in a room and where they roughly are.

The idea is to figure out the distances of said people to certain projected inages and change them accordingly when these distances change.

Now I have some experience with laser baded Rime-of-Flight sensors, but those have too narrow of a field of view and wouldn't be able to cover the whole area (it would be a U shapes space). I had a look at OpenCV and YOLOv8 and bytetracker. That worked okay, but produced many false positives and jitters on my recorder-from-top test footage. Two people standing next to each other were classified as a single elephant, bear, horse all within the span of a single second. I had a look at training my own model, but I won't be able to film the footage at the space at which this should take place.

Is computer vision even the right option? If it is, does anybody know good models for people from top?

9 comments

[ 2.5 ms ] story [ 38.9 ms ] thread
pavel_lishin
Can you assume that every person is carrying a cellphone, or other similar device? You could use wifi or Bluetooth somehow to track those devices...

Alternatively, could you place sensors on the floor under carpets to measure the weight of human bodies?

Or use opencv or something like it, but use infrared cameras from above. I'm not sure how hats would affect that...

atoav
I'd like to avoid depending on the devices on people's bodies for philosophical reasons, also testing it could be even harder.

But I read some researchers had success to do pose estimation from a wifi router, I have just no idea how to replicate that myself.

A sensing floor could work for this, but given the scale of the area this could be too expensive.

I actually wanted to use cameras from above, infrared could be a good idea, I will check into it — thanks for your suggestions!

atoav
I just remembered I still have velostat lying around, so maybe I try to make a scale model of a sensing floor first to get a feel for how feasible that would be.
uberman
A team I worked work along side seemed to have very good luck tracking people and even gestures using several kinects. With luck you might be able to do the same
atoav
I will check kinects, on a first glance both size and positioning might be an issue.
atoav
Ok checked more, the space is huge, so unless we get upwards of 10 of those this is probably not an option, but I will keep it in mind for future projects.
PaulHoule
Look up "pose estimation" as a research topic

https://arxiv.org/search/?query=%22pose+estimation%22&search...

See also

https://github.com/pliablepixels/zmeventnotification

for a rather mature system that adds person and object detection to a security camera system.

atoav
Good tip, I read over this topic not thinking it would be of any help for my problem, but now that you mention it... maybe pose estimation works better than tracking. I havw to test this.
atoav
Okay I went and checked out pose estimation, which delivers a superior result (additionally I might be able to extract which way people are facing which would be amazing for art purposes).

Thanks also for the library you posted, I think internally it also uses YOLOv8.

Now the only remaining questions are:

1. How to do the math to translate the perspective camera image into positions on a floor plan, but I already got an idea where to look as this is known math.

2. How to stabilize results that pop in and out of existence — I guess I will just write my own overlay that has lingering states, at least for entities at the center of the image.

3. How to get all of that running with sufficient performance on my old Laptop with it's RTX2080m, because I won't put my tower into an exhibition space for a month. Either way it doesn't need to be fluid 60fps, maybe even 2 fps would be okay.

4. Into which OSC messages to translate the results, as I am collaborating with a VJ.

All of those questions are sort of managable I think.