This seems like an extremely advanced version of those sand tables at science centers that kids play with, where digging a trench in sand with your hand modifies the projection to affect virtual water flow.
The youtube video[0] from that page is especially interesting.
Hope it doesn't easily scale out to larger spaces and crowds, or the current tech industry would soon have public spaces filled with ads projected on peoples belongings.
Disney has a (poor) version of this idea. They project an animation onto wedding cakes at their resorts. They're pre-rendered though, rather than dynamic like this. It'd be fascinating to see what uses they could come up with for this stuff.
Disney uses something similar in a lot of their stage shows at their parks. They use a water fountain to create a screen and project onto the water sheet. It works surprisingly well, but I would assume that it is pre-rendered.
The water screens are actually a pretty old technique that you can do with any traditional light projector (flim or digital). The hard part is making the water sheet.
Disney does a lot of projection mapping in the parks right now, but it's all (AFAIK) the traditional, pre-rendered type, with animations projected onto large static surfaces like the Magic Kingdom's castle, or onto animatronics with pre-planned motion paths.
I'm having a little trouble visualizing how this non-rigid projection mapping could get applied in a practical way in the parks today, mostly because it seems like it has a fairly small "active" area (determined by your projector and sensor resolution, essentially). I could imagine this being used in a parade or stage show, for example, but this system seems like it would be pretty restrictive as to where the performers could move and remain in the projection space.
Those projections onto the castle are quite stunning. The technology for producing an enormous, apparently seamless image on that uneven canvas is amazing (even if it's static). And a lot of creativity went into building an animation that used that very specific venue so well.
I could imagine doing something for the house of terror like projecting ghosts or something like that, in the curtains to a doorway or on sofas/couches after you stand up (and have your shadow follow you while doing creepy things for instance)
Of course, I'm no expert but throw a few million dollars into this and you can probably come up with some neat stuff even if the projection space is small-ish for now.
What if the projection space were mobile? One of the applications is to put projectors on moving objects, and project onto static (or moving) surfaces, assuming the system gets small & portable enough. How about one or more projectors mounted on each car of the Haunted Manison ride, or projectors mounted on flying drones aimed at the Spaceship Earth (the geodesic Epcot globe) during the night light show? Combine multiple projectors and a position tracking system, maybe even viewer head/eye tracking too, and I think there might be some amazing possibilities...
I think you could even go and apply the same algorithms to AR projections though. This would have massive applications in parks. Guests wearing AR projection goggles while touring the park allowing for a hugely immersive experience
>What do you think the word 'poor' means when referring to quality?
poor (adjective): of a low or inferior standard or quality.
>Might be kinder to say 'less advanced' or 'simpler' or 'earlier'.
Kinder towards what? Is a multinational like Disney sensitive or is the technology sensitive to the choice of words criticising it? Or will the researchers take offense to their technology, which is objectively inferior, being described as "poor"?
We're stretching this too thin, inventing issues where there are none...
The connotative meaning of “poor” isn’t really anything that isn’t the absolute best. Words have meaning beyond the dictionary definition. “Poor” in this case means you think the researchers did a bad job, not that their work was a step on the path to something better. “Earlier”, or even “dated”, would have been a much more charitable depiction. And plenty of people are deriving joy from the application of that technology.
I’m really not. And I don’t get offended by very much. Maybe you’re not a native speaker? “Poor” as it refers to quality is a negative qualifier. And it’s typically used in a subjective context. That’s the connotative meaning.
Even the definition used here (taken from Google) indicates so, if it weren’t truncated:
poor (adjective) worse than is usual, expected, or desirable; of a low or inferior standard or quality.
The list of synonyms is even more telling about it’s real meaning: shoddy, bad, deficient, defective, lamentable, deplorable, awful, etc.
These are not words I’d use to describe the technology, having seen it first-hand. The technology is not worse than usual and I really don’t see how it’s worse than expected or is otherwise undesirable. It’s out there and being enjoyed by people.
First of all, I'm not calling it poor, the grandparent did.
I'm saying it's nothing special to call it poor.
We say 10x harsher things everyday in HN for frameworks, languages, etc. Heck, check any thread about Apple products. Don't real people work on those?
Plus, ever read art criticism, or restaurant criticism, or political critiques even in the most respected newspapers? "Poor" is the least harsh of the terms they use. And those are also real people they level those things at...
The second example doesn't, IMO, demonstrate the kind of deformation tracking the example in the OP does. As far as I can tell, they are able to get a depth representation, segment objects from a video, get an approximation of surface normals and reflectivity for those objects, and project a shaded surface onto those objects.
What they do not seem to be able to do without IR markers is project a diffuse texture to an object so that it would stick properly. See the one example with a non-uniform texture, where the fingers of a hand are fanned out - the texture warps noticeably.
Another use of a high speed projector would be to create real 3d display anywhere in a volume swept out by a moving surface. Objects will be translucent, but otherwise real 3d with wide viewing angles.
That’s an amazing idea. A naive approach would be to slice the volume into N surfaces (say, ~30 layers to stay above 30fps) but there is probably a much more efficient organic pattern or interlacing that would give good volume and angle coverage at much higher resolution - think of crumpled cloth blowing in the wind at speed.
The first I heard of it in the 1980s. TI used a laser to project point on a rotating surface. Then someone used a flexible mirror in front of a speaker. The mirror would flex convex/concave changing the apparent distance to a vector display screen.
Just saying a modern 1000fps display could do this much better.
I think it would be very interesting idea for a music show. Tell everyone to dress using white tshirts and create this kind of projection from multiple beacons standing spread in the venue. Not sure if that would be possible but very cool to see IRL and talk about
This is so cool. Now I want to take this ( and so many more things like it that have come up recently ) and show the students at the art school I attended that projection mapping can be so much more than lining up all the parallel lines.
Very cool. I do live projection work[0] and latency is always the killer with immersion. Anything higher than a 1000/90ms latency breaks integration at normal dancer movement speeds. 1000fps seems like overkill but it allows for very fast movement.
It is, but I cheat! The dancer has a small android device with a custom gyroscope app, mounted in the middle of her back. I can get her general orientation accurately this way (more accurately, and faster, than state of the art pose estimation).
I am in the process of bringing together a community around art making like this. Let me know if this is something that interests you.
It is of interest! I am in the process of learning TouchDesigner, looking to integrate it with Ableton Live. I doubt I’ll have enough GPU power for real-time 3D renders based on changing sound or visual input though…
Currently I am considering pre-rendering scenes to given BPMs (where applicable) and doing only limited realtime alterations with TD nodes.
In 1899, H.G. Wells wrote "A Story of Things to Come", which later was adapted become the 1936 film "Things To Come". In the original story, the main characters mention being irritated by the advertisements projected onto the backs of the people they walk behind. Old idea, only now possible without image distortion.
The demo video answered my questions, is short and the tech is impressive. Not entirely sure of the business model though. The non-rigid tracking might be more useful than the projection -- perhaps a Defense application?
Skimming the description, they used a structured light approach (active) for geometry deformation tracking. This is still probably useful for certain defense applications but an ideal goal is often to use passive tracking systems. It's impressive either way.
Lots of theater, performing, and visual art uses come to mind.
Some retailers (especially Japanese ones) have a fascination with the idea of allowing customers to "try on" different colors/styles via an on-screen avatar that's based on a scan of the customer. This would seem like a logical next step.
I'm unsure how much customers would actually want it though, at least after the novelty wears off.
One downside with this approach is that you need something to bounce the light off of (e.g., a surface), so adding virtual objects to AR is difficult if the objects aren't positioned at the surface of real world objects. That's an effect you often want to achieve in AR applications.
132 comments
[ 3.3 ms ] story [ 190 ms ] threadThe youtube video[0] from that page is especially interesting.
0: https://www.youtube.com/watch?v=-bh1MHuA5jU
And they can now do the tracking without the infrared paint: http://www.k2.t.u-tokyo.ac.jp/vision/MIDAS/index-e.html
Hope it doesn't easily scale out to larger spaces and crowds, or the current tech industry would soon have public spaces filled with ads projected on peoples belongings.
Edit: discard this comment, I misread the parent comment.
https://en.m.wikipedia.org/wiki/Vantablack
They literally could have done this for 100+ years.
They do not.
Do you see how what you say makes no sense other than a fear of technology?
You've seen projectors, you've seen how they don't just project them anywhere, it's hard to see how one makes this leap.
I guess Disney Imagenineering dept. would love to do some cools things with this. Or perhaps they already have this capability?
These are the kind of things were I'd like to have a commercial version already!
Edit: Also, the url in the original post is from 2016! wow!
(@dang maybe add (2016) to the title?)
I'm having a little trouble visualizing how this non-rigid projection mapping could get applied in a practical way in the parks today, mostly because it seems like it has a fairly small "active" area (determined by your projector and sensor resolution, essentially). I could imagine this being used in a parade or stage show, for example, but this system seems like it would be pretty restrictive as to where the performers could move and remain in the projection space.
Of course, I'm no expert but throw a few million dollars into this and you can probably come up with some neat stuff even if the projection space is small-ish for now.
Seems an unnecessarily nasty way to refer to what I guess is earlier less advanced work.
poor (adjective): of a low or inferior standard or quality.
>Might be kinder to say 'less advanced' or 'simpler' or 'earlier'.
Kinder towards what? Is a multinational like Disney sensitive or is the technology sensitive to the choice of words criticising it? Or will the researchers take offense to their technology, which is objectively inferior, being described as "poor"?
We're stretching this too thin, inventing issues where there are none...
It really does not. You’re inventing things to be offended by. It simply means it is low quality version of the same thing, which it very much is.
In retrospect, Ford’s first cars were a poor version of their 2019 ones. Doesn’t in any way diminish the accomplishments of the past.
Even the definition used here (taken from Google) indicates so, if it weren’t truncated:
poor (adjective) worse than is usual, expected, or desirable; of a low or inferior standard or quality.
The list of synonyms is even more telling about it’s real meaning: shoddy, bad, deficient, defective, lamentable, deplorable, awful, etc.
These are not words I’d use to describe the technology, having seen it first-hand. The technology is not worse than usual and I really don’t see how it’s worse than expected or is otherwise undesirable. It’s out there and being enjoyed by people.
I'm saying it's nothing special to call it poor.
We say 10x harsher things everyday in HN for frameworks, languages, etc. Heck, check any thread about Apple products. Don't real people work on those?
Plus, ever read art criticism, or restaurant criticism, or political critiques even in the most respected newspapers? "Poor" is the least harsh of the terms they use. And those are also real people they level those things at...
In 1998 I was hired on a contract to do a Fashion Show, where I developed graffix to project onto models who were on the catwalk in this theater....
Basically it was a music + visuals show where the models would walk down the catwalk and present themselves....
Guess what tool I had available to me to make this happen...
FUCKING POWERPOINT
I F5'd that bitch and projected a slide per model on a projector borrowed from work with me manually changing slides with --> for each model....
What they do not seem to be able to do without IR markers is project a diffuse texture to an object so that it would stick properly. See the one example with a non-uniform texture, where the fingers of a hand are fanned out - the texture warps noticeably.
Could you imagine using something like this for reviewing finish options for a product with zero turnaround time?
https://thefutureofthings.com/3029-the-return-of-the-3d-crys...
https://www.youtube.com/watch?v=9U80TK6PAVc
Just saying a modern 1000fps display could do this much better.
Iron Man: The Musical
Benjamin and the Buttons in Concert
0. https://youtu.be/ggRcDQZWD_8?t=1281
In any case, very impressive.
I am in the process of bringing together a community around art making like this. Let me know if this is something that interests you.
Currently I am considering pre-rendering scenes to given BPMs (where applicable) and doing only limited realtime alterations with TD nodes.
https://www.youtube.com/watch?v=qn5YQVvW-hQ
Much easier problem to solve at 1k fps but still cool.
we really don't need this stuff to overload our already maxed out sense organs but here we go!
Lots of theater, performing, and visual art uses come to mind.
I'm unsure how much customers would actually want it though, at least after the novelty wears off.
This is the real beginning of augmented reality, not VR or cell phones
https://youtu.be/deJnHFQtbiE
This gives you an idea but in person the illusion is stronger.