Implants which modulate and reflect incoming radio signals back to the radar device, especially powered by the radio wave itself, is nothing new in clandestine surveillance.
This is beyond even science fiction. I could have never imagined something like this was even possible - I still can’t. Is there a demo of this tech in action?
Rough translation of video transcript (YT transcript translated by deepl.com):
Lenses.
This year's theme is now holograms
and it's pretty funny that this is
is called a holocam, and it's pretty funny.
If you look at it right now, if you look at the spring right here.
you can see a little bit of blue
you can see a little bit of blue, which is actually
what is it?
it's actually acting as a camera, so
Now, if you listen to the explanation
what we call holography.
is that it's now reflecting light, so we can gather that light
and you can collect that light and act like a camera
like a camera, so for example, you can now
what?
the meat.
it can recognize the blue color
and only recognize the blue part.
Jace's
holocam. It's quite
interesting.
For sure, was disappointed with the original article for only linking generic keywords back to their own site. At least the CES one is from the horse's mouth.
> The Holocam technology "uses holographic in-coupling, light guiding and de-coupling elements to redirect the incoming light of a transparent medium to a hidden image sensor."
It looks like it still requires a sensor, it can just be hidden in the frame of the glass.
Sure it has a sensor, it's not magic. But OTOH if it can capture the POV of a real line of eye contact with another image projected _through_ the camera, the effects are transformative, it doesn't matter whether ultimately there is a sensor somewhere or not
Do a search on "non line of sight imaging" (aka NLOS). Here's just one public paper.[1] If this is what is being published, what do you think the intelligence agencies have?
> Given the current fear around hidden cameras in Airbnbs, the idea of every single window (or even shower door) in a rental property being able to spy on you is a little disconcerting.
While there are some really interesting potential applications for this tech, it is also more than a little disconcerting.
The ubiquity of camera phones and the emergence of tech like those Meta glasses is already pushing us to disconcerting (albeit interesting and in some cases very useful) places, but some of these cutting edge concepts worry me. WiFi seeing through walls also comes to mind…
I'm surprised this isn't the top comment. I'm all for the benefits of this tech, and hadn't even thought about the airbnb style implication.
People didn't like that Google Glass could always be filming, now we don't even have a physical camera.
Rayban/Meta (I believe) have a sensor to detect that the wearer has not attempted to cover the light which shows that the camera is in use, but how will that work when every piece of glass is a camera.
The article is poorly written, as it only discusses the camera component. Strangely, they chose stock images of holographic and optical displays, but didn’t mention that even once.
1) ZEISS unveils holographic Smart Glass at CES 2024, both for displays/projection/filtering, but also another component which is a holographic camera
2) The holocam works by utilizing coupling, decoupling, and light guiding elements to redirect incident light to a concealed sensor, eliminating the need for visible cutouts or installation spaces in visible areas.
3) ZEISS doesn’t plan to be manufacturer, so other companies can use the tech
> The Holocam technology "uses holographic in-coupling, light guiding and de-coupling elements to redirect the incoming light of a transparent medium to a hidden image sensor."
That suggests, at least to me, that you'll need something more than just a simple sheet of glass. There's probably some engineering required to allow light to be guided and redirected towards what sounds like a typical camera sensor.
I don't actually know what that specifically means in this case.
If I were to guess, I'd assume that "de-coupling" means "splitting the spectrum into RGB." RGB, because it's easy to reproduce for human viewing, but you could do this with other spectrums as well.
I imagine "in-coupling" refers to how they get the light to a sensor.
I used to worked for a company that would use lasers to "build" mirrors inside of blocks of glass. They could tune those mirrors to reflect just red, or just blue, or green, or any combo.
I can imagine using this sort of tech to build panels of glass that can route "de-coupled" RGB to travel spectrum specific sensors.
Or, it might be possible that they're just capturing the full spectrum of light, but using microscopic etchings or surface treatments to bounce light down a layer of glass, somehow bouncing "pixels" of light down the pane of glass in a way that isn't lost while capturing other "pixels".
I say "pixels", but what I mean is "a portion of the glass." If you think of the glass as the polished ends of fiber optic strands, all bundled together, each of them routed to a specific sensor pixel - that's sort of what I mean. If you can etch your prisms/combs/gates fine enough, you can have higher "pixel" density from your glass pane.
Then again, they're the lens experts and could be doing something else entirely. I'm really just guessing.
Yes I interpret it as guiding light inside the glass such that the sensor is on the lateral aspect of the glass embedded in a glass frame perpendicular to the target image.
Sounds like they embedded a light splitter with some sort of periscope style lense inside a glass pane.
So not actually bolted oblique to the glass as GP suggested.
>ESA and NASA space missions have carried this trailblazing ZEISS technology on board for many years. It is also well established in the semiconductor and medical technology sectors.
Huh. Seems like it should be fairly easy to find info on then... though some googling around makes me think they might just be referring to their more general diffraction gratings and whatnot.
Could you put a polarizer on this and have that filler out the other side of the display, so a smart window is one way visible? That could make smart glasses and headsets much better.
I doubt you'd even need to. The fact that it's not a uniform grey blob means they're already controlling how the light leaves the device. I suspect it's already directional.
Edit: theres an actually good demo video showing the real state of the tech rather than mockups: https://www.youtube.com/watch?v=NORPeCcIXRQ buried below in the comments so just surfacing higher. everything else is artists lying to you.
---
> Glass surfaces can also generate energy. The microoptical layer in the window pane absorbs incident sunlight and transmits it in concentrated form to a solar cell. This combines the advantages of conventional windows – natural light and an unrestricted view – with the additional benefit of efficient energy production.
Nah, this will only be good enough for sensor-level power (like 5W from a whole window). Only useful in very limited circumstances. It's not going to replace normal solar power.
you mean because it can also power a wireless transmitter or a large memory array storage? cuz a hologram that's not "plugged into something" might turn out to not be that useful.
It is, but do you really want your windows to have integrated cameras? You can't just add this on to a window because the light is transported to the edge of the glass and that is inaccessible in normal windows.
> coupling, decoupling and light guiding elements to divert incident light to a concealed sensor
So, there's a camera in the dash looking up at the windshield and focusing where it expects to see a face, thereby using the windshield as a reflector? And maybe there's some additional etching and deposited films in the windshield to support the angles required?
And perhaps you can put cameras elsewhere, and similarly subtly modify the windshield or other glass to look at other things as well?
Wow. The original article was talking about doorbells and TV broadcast applications. That looks like the quality of the original moon landing broadcast.
Second, #3 is always a red flag for me. It is sometimes code for "we can do this in the lab but we have no idea how one would manufacture it." A similar analogy is "we've got this great idea for a program you can license but no one here knows how to actually code it up."
Third, the impact of this going mainstream would be hard to underestimate. All those people working on transparent displays like they do in sci-fi movies? Yup they could do that. A video conference system with solid eye contact (mentioned in a couple of places) sure you could do that too. A mirror that could show you wearing different clothes? Yup I could see how that would be coded.
That #3 though. That is what tempers my enthusiasm. Did I miss any announcement that they had a display at CES? (or was it just an announcement) If the former I would seriously consider flying over to Vegas to check this out.
Does Zeiss manufacture any public facing camera currently ? All their general public facing projects are collaborations with Sony or Vivo or other makers, and I think they only manufacture lenses, so nothing with a high quality chip in it.
It's probably another story on the medical side, but this doesn't fit a highly specialized, peofessional only niche with a PC doing the processor on the side.
I don't disagree. One way to know if it can live up to the hype is that it will be the lighthouse feature of the next iPhone which will both not have a notch for the front facing camera, it will do "touch anywhere" touch ID and process all of the screen operating gestures visually rather than capacitively so that you can operate you phone while wearing ski gloves if you want too.
Ooh thanks for that! Showing monochrome only. I wonder if three layers of glass would get you color? And fixed focus (not too surprising) but that answers two questions I had initially.
As they claim this can also do display (presumably to replace the camera element with a projection element) it would be interesting to see what level of opacity they can achieve in a brightly lit environment.
Next someone will tell me they have a way to “ssh” into the universe and get process details on every entity in it including every human and what’s happening.
I guess the closest thing you can get to that -- ssh into somewhere and get details on every human -- is to ssh into the government's database on everyone remotely of interest, or even accidentaly connected, (in essence almost everyone), both made of computer and human agent collection and analysis.
It's a reverse light guide. We've been beam forming for a long time, it's unsurprising that the reverse is possible (imaging through a light pipe).
The principal issue will be gathering enough energy. A well lit source like a bathroom mirror (mirror behind the light guide) could work pretty well I'd wager. If the light guide is too efficient then it will appear opaque, so there is a trade-off.
I find "turns any window" pretty misleading. Unless I'm missing something this needs very special glass or at the very least a special coating/laminate.
For folks worried about privacy, it will almost always be more convenient and cost-effective to install a tiny spy camera somewhere.
Zeiss isn't going to aspire to sell cheap glass on razor thin margins.
Whilst I kind of see where you and the sibling comment is coming from, Zeiss do make glass- they make precision glass. Making cheap glass would be quality diversification, which is not exactly unheard of.
It sounds reasonable, but if they diversified quality, it would mean making precision glass and then also somewhat less precise glass. They will not start mass producing large panes of low margin glass. It's not what they're good at and it's not a lucrative market. Like Intel isn't going to start making jellybean parts.
They could license their name to some existing glass company, but they still wouldn't really be the ones producing it, and I think Zeiss has avoided diluting their brand name like this so far.
Zeiss and Schott are both owned by the Carl Zeiss foundation; the origin story is somewhat covered by Material World by Ed Conway. It's a fantastic book and well worth a read.
As a side hustle, why don’t you start building and selling desktop PC’s to local businesses, competing with HP, Dell, etc on margin? That’s also tech, right?
While building cheap PCs at razor thin margins is adjacent to whatever your tech job is, and probably something you’re able to do, it probably wouldn’t be the most profitable use of your time
> It's a reverse light guide. We've been beam forming for a long time, it's unsurprising that the reverse is possible (imaging through a light pipe).
Beamforming in RF frequencies is old, beamforming at optical wavelengths is pretty new shit. Doing the reverse, that is receiving a signal, is brand spanking new fresh out the cow hot shit at any frequency.
No, it's not really new. Small form factor is relatively new but even then the issue lies in yield and quality control.
Projector light engines include potentially many light-beam forming condensing and projection lenses precisely to concentrate light into a uniform quadrilateral. That's not new. The industry continues to advance, though.
Even holographic projection isn't new. This is just that, except the light is (sort of) taking the reverse path.
Technically there is no reason why it can't also project light outwards simultaneously. However light guides aren't really reversible like that: the light usually exits through a small fraction of the guide's external surface area. Reversing that means the entire external surface area is potentially collecting light, which would result in some undesirable caustics in all scenarios I can imagine. Light engines are in part designed to account for this (by reflecting or sinking light into an absorber) but this is still pretty different from what Zeiss is promoting.
For a small permanent installation where you are in control of the lighting I could see this working relatively well, but I have a hard time imagining you could get close anything resembling photo quality without a lot of environmental treatment. Conversely holographic projection is pretty doable on a mobile platform like a headset.
This is pretty new only in that it's probably at least an order of magnitude more difficult to accomplish and thus hasn't been viable up until now. Fundamentally none of the concepts are new, as far as I can tell.
As an example look at your nearest window and imagine it divided into a grid of uniform pixels. Each pixel is a small mirror that reflects a point of focus (wherever you are) to a another, smaller point on an imaging sensor somewhere in the frame. This would look pretty jarring (and jagged) to most people, until you layer a complimentary piece of glass on top of it to make the exterior flush. The two pieces of glass would be high enough precision that once you put them together they are effectively one piece of glass. Ta da.
This is effectively the same process as making any other multi-lensed glass, and that's Zeiss's wheelhouse.
> Zeiss isn't going to aspire to sell cheap glass on razor thin margins.
Especially true since Zeiss isn't planning on selling these at all. They're selling licenses to the tech.
I doubt that undermines your privacy argument in any meaningful way however. Even if the license was free, the cost of producing the components is certainly magnitudes higher compared to the cost of current tech that accomplishes similar goals.
3D print lenses in a header for threads to be arbitrarily placed in a [COMPOUND] lens and pull that feed with AI spatial mapping (yes these are easy now)... EYE
And you can make these in many increments - micro even... "Hey NSa, super simple optical prince Rupert drops on a composite eye" (self destrucing fiber lens when discovered)
I remember a friend on a team adjacent to the surface table team talking about them trying to do this when they switched from projection tables to tables with screens in them. As I recall with the projector based tables (the ones that looked like old cocktail arcade tables) I think they were using an ir camera to detect items and touches on the 'screen'.
Yes, the original Surface tabletop used IR cameras. Its successor, the Samsung SUR40 had in-cell sensing, i.e. photosensors embedded into the LCD so that it could capture a 960x540 image of the table surface.
After some searching I found a patent I think may be related to this https://patents.google.com/patent/WO2020225109A1/en because it uses the phrase "Holocam", is German and was filed by Audi (the press release mentions automotive applications as the primary initial use case). It's a translation from German which makes it a bit tougher to parse than the usual patent.
The total lack of any deeper information beyond the bold yet vague claims in the press release is frustrating. The PR makes it sound like a miraculous breakthrough destined to change everything. The source release on the Zeiss site only adds two bits of info.
> "The transparency of the holographic layer has only a minimal effect on the brilliance of the image reproduction. It is also possible to detect spectral components as additional information to complement the visible image. The resulting data provide insights into environmental contamination such as air pollution and UV exposure."
However, experience shows that in reality bold+vague claims like this inevitably come with significant trade-offs and constraints which limit its applications (little things like cost, power, fidelity, size, speed, etc). This is especially true in early implementations of new tech. That said, it may still be both interesting and useful. Unfortunately, we have no way to even think about how it might be useful because Zeiss marketing has chosen to play 'hide the ball' instead of just releasing a technical explainer outlining relevant trade-offs, limitations, etc.
If I was talking to someone from Zeiss my first questions would be about how much the additional components impact the optical characteristics of the glass, what the resolution of the resulting image data is, how large are the components needed at the edges and how far away can they be from the capture zone? Then, of course, how the output of the resulting imaging system maps into traditional camera/lens metrics like f-stops, aperture, imager size/density, gain, focal length, etc. Zeiss is an optics company after all.
> This means that everything from the window in your car to the screen on your laptop to the glass on your front door can now possess an invisible image sensor.
Retailers, marketers, and data brokers are salivating.
I can't wait until the windows in our homes plaster ads over everything every time we look outside.
It'll sure be distracting when it's the windshields of our cars, but I do look forward to the legal drama when companies get sued for painting their "holographic" ads on top of the adspace other people already paid to pollute with their own advertisements.
> Holographic 3D content permits more design, branding, guidance and information functions. For example, side and rear windows can be used for eye-catching Car2X communications. It is also possible to black out window glass or make projected text and images visible only from the inside or outside. Video content is also supported. (https://www.zeiss.com/corporate/en/about-zeiss/present/newsr...)
I'm not sure where you live, but in many places there's strict city/county level ordinances restricting signs (Particularly lit up signs) and advertising. There's a reason people's backyards aren't littered with billboards.
Anyone that's experienced a cracked windshield understands that this won't be going anywhere outside of some very niche and expensive cars.
They're specifically marketing to car manufacturers in the press release video. Which is unfortunate because I don't need more non-haptic buggy interfaces in my car. The technology itself is amazing even though the holo-cam appears to be just greyscale and rather blurry right now. The ability to embed the optics in/near (hard to tell from the marketing) the edge of the glass is awesome as you don't need a projector that is separate (by much) from the display surface.
Yeah it sounds like a great option to place a camera invisibly in the middle top half of your screen so you can actually look a person in the eye when videoconferencing. No more weirdly looking down or to the side for everyone.
I'm sure it can be used for creepiness as well but I see the benefits too.
I never understood bigger and more numerous camera bumps until I used optical zoom and took photos at night. Image quality with thick lenses is incomparably superior to that of thin lenses and 3 image sensors collect 3x as many photons as 1 so you don't have to hold the phone still for such a long time. Ideally the entire backside of my phone would be lenses
I regularly use my DSLR and print photos with my own canon photoprinter. i bought a 70mm-300mm lens for a bronyconvention (galacon) and it is simply impossible to get images of that quality with a phone.
and the 50mm f1.8 is really great considering the low price.
Yes and I manually sort through thousands of sunflower seeds every year to get the good ones for my breeding program but our obscure hobbies aren’t the norm.
I tried to go on holidays with my dslr… what a pain. So heavy (unless you buy a $3000 mirrorless) for photos I never print and the times I print them no one ever take the time to look at them. They stay in a closet until thrown out.
The quality from my iPhone is not that great as a 4K monitor wallpaper compared to my DSLR. But given the fact that it weights nothing and on holidays the iPhone allows me to text, use GPS and google maps, play, read articles, watch videos and allows an immediate editing of my photos in raw and created a shared album right away … it’s a no brainer anymore
You're right about the strengths of the phone camera, and I also take many more pictures with it. But printing large prints, framing and hanging pictures is absolutely worth doing, much more than printing 4×5s to flip through. I love having them and everyone who comes into my home takes some time to look at them.
Just buy a used mirrorless camera, it’ll set you back a couple hundred, not 3k.
New technology is fun and all, but even decade old digital cameras are more than good enough for good holiday snaps that you might want to crop and print.
Buy any compact camera with a 30x optical zoom and it will take better pictures than a smartphone. Maybe not all of them, but you can zoom on details from a distance, take pictures of animals that would flee if you try to get close, etc. The last one I bought about 10 years ago was about 300 Euro, maybe 200. It has wifi to backup pictures to my phone.
Your 500 euros DSLR sucks compared to my 1500 euros phone for gaming. Do you carry your DSLR in your pocket every time you leave the house? These is obviously a matter of specialized vs general use.
For the everyday needs of the average non-enthusiast consumer, I'd argue that an iPhone is simply better than a DSLR or mirrorless - not just more portable or more convenient, but capable of producing reliably better images.
Sure, the DSLR is the obvious choice if you're a serious photographer, but most people aren't serious photographers. If they buy a DSLR or mirrorless camera, they're going to use the kit lens and leave the mode dial on auto. For people who just want to point and shoot, the iPhone's computational brilliance shines through.
The iPhone isn't so much a camera as a generative algorithm that happens to use image sensor data as a prompt. That's infuriating if you're a photographer who just wants full control over a big sensor, but it's tantamount to magic if you don't know what an f-stop is and have no inclination to learn.
I'd bet that if you gave my mother an iPhone and a DSLR, she'd get consistently better images from the iPhone, even if we gave her a one-day crash course on photography first. Sure, she might fluke the odd decent photo with the DSLR, but 90% of the time, the iPhone's algorithmic guesswork is going to beat better imaging hardware with dumber software.
Phone-cameras produce muddy trash. Ever tried to make a good portrait with difficult light? That was the reason i began photographing seriously, because the photos i made with my phone on the galacon a year prior were trash.
And yeah u have to learn how to properly use a dedicated camera which is why i only shoot with manual settings, for casuals phones are enough.
So we should call those shots snapshots when made on a phone. Well, the only thing you need to know when shooting with dslrs is to move the dial to Auto/P. If you want to play with DOF, move it to Av. If you want to shoot the dog, move it to Tv and 1/500. If you want to shoot an airshow move it to 1/1200 or higher. Is that too much? :D
Yeah I seriously doubt that. Not without a lens that costs 3x-4x times the camera. If you spent 500€ on a kit that makes iPhone or Pixel photos look bad... you bought it stolen, lol.
Given that this will probably only produce a relatively low res, high SNR image, it might be the sweet spot is to use this tech to capture a true eye-contact perspective from the glass in front of a screen, then neural net combine it with images from offset cameras, to create a 'true' through-the-screen view.
It seems to work fine with glasses for me. Only it loses tracking if I look too far away (bottom of the monitor). Also it thinks I have olive green eyes for some reason.
Having the screen be the camera 1) removes the need for the holepunch as well as magic island workaround 2) could clearly be used in tandem to improve the effect, even potentially reducing the required computation. Sure, Google also fixes images in post, but that's still not a substitute for better cameras. You're always information limited.
This doesn't solve the issue, because it looks like the person is always making eye contact! You don't want that—you want it to look like the person is making eye contact only when they are actually making eye contact. Constant eye contact is exhausting.
It does what??? How have I never heard about this before? Is this deployed right now? Is it always enabled? I have so many questions, don't even know where to start. That sounds amazing.
If I look at my camera lens when I'm talking and I look at the screen while you're talking, and then you do the same, then there's no issue. Am I the only one that does this? (I know the answer. I am. At least at my company, I am.)
Years ago I was able to visit Zeiss in Oberkochen. They had a fantastic headquarters with a few older lithographs. I think a couple of the instruments were 80 millions dollars or so.
There’s a facility across from the Autobahn that was so sensitive trucks going by would throw off their machines. They had to put padding on the autobahn to prevent it. This was after they put the foundation on some kind of suspension. My coworker said they hire the most PHDs in Europe.
Sounds like LIGO, but they're in the US. They had to put the AC unit for their entire facility on suspension because their instruments were so sensitive. And they ask people to not accelerate or decelerate so quickly when driving around the campus.
You need to attract PHDs and account for logistics. If you want to attract top talent you need to be in a desirable place or be a desirable place with access to desirable places.
"Webcams that enable you to look anywhere on your screen."
Minor aside, but does anyone actually care about this? Forever ago, I was told to try and look into the camera in order to project eye contact during video calls, but now that just seems like a cultural hangup that arose from people not being familiar with video calling. Now that it is more ubiquitous, I feel like we all have collectively agreed that the eye contact thing is unnecessary?
It's not just cultural. We've evolved to recognize eye-contact. Newborn babies immediately know when you look at them. Eye contact with a dog communicates dominance.
You've just acclimated to losing that signal during video chats. Bring it back, and you'll have a richer experience.
More generally, we have evolved special neural hardware to recognize gaze, with a special case for when gaze is directed at our eyes. Gaze is important because it's a sign of where people are attending. And as Herb Simon said: "in the information age, attention is the scarce resource."
No, and I'm absolutely not willing to give up the mechanical camera shutter I have on my laptop for that feature. They could add a mechanical shutter to the hidden camera in the frame but why do all of this.
Edit: Should have said "mechanical lens cover" instead of "mechanical camera shutter"
This is an odd, almost reverse nitpick. The word "shutter" is general-purpose. It's literally a thing that shuts the opening. There's one inside an SLR camera, as you point out. There's often one on a webcam. They are common on windows on houses.
All words are general-purpose, but when we arrange them they adopt new meaning. I'm not nitpicking, and just pointing out that the meaning of those words as arranged may not be perceived as the author originally intended.
Alternatively, I'd suggest a different arrangement of words like "lens cover", "webcam cover", "webcam cover slider", or "webcam privacy shield" to be more accurate.
Of course we were able to grock what "mechanical camera shutter" meant in this context, although at first I was confused and wondered if in fact there were webcams with mechanical focal plane or leaf shutters.
Idk, I’ve noticed when I’m on sales calls, some sales people make a very conscious effort to look into their webcam, and the effect of them “looking at me” in the video feels:
A)Initially, a little bit odd, because 98% of people don’t do it/ I’m not use to seeing that
B) Then feels pleasantly good/ natural.
I’m not going to make a decision on a performative behavior like looking at the webcam - at least, not consciously - but I’m sure if/ when this becomes the norm, webcams NOT focused on eyes will seem very “off”
> but now that just seems like a cultural hangup that arose from people not being familiar with video calling.
Eye contact is about as old as humankind. So maybe in 300,000 years we can check if we get over it with video calls.
It is not that communication is not possible without it. It just removes an additional layer.
Eye contact conveys all kind of useful meta data. Depending on the circumstances and other verbal and non-verbal cues it can mean "I'm listening to you", "oh, maybe let's push more on this topic", "have you also caught that?", "perhaps it would be wiser to not talk about that", "i'm specifically addressing you" or "can you help me out here?". And these are just the ones immediately applicable in business settings. Let's not even talk about all the ways it can be used during flirting.
And it works on a subconscious level. I regularly DM role playing games, both online and in-person. The lack of eye contact is a serious impediment in online games. One can use eye contact, or the lack of it to judge engagement. One can signal with eye contact turns or who an NPC is talking to. It can also be used to help less talkative players seize amazing role playing moments.
Can we get by without it? Sure. We can communicate anything using a 1-bit communication channel using morse code. It is a bit like dancing in shackles. Can be done, just leaves something extra on the table.
To clarify I didn't mean that all eye-contact is cultural, I meant specifically during video calls. I was told that if I didn't look into the camera during my interview, it would be perceived as rude or unprofessional.
No, we haven’t agreed that. I continue to feel somewhat rude when not looking at the camera during a conversation. And it matters to nonverbal communication.
I also still habitually look at the camera (due to said training early in my career), but I think that the important thing is I don't think it is rude when other people don't do it. And in my experience, the vast majority of people I interact with don't either, which is why it feels like we have agreed that it is not important.
Consider the possibilities when the screen-glass camera view could be different for each of the participants on your videoconference screen - so as you shift your eyes from one to another, they get different experience of eye contact.
Staring into your camera on a multiparty call means everyone feels like they are getting your eye contact, which devalues it. Eye contact moving between participants is the gel that holds conversations together and it's why video calls are still stuck in the conference call era in terms of talking over one another.
I feel this is not an issue with 13 inch laptops. But I recently installed a webcam on my 32 inch desktop monitor and it is really distracting how I'm always looking way below the camera. Currently using Nvidia broadcast to make it better but it does lose tracking if I look at the bottom of the screen.
> Glass surfaces can also generate energy. The microoptical layer in the window pane absorbs incident sunlight and transmits it in concentrated form to a solar cell. This combines the advantages of conventional windows – natural light and an unrestricted view – with the additional benefit of efficient energy production.
Agree. Obviously a few orders of magnitude in cost reduction would be required... but this seem like it could have interesting potential for energy generation in high-rise buildings, which currently have near-zero solar footprint. With that said, the lack of any mention regarding efficiency makes this part of the press release seem like a bit of smoke and, erm, mirrors.
The telescreen received and transmitted simultaneously. Any sound that Winston made, above the level of a very low whisper, would be picked up by it; moreover, so long as he remained within the field of vision which the metal plaque commanded, he could be seen as well as heard. There was of course no way of knowing whether you were being watched at any given moment.
252 comments
[ 3.2 ms ] story [ 325 ms ] threadThere has to be an imaging sensor somewhere. Where is it?
I strongly suspect most of this "magic" will condense out of this cloud into a puddle of marketing sponsored bovine excrement.
It looks like it still requires a sensor, it can just be hidden in the frame of the glass.
1: https://www.pnas.org/doi/full/10.1073/pnas.2024468118
While there are some really interesting potential applications for this tech, it is also more than a little disconcerting.
The ubiquity of camera phones and the emergence of tech like those Meta glasses is already pushing us to disconcerting (albeit interesting and in some cases very useful) places, but some of these cutting edge concepts worry me. WiFi seeing through walls also comes to mind…
People didn't like that Google Glass could always be filming, now we don't even have a physical camera.
Rayban/Meta (I believe) have a sensor to detect that the wearer has not attempted to cover the light which shows that the camera is in use, but how will that work when every piece of glass is a camera.
The Zeiss site is a much better read:
https://www.zeiss.com/corporate/en/about-zeiss/present/newsr...
In summary:
1) ZEISS unveils holographic Smart Glass at CES 2024, both for displays/projection/filtering, but also another component which is a holographic camera
2) The holocam works by utilizing coupling, decoupling, and light guiding elements to redirect incident light to a concealed sensor, eliminating the need for visible cutouts or installation spaces in visible areas.
3) ZEISS doesn’t plan to be manufacturer, so other companies can use the tech
So you can't just stick on any existing window or pane of glass? That's good.
> The Holocam technology "uses holographic in-coupling, light guiding and de-coupling elements to redirect the incoming light of a transparent medium to a hidden image sensor."
That suggests, at least to me, that you'll need something more than just a simple sheet of glass. There's probably some engineering required to allow light to be guided and redirected towards what sounds like a typical camera sensor.
If I were to guess, I'd assume that "de-coupling" means "splitting the spectrum into RGB." RGB, because it's easy to reproduce for human viewing, but you could do this with other spectrums as well.
I imagine "in-coupling" refers to how they get the light to a sensor.
I used to worked for a company that would use lasers to "build" mirrors inside of blocks of glass. They could tune those mirrors to reflect just red, or just blue, or green, or any combo.
I can imagine using this sort of tech to build panels of glass that can route "de-coupled" RGB to travel spectrum specific sensors.
Or, it might be possible that they're just capturing the full spectrum of light, but using microscopic etchings or surface treatments to bounce light down a layer of glass, somehow bouncing "pixels" of light down the pane of glass in a way that isn't lost while capturing other "pixels".
I say "pixels", but what I mean is "a portion of the glass." If you think of the glass as the polished ends of fiber optic strands, all bundled together, each of them routed to a specific sensor pixel - that's sort of what I mean. If you can etch your prisms/combs/gates fine enough, you can have higher "pixel" density from your glass pane.
Then again, they're the lens experts and could be doing something else entirely. I'm really just guessing.
Sounds like they embedded a light splitter with some sort of periscope style lense inside a glass pane.
So not actually bolted oblique to the glass as GP suggested.
Huh. Seems like it should be fairly easy to find info on then... though some googling around makes me think they might just be referring to their more general diffraction gratings and whatnot.
Product page: https://www.zeiss.com/oem-solutions/products-solutions/multi...
NASA/ESA mention: https://www.linkedin.com/posts/zeiss_automotive-augmentedrea...
Could you put a polarizer on this and have that filler out the other side of the display, so a smart window is one way visible? That could make smart glasses and headsets much better.
---
> Glass surfaces can also generate energy. The microoptical layer in the window pane absorbs incident sunlight and transmits it in concentrated form to a solar cell. This combines the advantages of conventional windows – natural light and an unrestricted view – with the additional benefit of efficient energy production.
what? holy shit?
you mean because it can also power a wireless transmitter or a large memory array storage? cuz a hologram that's not "plugged into something" might turn out to not be that useful.
> coupling, decoupling and light guiding elements to divert incident light to a concealed sensor
So, there's a camera in the dash looking up at the windshield and focusing where it expects to see a face, thereby using the windshield as a reflector? And maybe there's some additional etching and deposited films in the windshield to support the angles required?
And perhaps you can put cameras elsewhere, and similarly subtly modify the windshield or other glass to look at other things as well?
Second, #3 is always a red flag for me. It is sometimes code for "we can do this in the lab but we have no idea how one would manufacture it." A similar analogy is "we've got this great idea for a program you can license but no one here knows how to actually code it up."
Third, the impact of this going mainstream would be hard to underestimate. All those people working on transparent displays like they do in sci-fi movies? Yup they could do that. A video conference system with solid eye contact (mentioned in a couple of places) sure you could do that too. A mirror that could show you wearing different clothes? Yup I could see how that would be coded.
That #3 though. That is what tempers my enthusiasm. Did I miss any announcement that they had a display at CES? (or was it just an announcement) If the former I would seriously consider flying over to Vegas to check this out.
Does Zeiss manufacture any public facing camera currently ? All their general public facing projects are collaborations with Sony or Vivo or other makers, and I think they only manufacture lenses, so nothing with a high quality chip in it.
It's probably another story on the medical side, but this doesn't fit a highly specialized, peofessional only niche with a PC doing the processor on the side.
As they claim this can also do display (presumably to replace the camera element with a projection element) it would be interesting to see what level of opacity they can achieve in a brightly lit environment.
Next someone will tell me they have a way to “ssh” into the universe and get process details on every entity in it including every human and what’s happening.
Kind of like in the matrix with The architect.
The principal issue will be gathering enough energy. A well lit source like a bathroom mirror (mirror behind the light guide) could work pretty well I'd wager. If the light guide is too efficient then it will appear opaque, so there is a trade-off.
I find "turns any window" pretty misleading. Unless I'm missing something this needs very special glass or at the very least a special coating/laminate.
For folks worried about privacy, it will almost always be more convenient and cost-effective to install a tiny spy camera somewhere.
Zeiss isn't going to aspire to sell cheap glass on razor thin margins.
why not?
They could license their name to some existing glass company, but they still wouldn't really be the ones producing it, and I think Zeiss has avoided diluting their brand name like this so far.
Thanks for the book rec, I'll probably start digging into it this weekend.
As a side hustle, why don’t you start building and selling desktop PC’s to local businesses, competing with HP, Dell, etc on margin? That’s also tech, right?
While building cheap PCs at razor thin margins is adjacent to whatever your tech job is, and probably something you’re able to do, it probably wouldn’t be the most profitable use of your time
Beamforming in RF frequencies is old, beamforming at optical wavelengths is pretty new shit. Doing the reverse, that is receiving a signal, is brand spanking new fresh out the cow hot shit at any frequency.
Projector light engines include potentially many light-beam forming condensing and projection lenses precisely to concentrate light into a uniform quadrilateral. That's not new. The industry continues to advance, though.
Even holographic projection isn't new. This is just that, except the light is (sort of) taking the reverse path.
Technically there is no reason why it can't also project light outwards simultaneously. However light guides aren't really reversible like that: the light usually exits through a small fraction of the guide's external surface area. Reversing that means the entire external surface area is potentially collecting light, which would result in some undesirable caustics in all scenarios I can imagine. Light engines are in part designed to account for this (by reflecting or sinking light into an absorber) but this is still pretty different from what Zeiss is promoting.
For a small permanent installation where you are in control of the lighting I could see this working relatively well, but I have a hard time imagining you could get close anything resembling photo quality without a lot of environmental treatment. Conversely holographic projection is pretty doable on a mobile platform like a headset.
This is pretty new only in that it's probably at least an order of magnitude more difficult to accomplish and thus hasn't been viable up until now. Fundamentally none of the concepts are new, as far as I can tell.
As an example look at your nearest window and imagine it divided into a grid of uniform pixels. Each pixel is a small mirror that reflects a point of focus (wherever you are) to a another, smaller point on an imaging sensor somewhere in the frame. This would look pretty jarring (and jagged) to most people, until you layer a complimentary piece of glass on top of it to make the exterior flush. The two pieces of glass would be high enough precision that once you put them together they are effectively one piece of glass. Ta da.
This is effectively the same process as making any other multi-lensed glass, and that's Zeiss's wheelhouse.
Especially true since Zeiss isn't planning on selling these at all. They're selling licenses to the tech.
I doubt that undermines your privacy argument in any meaningful way however. Even if the license was free, the cost of producing the components is certainly magnitudes higher compared to the cost of current tech that accomplishes similar goals.
I lit (pun) came to gripe about light pipes...
However here is you solution:
3D print lenses in a header for threads to be arbitrarily placed in a [COMPOUND] lens and pull that feed with AI spatial mapping (yes these are easy now)... EYE
And you can make these in many increments - micro even... "Hey NSa, super simple optical prince Rupert drops on a composite eye" (self destrucing fiber lens when discovered)
https://www.microsoft.com/applied-sciences/uploads/publicati...
https://www.microsoft.com/applied-sciences/projects/the-wedg...
The total lack of any deeper information beyond the bold yet vague claims in the press release is frustrating. The PR makes it sound like a miraculous breakthrough destined to change everything. The source release on the Zeiss site only adds two bits of info.
> "The transparency of the holographic layer has only a minimal effect on the brilliance of the image reproduction. It is also possible to detect spectral components as additional information to complement the visible image. The resulting data provide insights into environmental contamination such as air pollution and UV exposure."
However, experience shows that in reality bold+vague claims like this inevitably come with significant trade-offs and constraints which limit its applications (little things like cost, power, fidelity, size, speed, etc). This is especially true in early implementations of new tech. That said, it may still be both interesting and useful. Unfortunately, we have no way to even think about how it might be useful because Zeiss marketing has chosen to play 'hide the ball' instead of just releasing a technical explainer outlining relevant trade-offs, limitations, etc.
If I was talking to someone from Zeiss my first questions would be about how much the additional components impact the optical characteristics of the glass, what the resolution of the resulting image data is, how large are the components needed at the edges and how far away can they be from the capture zone? Then, of course, how the output of the resulting imaging system maps into traditional camera/lens metrics like f-stops, aperture, imager size/density, gain, focal length, etc. Zeiss is an optics company after all.
The paper mentioned in the Description section can be read here: https://www.academia.edu/52566311/_title_Volume_phase_hologr...
My guess is that each of the items in patent diagram labeled 20 and 22 are gratings of this sort, perhaps with the fringes angled at 45 degrees.
Some additional info may be gleaned from the one patent citing this one, invented by one of the co-inventors of the latter: https://patents.google.com/patent/DE102019206354A1/en
The demo looks greyscale, and super low res and low FPS.
I think then intention is it just needs to be enough to know where the users face is and project a HUD at them.
Retailers, marketers, and data brokers are salivating.
It'll sure be distracting when it's the windshields of our cars, but I do look forward to the legal drama when companies get sued for painting their "holographic" ads on top of the adspace other people already paid to pollute with their own advertisements.
Anyone that's experienced a cracked windshield understands that this won't be going anywhere outside of some very niche and expensive cars.
Don't do the whole windshield. My car uses a bit of plastic on the dash to reflect a display. I love it!
https://i.imgur.com/jXfWf5b.png
I'm sure it can be used for creepiness as well but I see the benefits too.
Even the zoom is pretty good nowadays.
Often the choice isn’t iPhone photo or DSLR photo it’s iPhone or no photo at all.
It’s a right faff getting the DSLR out and carrying it around. I brought mine to NYC over Xmas and ended up using my phone mostly.
Also no one I know prints photos out, they look at them on their phone, iPad or maybe TV if they care enough or have a screensaver slideshow.
I’d love to use my DSLR more and every year I tell myself I’m going to but I just… don’t.
and the 50mm f1.8 is really great considering the low price.
New technology is fun and all, but even decade old digital cameras are more than good enough for good holiday snaps that you might want to crop and print.
Sure, the DSLR is the obvious choice if you're a serious photographer, but most people aren't serious photographers. If they buy a DSLR or mirrorless camera, they're going to use the kit lens and leave the mode dial on auto. For people who just want to point and shoot, the iPhone's computational brilliance shines through.
The iPhone isn't so much a camera as a generative algorithm that happens to use image sensor data as a prompt. That's infuriating if you're a photographer who just wants full control over a big sensor, but it's tantamount to magic if you don't know what an f-stop is and have no inclination to learn.
I'd bet that if you gave my mother an iPhone and a DSLR, she'd get consistently better images from the iPhone, even if we gave her a one-day crash course on photography first. Sure, she might fluke the odd decent photo with the DSLR, but 90% of the time, the iPhone's algorithmic guesswork is going to beat better imaging hardware with dumber software.
And yeah u have to learn how to properly use a dedicated camera which is why i only shoot with manual settings, for casuals phones are enough.
And yes, it is better than any phonecamera.
how did we even get here?
https://ios.gadgethacks.com/how-to/disable-facetimes-creepy-...
It's a Facetime exclusive, though - no API so other videoconferencing apps can't use that.
There’s a facility across from the Autobahn that was so sensitive trucks going by would throw off their machines. They had to put padding on the autobahn to prevent it. This was after they put the foundation on some kind of suspension. My coworker said they hire the most PHDs in Europe.
Minor aside, but does anyone actually care about this? Forever ago, I was told to try and look into the camera in order to project eye contact during video calls, but now that just seems like a cultural hangup that arose from people not being familiar with video calling. Now that it is more ubiquitous, I feel like we all have collectively agreed that the eye contact thing is unnecessary?
You've just acclimated to losing that signal during video chats. Bring it back, and you'll have a richer experience.
More generally, we have evolved special neural hardware to recognize gaze, with a special case for when gaze is directed at our eyes. Gaze is important because it's a sign of where people are attending. And as Herb Simon said: "in the information age, attention is the scarce resource."
Edit: Should have said "mechanical lens cover" instead of "mechanical camera shutter"
Alternatively, I'd suggest a different arrangement of words like "lens cover", "webcam cover", "webcam cover slider", or "webcam privacy shield" to be more accurate.
Of course we were able to grock what "mechanical camera shutter" meant in this context, although at first I was confused and wondered if in fact there were webcams with mechanical focal plane or leaf shutters.
A)Initially, a little bit odd, because 98% of people don’t do it/ I’m not use to seeing that
B) Then feels pleasantly good/ natural.
I’m not going to make a decision on a performative behavior like looking at the webcam - at least, not consciously - but I’m sure if/ when this becomes the norm, webcams NOT focused on eyes will seem very “off”
Eye contact is about as old as humankind. So maybe in 300,000 years we can check if we get over it with video calls.
It is not that communication is not possible without it. It just removes an additional layer.
Eye contact conveys all kind of useful meta data. Depending on the circumstances and other verbal and non-verbal cues it can mean "I'm listening to you", "oh, maybe let's push more on this topic", "have you also caught that?", "perhaps it would be wiser to not talk about that", "i'm specifically addressing you" or "can you help me out here?". And these are just the ones immediately applicable in business settings. Let's not even talk about all the ways it can be used during flirting.
And it works on a subconscious level. I regularly DM role playing games, both online and in-person. The lack of eye contact is a serious impediment in online games. One can use eye contact, or the lack of it to judge engagement. One can signal with eye contact turns or who an NPC is talking to. It can also be used to help less talkative players seize amazing role playing moments.
Can we get by without it? Sure. We can communicate anything using a 1-bit communication channel using morse code. It is a bit like dancing in shackles. Can be done, just leaves something extra on the table.
Staring into your camera on a multiparty call means everyone feels like they are getting your eye contact, which devalues it. Eye contact moving between participants is the gel that holds conversations together and it's why video calls are still stuck in the conference call era in terms of talking over one another.
This is pretty cool.
[1] https://en.wikipedia.org/wiki/Light_of_Other_Days