This doesn't make sense to me.. is the article suggesting a picture is created from all previous pictures taken? This would mean 1. No more new pictures in time of how things have changed (a new building is where the car park was) and 2. My picture of thatbird that just landed on that branch is not quite going to turn out.. hell the tree might not even be in the picture..
It's talking about image synthesis. Yes, a big part of that is creating a picture from other pictures, but there is also intelligent inference from sensors and other data sources besides CCDs.
You're right that the idea isn't a good fit for instances like the ones you described, but I think it certainly has applications. For one thing, you could potentially synthesize an image of a part of a city from a particular era taken from a point of view that no actual photograph was taken from. Instead, it's a synthesis of contemporary photos with inferences made from contemporary weather data, contemporary maps, potentially drawings, etc. It's an exciting idea.
Not enough people are photographing every flat surface they can see to get texture details, much less publishing the results or the compositions and designs of their expensive buildings.
It may be useful for historical reconstruction, but this process will never be able to account for the possibilities covered by a real camera. For example, I use cameras to take pictures at sporting events. Google isn't going to be able to predict exactly how the baseball players swing the bats. At best, they will produce a probably image which likely will deviate from reality. Ultimately, I see this tool as useful for historians but no good reason for us normal people to use it on a day to day basis.
I agree. Maybe we're taking the article too seriously though. There is no tool. It's just a really bad article. My guess is it was written by an excited very inexperienced person or someone who wanted to make clickbait.
But what would the day-to-day applications of this be? I mean, if you really needed a photo of a side of a house that is now torn down, or if you needed "virtual" access to an unmapped/photographed area, then sure. But the article is talking as if this is going to replace cameras for all scenarios, specially recreational (as that's the only thing focused on in the article).
I see no exciting uses of this technology for recreational purposes. You wouldn't be able to take a photograph of the Library of Alexandria because that existed before we started gathering the data. And taking synthesized photos of your trips would make no sense, why would you even take the picture at all in that case?
My point is that it's a bad article. Technology is cool though.
It's only a bad article in the sense that it proposes synthetic photography as a good thing. The general public, however, might well embrace it one step at a time, accepting what I agree are terrible consequences. Cameras won't go away overnight. Lower-quality cameras instead become acceptable, as the results are uploaded to and enhanced in "the cloud". You eventually won't be able to capture certain images, but the predictable majority of synthetic "photos" taken by most people will work just fine, with better quality on cheaper, thinner devices. Heck, scatter some subtle product placement in the results, and the whole endeavor can be subsidized. Then we ban real cameras and end child-pornography. You're not against that, are you? Finally, we can conveniently alter any photos that might embarrass corporate or government interests! How delightfully dystopian.
It's also a bad article in the sense that it proposes it as a possible thing. Sure a few of the steps he mentions may be possible, but the assumed conclusion of "and then there will no longer be any use case for cameras" is absurd.
Of course it's not going to work miraculously as the author imagined; there are enough responses here already acknowledging that it's self-evidently absurd for replacing cameras as we know them today. My concern is that few here are imaginative enough to separate that fact from whether or not a future of synthetic photography could happen anyway.
This is sort of like how the Star Trek Holodeck works as a content creation tool.
By modern definitions, the Holodeck clearly has the functionality of a 3D modeling tool and game engine: the crew frequently uses it to build VR prototypes of objects, situations and entire entertainment experiences.
It doesn't offer anything that we would recognize as modeling or animation tools, though. Instead the user describes objects and settings using any degree of precision -- "a steel table 3 meters long", "19th century London" -- to get a starting point, then iteratively adjusts the result by instructing the system.
(Nobody takes selfies on the Enterprise either. I guess they've lost their appeal when you can always just ask the computer: "Show me myself and Geordi smiling against a wall when we visited the Klingon High Council last year.")
I was hoping for something more ambitious like "once you can detect the full electromagnetic spectrum hitting your phone from any angle, you won't need a lens to reconstruct a focused picture." The proposed idea is very tedious and requires a tremendous amount of up-to-date data (just the open state of a door can seriously affect the lighting conditions in a photo).
That's exactly what I was thinking, too, when I saw the picture of the back of the phone without a camera. "Of course! The entire back of the phone could be sprayed with nano-scale sensors that collect photons at varying angles, then you computationally assemble these into a image, using algorithms and the sheer number of data points to compensate for the inevitable noise! Full-field photography, and maybe even enough parallax to capture 3D images! Brilliant! Oh wait, that's not where he's going at all..."
Yeah I was ready to point out that if you don't have a lens, it can still be a camera.
But then it turned out to be a dishonest clickbait title with an article where the author wasn't really thinking clearly or with any sort of deep perspective at all about what cameras are for, or the value of what photographs of actual messy reality deliver.
This arguments only considers photography as a tool that produces images, while ignoring that it is also a tool that records reality.
As an image production apparatus, you can replace photography by machine learning (or really any other image creation process, drawing, painting, raytracing, whatever), and in these cases yes, photography becomes a little less important.
However, a photograph is not only an image, it is a trace of actual photons that existed and imprinted a photographic material. When you look at a photograph, you're not only looking at a picture, you're also looking at the imprint of reality. It's the same emotional effect as looking at your kid's hand print on paper, or wearing your grandmother's ring. There is affect involved that comes from the reality of the experience.
A souvenir picture is not only an image that helps you remember the time when you and your friends were doing something, it's an actual fossil of that moment. And that's what matters in photography. And that's why it's not going away any time soon.
If you treat photos as pleasurable pictures, then machine-generated photos will serve fine (e.g. stock photos).
What if you replace "photos" with "music", "art", or "literature"? Sure, we will get to a point where a computer can write a symphony that fits all the characteristics of Beethoven, or write a stylistically accurate Shakespearean sonnet. But it won't have the weight of the artist's observations behind it.
You are missing the whole point of photography and the reason why people invest into cameras and whole systems. In a world of fake and illusions of today, capturing reality is an important effort that can't be replaced, photos of our memories work with us on deep emotional levels. I am really unimpressed by primary goals of these efforts, albeit for other purposes it might be useful (ie VR in realistic locations).
> If you treat photos as pleasurable pictures, then machine-generated photos will serve fine (e.g. stock photos).
I don't know anybody personally who does that, actually its silly. Or maybe normal to people who (poorly) photoshop themselves with ferraris, luxury and whatnot. Apart from our HR and marketing department of course, they love stocks :)
> Sure, we will get to a point where a computer can write a symphony that fits all the characteristics of Beethoven, or write a stylistically accurate Shakespearean sonnet. But it won't have the weight of the artist's observations behind it.
"I've noticed the many photographers here, [...]. Always the same conventional eyes, noses, mouths, waxy and smooth and cold. It still always remains dead. And the painted portraits have a life of their own that comes from deep in the soul of the painter and where the machine can't go." - Van Gogh
We've been asserting souls for quite a while now and we've been wrong every time. What does the weight of the artist's observations mean? How is that different from the result of machine learning, which is nothing if not carefully considered observations.
The question is whether would you make a difference between a photograph of your kid, and a photograph of someone who looks exactly like your kid (and that you know is not your kid).
I don't expect any reasonable answer to this, but...
There are two sets of photos, both exactly alike, yet one of those sets was created by capturing photons while the other was created by computers. Both were created by pointing a machine at your kid during a trip, and subsequently giving you a digital image. How to you tell those apart?
How do you know it's your kid on your kid's photos anyway?
In Rio you can pay a fee to take a train up to the famous Cristo Redentor statue, where lots of people take pictures. But all around the train station there are little green-screen photo studios that will make a fake picture at the Christo Redentor statue for you for a much lower fee.
Given the choice between these options, which would you pick? Or, at what price points would you consider each to have equal value?
1. Train ride with half hour viewing time to observe the Cristo Redentor. No cameras allowed.
2. Train ride to the statue for a photo of you in front of it, but while the camera can see both you and the statue, you can't see it from your position.
3. Green screen photo of you artificially in front of the statue.
I think a lot of people would, if they thought about it, strongly prefer the first option and ascribe little value to the others. But because they do have cameras and there's an expectation that you take a picture as a tourist, they end up viewing their trip through their camera lens or smartphone screen.
I think it would be a compelling sell if the camera app on a smartphone could prompt users with an offer of a professional-quality photo or video (optionally with the user green-screened in) of a concert, sporting event, or landmark. "Studio X has a professional recording of this event from N angles with better equipment than yours. Add this video to your library instead?"
The killer use for this might be to record/sync school concerts, plays, and sports. So many parents with mediocre cameras all trying to capture the event, and no good way right now to share the output.
Agreed. When I take pictures I don't want a result stitched together from other pictures I took or someone else took. I want to remember my experience that I had at that moment.
This article is lame, and would be unable to generate an accurate picture of the actual scene. There would also be loss of detail, missing objects, missing people, etc. It would feel immensely artificial, too.
I was also hoping for something more ambitious, such as "once you can detect the full electromagnetic spectrum hitting your phone from any angle, you won't need a lens to reconstruct a focused picture," as mentioned by blixt.
At a minimum, I was hoping for conceptualization and theorized advancement of the current display-pixel-as-camera-pixel train of thought.
Actually, not true - highly probable, yes, but not true - opencv already supports everything necessary to reproject a synthetic image based on a set of source images. If I'm taking a picture of someone directly in front of me and they're also visible to two cameras to my left and right in the same visual plane as I am the solution is trivial - after that it does get to be a question of how many camera, how much in phase, and your taste for chewing up gazillions of machine cycles.
Yeah, like when I take pictures to remember what my kids were doing that day, really all that must be done is to extrapolate their stage of growth using their birthday (derived from publicly available databases), infer their likely location and pose from the sounds they are generating, etc. So much simpler than a CCD!
It strikes me that, in a world where "photos" are generated from non-optical data, there would likely be a counter-movement of artists who deliberately seek or build scenes that could not be generated at the time. (In some ways this is just a continuation of our current novelty-seeking; "interesting" images are those that you couldn't simply find on the internet.)
I don't disbelieve for a second that the expressiveness and realism of automatically-composed images won't have a huge explosion and become part of popular media. But I think there will be very long period during which some datasets will be much more salient than others. Think of how many orders of magnitude photos we have of dogs and cats compared to frogs and yaks... can they take the camera off of a product that can't image my friends petting a yak? I'm very curious when is enough, because the human ability to contrive absurd scenarios is frustratingly extensive, and unique "outlier" moments are exactly the ones many people want to capture.
Also I think I _have_ seen a similar art project like this, where the artist took a photo album of Paris(?) and gave people "cameras" that just identified the closest image location by GPS and would just keep regurgitating that image no matter how many you took. I can't seem to find it now...
The article is a great imagining of our grim technological future. A family on vacation crowds in front of the Parthenon among thousands of other eager tourists posing for pictures in the rain. Sweaty, ruffled hair, exhausted smiles (except for Suzie), the photo is sent to Google for processing. The result is spectacular: crowds gone, scaffolding removed, sun high in the sky, clothes unwrinkled, smiles whitened, acne softened, and everyone's face captured at just the right moment. The $2.99 GPets add-on let's you insert the family parrot into the picture too. Years later over dinner the family reminisces about how fun that sunny day at the Parthenon was.
Our memory is fragile and our eyes easily deceived. I'm hopeful this technology is introduced to the public first, so other institutions aren't able to abuse it before most people understand how untrustworthy pictures are.
Except that's what people want. If you do any diner with friends of familly, people always talk of past events in a better way that it really happened. If you go to facebook, people don't post their life, they craft the life they want people to think they have.
Why do you think magazines use photoshop so heavily? People don't want the hot girl spare tires. Then want the perfect hair and big boobs.
It's all fake, but it's fake that is requested and voted for with money.
The problem here is not the tool, it's what kind of society we are building.
If such a database existed, I think there'd be vastly more interesting use cases and applications implied than 'replacing an extremely specific minimal subset of photography' - for example, seamless visual world-travelling in VR?
If you insist on inserting yourself or family members into the happy tourist snap, that shouldn't be too much of an imposition, either: just take a bunch of selfies in advance and the software will stitch the two images together for you.
This doesn't seem like something people would want.
For example: Banksy just tagged a building downtown. You rush there to get your picture in front of it. You can't take a photo of yourself with it because it's not yet in any of the databases used by your quasi-camera to stitch together photos.
It seems like the world is too variable for something like this to function practically for any but the most boring photos.
In the near-term, your camera snaps a low-res image that's sufficient to identify Banksy's work and generate a gigapixel zoom that will allow you to appreciate it as if you were standing in a nearby window. A discerning eye might notice the details aren't quite a 1-to-1 match, but it's close enough for the ignorant masses to miss. You complain on HN about the inaccuracy, finding some agreement, but within hours, future images of the same tag are spot-on. The next time you view the image on your phone, those inaccuracies have been corrected. If you hadn't downloaded it off your camera to show here, there would be no evidence anything was off.
Five years later, you snap another photo of Banksy's latest, with an even lower-res camera. Two possible outcomes:
1. You've violated Banksy's copyright, whether he wanted that copyright or not, and the offending image is not present in your photo. You may or may not receive an automated warning.
2. You've recorded a crime, the authorities are notified immediately as your photo processes, and the image is again not present in your photo so as not to encourage the proliferation of graffiti.
The endgame leaves you with no camera, no one cares about Banksy who's in jail by now (with your help!), and why are you so concerned about violating copyright and encouraging crime, anyway? Why should technology enable you to do any such thing?
I guess I've confused the issue with the Banksy example. Forget about that. Say I want to take a picture of a robin on the first day of spring. I take a picture of my deck with the robin sitting on it. This software knows I'm in my house looking out at my deck. It knows what my deck looks like. It doesn't know that there's a robin there at that moment, what exact location it's in, how big it is, etc, so it's not going to be able to reproduce that.
Maybe I could carefully explain to the software, "there's a male robin in this exact location and pose on the deck" and it could generate one, but why would I want to do that instead of just having a regular camera?
What does this do that I can't do with a camera? It sounds like a way more complicated, processing-intensive method to achieve an inferior result. It's a solution in search of a problem. It's not like decent cameras are prohibitively expensive, seeing as we've managed to include one in basically every phone now.
Most of the pictures I'd want to take are of something unique. Software may be able to figure where I'm standing, what permanent features are nearby, what the weather is like, what the people I'm with look like, but it's not going to know the transient features, like there's a lady with a weird looking hat in the background that I wanted to capture. So, what's the point?
You're right, what the author here is suggesting is not something you would want. He's imagining a magical omniscient fantasy technology, the closest approximation of which would leave you disappointed and robbed of agency as a photographer.
I propose, instead, you consider the elements he stitched together to reach his delusion and whether or not they could still lead to a very similar, regrettable future.
What's described in this article is some kind of automated postcard production: when you're at a tourists' attraction, just press a button on your phone and it produces an image of the location (with or without an insert of your face) based on a library of images.
But that's not really "photography" (it's to photography what miniature plastic Eiffel towers are to architecture).
A much more interesting and much more futuristic approach would be a device capable of recording a whole scene without a conventional sensor and lens; something that would record photons not because it's hit by them, but because it knows where they are.
So this device would record all the light waves's position in a scene, at a given instant (the instant the image is taken), and then it would let you later reconstruct / produce any image from any position in the scene, with any kind of focus or bokeh, or whatever.
It would also let you walk into the scene like in a real "mannequin challenge", etc.
There was a raspberry pi project with a screen, GPS, 3G which was a camera without a lens. It would look up the closest Flickr picture to your current location and possibly orientation from metadata and add it to your collection. Can't seem to find it though.
so instead of snapping a picture, you just select one from the set of pre-made professional ones. wait we've been able to do this for decades, and yet people still take their own pictures. this is also a bastardization of what google's gcam technology is doing. It stitches together images at various exposure rates to get what would otherwise require a massive sensor, or a super steady hand, it's not creating some deep learning based frankenimage, as this article suggests.
51 comments
[ 3.0 ms ] story [ 107 ms ] threadYou're right that the idea isn't a good fit for instances like the ones you described, but I think it certainly has applications. For one thing, you could potentially synthesize an image of a part of a city from a particular era taken from a point of view that no actual photograph was taken from. Instead, it's a synthesis of contemporary photos with inferences made from contemporary weather data, contemporary maps, potentially drawings, etc. It's an exciting idea.
I see no exciting uses of this technology for recreational purposes. You wouldn't be able to take a photograph of the Library of Alexandria because that existed before we started gathering the data. And taking synthesized photos of your trips would make no sense, why would you even take the picture at all in that case?
My point is that it's a bad article. Technology is cool though.
By modern definitions, the Holodeck clearly has the functionality of a 3D modeling tool and game engine: the crew frequently uses it to build VR prototypes of objects, situations and entire entertainment experiences.
It doesn't offer anything that we would recognize as modeling or animation tools, though. Instead the user describes objects and settings using any degree of precision -- "a steel table 3 meters long", "19th century London" -- to get a starting point, then iteratively adjusts the result by instructing the system.
(Nobody takes selfies on the Enterprise either. I guess they've lost their appeal when you can always just ask the computer: "Show me myself and Geordi smiling against a wall when we visited the Klingon High Council last year.")
For that matter, if we can simulate the lighting, etc. of an object, to the finest detail, why do we need the object itself?
But then it turned out to be a dishonest clickbait title with an article where the author wasn't really thinking clearly or with any sort of deep perspective at all about what cameras are for, or the value of what photographs of actual messy reality deliver.
As an image production apparatus, you can replace photography by machine learning (or really any other image creation process, drawing, painting, raytracing, whatever), and in these cases yes, photography becomes a little less important.
However, a photograph is not only an image, it is a trace of actual photons that existed and imprinted a photographic material. When you look at a photograph, you're not only looking at a picture, you're also looking at the imprint of reality. It's the same emotional effect as looking at your kid's hand print on paper, or wearing your grandmother's ring. There is affect involved that comes from the reality of the experience.
A souvenir picture is not only an image that helps you remember the time when you and your friends were doing something, it's an actual fossil of that moment. And that's what matters in photography. And that's why it's not going away any time soon.
What if you replace "photos" with "music", "art", or "literature"? Sure, we will get to a point where a computer can write a symphony that fits all the characteristics of Beethoven, or write a stylistically accurate Shakespearean sonnet. But it won't have the weight of the artist's observations behind it.
> If you treat photos as pleasurable pictures, then machine-generated photos will serve fine (e.g. stock photos).
I don't know anybody personally who does that, actually its silly. Or maybe normal to people who (poorly) photoshop themselves with ferraris, luxury and whatnot. Apart from our HR and marketing department of course, they love stocks :)
"I've noticed the many photographers here, [...]. Always the same conventional eyes, noses, mouths, waxy and smooth and cold. It still always remains dead. And the painted portraits have a life of their own that comes from deep in the soul of the painter and where the machine can't go." - Van Gogh
We've been asserting souls for quite a while now and we've been wrong every time. What does the weight of the artist's observations mean? How is that different from the result of machine learning, which is nothing if not carefully considered observations.
Note: The most expensive Van Gogh paintings sell for about 2 orders of magnitude more than the most expensive photographs.
(I was actually surprised at how much the most expensive photographs sold for)
I don't expect any reasonable answer to this, but...
There are two sets of photos, both exactly alike, yet one of those sets was created by capturing photons while the other was created by computers. Both were created by pointing a machine at your kid during a trip, and subsequently giving you a digital image. How to you tell those apart?
How do you know it's your kid on your kid's photos anyway?
1. Train ride with half hour viewing time to observe the Cristo Redentor. No cameras allowed.
2. Train ride to the statue for a photo of you in front of it, but while the camera can see both you and the statue, you can't see it from your position.
3. Green screen photo of you artificially in front of the statue.
I think a lot of people would, if they thought about it, strongly prefer the first option and ascribe little value to the others. But because they do have cameras and there's an expectation that you take a picture as a tourist, they end up viewing their trip through their camera lens or smartphone screen.
I think it would be a compelling sell if the camera app on a smartphone could prompt users with an offer of a professional-quality photo or video (optionally with the user green-screened in) of a concert, sporting event, or landmark. "Studio X has a professional recording of this event from N angles with better equipment than yours. Add this video to your library instead?"
The killer use for this might be to record/sync school concerts, plays, and sports. So many parents with mediocre cameras all trying to capture the event, and no good way right now to share the output.
I was also hoping for something more ambitious, such as "once you can detect the full electromagnetic spectrum hitting your phone from any angle, you won't need a lens to reconstruct a focused picture," as mentioned by blixt.
At a minimum, I was hoping for conceptualization and theorized advancement of the current display-pixel-as-camera-pixel train of thought.
I don't disbelieve for a second that the expressiveness and realism of automatically-composed images won't have a huge explosion and become part of popular media. But I think there will be very long period during which some datasets will be much more salient than others. Think of how many orders of magnitude photos we have of dogs and cats compared to frogs and yaks... can they take the camera off of a product that can't image my friends petting a yak? I'm very curious when is enough, because the human ability to contrive absurd scenarios is frustratingly extensive, and unique "outlier" moments are exactly the ones many people want to capture.
Also I think I _have_ seen a similar art project like this, where the artist took a photo album of Paris(?) and gave people "cameras" that just identified the closest image location by GPS and would just keep regurgitating that image no matter how many you took. I can't seem to find it now...
You mean like movie computer graphics?
Our memory is fragile and our eyes easily deceived. I'm hopeful this technology is introduced to the public first, so other institutions aren't able to abuse it before most people understand how untrustworthy pictures are.
Why do you think magazines use photoshop so heavily? People don't want the hot girl spare tires. Then want the perfect hair and big boobs.
It's all fake, but it's fake that is requested and voted for with money.
The problem here is not the tool, it's what kind of society we are building.
So what's the endgame for selfie-takers?
For example: Banksy just tagged a building downtown. You rush there to get your picture in front of it. You can't take a photo of yourself with it because it's not yet in any of the databases used by your quasi-camera to stitch together photos.
It seems like the world is too variable for something like this to function practically for any but the most boring photos.
Five years later, you snap another photo of Banksy's latest, with an even lower-res camera. Two possible outcomes:
1. You've violated Banksy's copyright, whether he wanted that copyright or not, and the offending image is not present in your photo. You may or may not receive an automated warning.
2. You've recorded a crime, the authorities are notified immediately as your photo processes, and the image is again not present in your photo so as not to encourage the proliferation of graffiti.
The endgame leaves you with no camera, no one cares about Banksy who's in jail by now (with your help!), and why are you so concerned about violating copyright and encouraging crime, anyway? Why should technology enable you to do any such thing?
Maybe I could carefully explain to the software, "there's a male robin in this exact location and pose on the deck" and it could generate one, but why would I want to do that instead of just having a regular camera?
What does this do that I can't do with a camera? It sounds like a way more complicated, processing-intensive method to achieve an inferior result. It's a solution in search of a problem. It's not like decent cameras are prohibitively expensive, seeing as we've managed to include one in basically every phone now.
Most of the pictures I'd want to take are of something unique. Software may be able to figure where I'm standing, what permanent features are nearby, what the weather is like, what the people I'm with look like, but it's not going to know the transient features, like there's a lady with a weird looking hat in the background that I wanted to capture. So, what's the point?
You're right, what the author here is suggesting is not something you would want. He's imagining a magical omniscient fantasy technology, the closest approximation of which would leave you disappointed and robbed of agency as a photographer.
I propose, instead, you consider the elements he stitched together to reach his delusion and whether or not they could still lead to a very similar, regrettable future.
But that's not really "photography" (it's to photography what miniature plastic Eiffel towers are to architecture).
A much more interesting and much more futuristic approach would be a device capable of recording a whole scene without a conventional sensor and lens; something that would record photons not because it's hit by them, but because it knows where they are.
So this device would record all the light waves's position in a scene, at a given instant (the instant the image is taken), and then it would let you later reconstruct / produce any image from any position in the scene, with any kind of focus or bokeh, or whatever.
It would also let you walk into the scene like in a real "mannequin challenge", etc.
Looking at my favorite photos I've taken, there are:
- Some photos of Iran (Google, being a US company, wouldn't operate there)
- The inside of a crashed plane
- A closeup of a dying hummingbird
- A laser that I use personally
- A boat adrift in the middle of the sea
All of these moments couldn't have been taken in this method, even with a boatload of content-aware scale and other people's photos stitched together.