This whole research domain is confusing to me. I do get that it's a hard and cool problem to solve and I do see why researcher could work on it on those grounds.
But I'm concerned about what solving this is useful for, especially with the focus on generating human faces. Obligatory Jurassic Park quote: "Your scientists were so preoccupied with whether they could, they didn't stop to think if they should.".
I can imagine this technology to be used in the entertainment industry (movies, video games) as a way to save money on art. But it's quite obvious that this technology can be weaponized and used in the propaganda industry (if it isn't already).
So, this could save money in fun applications on one end, and provide means for mass opinion manipulation weapon on the other end. I am certainly biased but I'm not sure the recreative use case is worth the risk of the weaponized one (a bit like using tactical nukes for a firework show).
What am I missing then? Are there any other use cases that would make this technology actually desirable for the greater good or humanity, or is it just a weapon with some potential recreational use cases?
If it's "just code" someone somewhere is going to do it eventually, restrictions notwithstanding. Might as well do it in the open and publish about it so the problem space and its implications are (at least potentially) the object of further research.
Oversimplification: Jurassic Park is about the lack of robust peer-review before going to prod!
I don't think of this as "we really wanted the ability to synthesize faces, this was very important to us as a civilization."
Rather, I think of it as "if we create an AGI, it will have required the ability to imagine things. Also, we have huge datasets of faces that all sorta point the right way and have a good medium of feature complexity, and we're extremely specialized in spotting visual errors in faces."
Consider the similar if slightly smaller research in creating photos of living rooms. (Or OpenAI's famous "comfy chair in the shape of an avocado.") Same reasoning. It's not about faces, faces just have beneficial properties.
So you're not terminator-worried, just jurrasic park-worried then. (that was what my question was about)
>I'm concerned about what solving this is useful for
As for all research, it is pretty hard to predict what and what for something will be useful (despite researchers being pushed to say stuff like "my research is going to cure cancer" to get funding). But about neural networks for image processing, they certainly start to be useful for radiology.
Consider the case of how to create a very strong facial recognition system, without relying on photos of actual people. This system provides a data generation capability suitable to produce a training set with billions of "people", none of whom have had their privacy wounded by having their specific face in the training set.
Yea I mean the most eminent applications for this stuff is, putting unconsenting peoples likeness into porn, black mail/framing people, facial recognition, deceptive propaganda(because it's not deceptive enough already), weakening criminal evidence, etc. Then comes stuff like cgi for movies/games. Seeing a lot more of the former use cases not much of the latter. Pretty sure most people doing modern cv don't really realize where it's going, and people aren't publishing on the hazards so a lot of the criminality is and will remain novel until someone works it out..
So short sighted. A strong facial recognition system can locate lost people in a wilderness from drone footage. A strong facial recognition system can discriminate between twins, near age siblings, and ethnically similar but unrelated individuals. A strong facial recognition system retains reliability when test imagery is poor quality. A strong facial recognition system can identify kidnapped children being trafficked, even after they have aged. A strong facial recognition system compensates for the complete lack of quality operator training within the facial recognition industry.
I guess the non-creepy application I'm seeing for the future of this research is the seamless conversion of a 2D image into 3D for when 3D projectors become more commonplace and perhaps overtake the ubiquity of flatscreens on our devices. And I am not sure, but it might also be applicable to the video-cam issue where we want to make it look as though the person we're having a video conference with is looking at us instead of the screen image of us and vice-versa.
> I do get that it's a hard and cool problem to solve and I do see why researcher could work on it on those grounds.
There are quite a lot of harder and potentially even more useful if solved problems that receive significantly less attention and funding in this domain. Vast majority of GAN papers nowadays are just different applications of them, without significant contributions to underlying theoretical foundations.
Didn’t Facebook just get sued for illegally harvesting faces & tags of people? If derivatives of that data make it into research like this, where does that stand from a legal perspective?
It's only unsettling if you are completely unfamiliar with the machine learning field. FAIR publishes research in many ML/AI fields. They're responsible for plenty of advances and contributions.
While watching it, I wondered whether the "this x does not exist," GAN sites also implies a subset of "this x must exist" and if looking at it that way could yield discoveries in other areas. e.g. This X must exist, where x was a solar system, protein, wave function, chemical substance, cell, transcendental number, etc. Naive view, but when you're extapolating from a field of arbitrary 2D data, it seems there are a lot of ladders you could climb with that.
I see there are a lot of questions/doubts in the comments about the usefulness of such research. This type of research is the precursor to gaining the ability to generative photorealistic 3D worlds. What is that useful for? Anywhere we currently generate 3D environments, including movies, games, VR/AR.
26 comments
[ 3.2 ms ] story [ 67.2 ms ] threadBut I'm concerned about what solving this is useful for, especially with the focus on generating human faces. Obligatory Jurassic Park quote: "Your scientists were so preoccupied with whether they could, they didn't stop to think if they should.".
I can imagine this technology to be used in the entertainment industry (movies, video games) as a way to save money on art. But it's quite obvious that this technology can be weaponized and used in the propaganda industry (if it isn't already).
So, this could save money in fun applications on one end, and provide means for mass opinion manipulation weapon on the other end. I am certainly biased but I'm not sure the recreative use case is worth the risk of the weaponized one (a bit like using tactical nukes for a firework show).
What am I missing then? Are there any other use cases that would make this technology actually desirable for the greater good or humanity, or is it just a weapon with some potential recreational use cases?
Rather, I think of it as "if we create an AGI, it will have required the ability to imagine things. Also, we have huge datasets of faces that all sorta point the right way and have a good medium of feature complexity, and we're extremely specialized in spotting visual errors in faces."
Consider the similar if slightly smaller research in creating photos of living rooms. (Or OpenAI's famous "comfy chair in the shape of an avocado.") Same reasoning. It's not about faces, faces just have beneficial properties.
>I'm concerned about what solving this is useful for
As for all research, it is pretty hard to predict what and what for something will be useful (despite researchers being pushed to say stuff like "my research is going to cure cancer" to get funding). But about neural networks for image processing, they certainly start to be useful for radiology.
There are quite a lot of harder and potentially even more useful if solved problems that receive significantly less attention and funding in this domain. Vast majority of GAN papers nowadays are just different applications of them, without significant contributions to underlying theoretical foundations.
1. They generate a low-resolution position map using NeRF => Good camera and 3D control
2. They then upsample that with StyleGAN2 => Good features in the final 2D image.
So in effect, this is a StyleGAN2 network where the latent space has the 3D-related components split off into a different generator network.
it's not a nerf based solution but since the goal is high resolution image syntesis..
That said, the results are actually very impressive.