When generating a new person (or whatever it is that does not exist) can you know that it isn’t like any of the images that went into the training data set?
How likely is it for it to actually exist after all?
The model has 18x512 parameters, and all 9000 parameters are 32-bit floats.
Even assuming only 16 bits of randomness for each parameter (to keep them small enough, so you don't get too wild, because the center of 0 is a pretty homogenized face), 16^9000 is a lot of permuations of faces.
They will, of course, borrow from all of the 70k faces on Flickr that power StyleGAN, in varying degrees.
> Even assuming only 16 bits of randomness for each parameter
Well, this is the question. How do you know that you can make that assumption? Also even though you have ~9000 parameters they could be highly dependent.
Some of the backgrounds are surprisingly consistent, but some are distorted in ways that can only be done by a deep NN that sort-of knows what it's doing. The fact that the face is (almost always) still completely believable seems like good evidence against overfitting.
I remember it from the subproject, < http://thiscatdoesnotexist.com > although that doesn't seem to have improved. It feels more like a "this transporter accident doesn't actually exist dot com". (Just a joke; its fun.)
It has a "machine learning" based projection mode where you can project a single photo of a face on the head of a character mesh and it will create somewhat believable set of 3D features from the photograph on the model. It was fun to take a couple of the generated faces from this site and apply them to the models.
I imagine you could do a lot with just those two things if you had the talent of the artist in the video I posted above. You could create fully rigged and clothed characters for games in a fraction of the time it would take to make them entirely from scratch, and for much less expense and cleanup (I would have thought) than full body scanning.
Wow! Imagine allowing people to upload their profile picture in and RPG and suddenly their character actually looks like them. Imagine playing online with your friends and all of their characters look just like them.
StyleGAN (v2, which is this, and v1 before it) is spectacularly high quality.
Even more interestingly, you can interpolate between faces very smoothly. (For instance, I was able to give some face off the street a drag makeover, by interpolating that face plus or minus the face of a drag queen minus that performer's face out of drag: https://leebutterman.com/assets/lsb5-plus-trixie-minus-brian... )
But it can be tricky to encode faces if they look out of the ordinary, where "ordinary" is the 70k faces on Flickr that trained StyleGAN.
I'm intrigued about the possibility of using these as stock photos - it dies away with the need for model permissions (or does it?). Presumably the copyright on the image resides with the person who ... programmed the neural net? Trained the neural net? Is running the server?
I would guess the one picking the photos and training parameters would have the largest part of the copyright. Whether courts will decide others have some rights to is TBD.
The persons whose images the net was trained with also might have, certainly if the network was trained with only a few images.
How does copyright work when you have no idea the photo even exists? Are these photos dynamically generated on the fly when a page is loaded? Because if so, then the visitor is the only person to have seen that particular photo, and there'd be no way for any copyright owner to even know they might hold the copyright over that particular photo.
>I'm intrigued about the possibility of using these as stock photos
Another (darker) possibility is to generate copyrighted images. Once you have (say) a trillion photos, it will be hard to generate a new one that doesn't look (in essence) like one you already have.
There was a pic of an older guy, but what was unsettling was the woman peeking over his shoulder; like a friend or a family member, leaning into the photo.
Her eyes were melting into black and white streaks curling down over her sagging cheeks; her smile was perfectly white, and just a little too long.
I refreshed on instinct, and haven't seen anything like it since.
You have to imagine some strong AI in 2030, video chatting with people using fake human avatars, and people can sometimes tell because there's weird fleshly artifacts moving in the background.
I feel like this is "the 2nd uncanny valley", on the other side of realism. It's more uncanny because the face is so realistic, indistinguishable from a real person. But the weird mistakes around the face reminds us that this is made by something entirely alien, with an alien understanding of what it means to be human. It's like The Thing.
I've seen some of those before - the algorithm is a bit strange about people who hang around the edges of the border and sometimes create some fairly horrifying abstractions of humans
Yep, I've seen some ones like that too. The NN is quite good at knowing what people look like, but it's much worse at knowing what people tend to be next to and sometimes you get some horrors.
An artist friend of mine is using this technology in an interesting way .. she's drawing regular street portraits of people she meets, and then using that as the seed to generate new pictures of people who do not exist.
It has produced some extraordinarily disturbing art, especially for those of us who were her subjects. Some of the images are so wild and out there, it really is an adventure into ones own psyche. I see so much in the generated images that creates an intense emotional response, and its very difficult to differentiate between the lines of the artist and the lines of the code, at least in a way that is easily discernible. Her art as the seed definitely amplifies things - the emotions of eyes, the despair of cheeks and lips - in a way that the generative programming enhances, 1000x ..
Very interesting stuff, and I'm looking forward to her exhibit of this work - especially the live demonstrations.
What is the likelihood that the generated person does exist?
I got the feeling this could turn out the same as people using fake email addresses to later discover that the domain in fact exists (that's why you should always use example.com).
Good question. I would guess it's very low if we're talking exact match. It would have to match N features. There's effectively an infinite number of combinations, many more than the 8 billion people curently living.
In reality, there are already real-world doppelgangers, where a typical observer would confuse two (unrelated) individuals. I'm guessing that could happen with a synthetic image too, but it's probably not any worse than a real-world matching error. (The bigger concern is the related one of deepfakes, which don't need any kind of synthesis, they just act directly on the real person's image.)
Now, the next obvious question is, how do you pick one of these non-existing persons and generate images of him/her doing different things in different places? (Or, of course, real persons. Is there a service where I can upload my photo and it starts to generate images of me doing random - or not so random - things?)
You can reverse-engineer faces into the original random seed/embedding, and then tweak the embedding to 'edit' the face. (There are a bunch of tools and Colab notebooks for that, but the best tool is definitely Artbreeder, which does a lot of other GAN models too: https://artbreeder.com/ )
However, since the model only does faces centered in an image, you're restricted to different headshots: make them smile or frown, or wearing sunglasses, or different hair color, or animate lip flaps, yes, but not 'doing different things in different places', unless you can figure out some way to insert a headshot into a bigger photo.
The background, clothing and especially other people in the image are still a dead giveaway that it's not a real person. It's kind of weird how the algorithm borks something relatively simple as the texture of clothing.
I don't know but I imagine somebody might want to generate an algorithm that puts random clothes on people. There's so much money going on in the fashion industry and beautifying. If you can see how a new shirt fits on an existing person you can also fit a shirt on a fake person right?
To my mind there is a massive need for us in the software world to pause and think about the ethics of all of this. Engineers more broadly have well developed ethical codes, we should too.
So are you saying software developers should have to take the equivalent of a Bar exam in order to practice software development in a given state? That way ethical code violations have teeth because you can be debarred.
As of now anyone and their dog can practice software development and I'm not sure what a nebulous "ethical code" would imply for them.
Arguably software engineering compared to engineering in a broader sense has a bigger issue with the “we did this because we could, not because we should” attitude.
I feel like a “formalised ethical code” would result in more clever approaches of avoiding it. We don’t need to look far to find the examples (looking at ya ad tech)
The entire point of a code of ethics is to provide a strong minimum standard to keep the morally bankrupt in line. We don't write them for moral people.
I think it would have to start with either regulation or customers requiring it.
I.e., contractually require someone licensed (where the license carries an ethic standard) or legally require it in areas that are 'in the public interest'
I've often wondered if software engineers would benefit from the accountability of licensing like other engineering disciplines, especially as it pertains to safety critical systems.
It's a bit shocking for me to know a mechanical, electrical, or civil design in, say, a power plant may require a Professional Engineer to officially stamp it but any ol' software engineer can write code on that system and fire it up. My understanding is that NCEES lists a controls systems license but I have yet to meet a single person with it or hear of a project that required it.
To a certain extent, I wonder if software development rigor/oversight is in an adolescence similar to what it must have been like for mechanical systems during the early parts of the industrial revolution.
This was a big debate in the 90s. The "industry" headed it off at the gates (partly thanks to communication media such as ... usenet :) It might resurface, but I expect to hear the same arguments used against it, all over again. I'd be suprised if it ever came to pass in the same sense as PE requirements.
1) programming is more like an art than a science, and so should not be regulated like engineering
2) the capital/labor requirements for programming are minimal - basically anyone could do it - so we should not worry about trying to license something that people could (and do) do at home.
3) there's no agreement about what would be required to get a license.
4) free software would die
Of course, some of these really do not hold up to close inspection. It's true that there is an "art" to programming, but not much more than there is in architecture for example, and there's no reason you couldn't require a license for cases where you do work for others rather than yourself. Since the 90s, it's become a _little_ more clear what the components of a s/w dev license might be, but perhaps still not enough to really pin it down. And yes, it would probably kill the free software world.
I wonder if they could parallel other engineering disciplines and give exemptions for specific areas or other avenues to ease the burden.
For example, as long as a certain percentage of principal engineers are licensed, others within the company don’t have to be. Similar with working under a licensed lead or specific article industry exemptions.
It would also give some power to the developers over management in key decisions. I’ve seen managers try to back engineers into a corner and when the engineer ultimately said they wouldn’t sign off on a design, management backed down because they didn’t have an equivalent to “just ship it now” and get it stamped later.
I understand the fear of it, but sometimes it seems like the sw development is run like the Wild West.
That's because other than the name "software engineering" has nothing to do with "engineering", as in a prescribed process and set of standards and practices that are required to be followed in the design and implementation of projects.
Probably something more like a PE. From the NSPE website a PE entails:
"To become licensed, engineers must complete a four-year college degree, work under a Professional Engineer for at least four years, pass two intensive competency exams and earn a license from their state's licensure board. Then, to retain their licenses, PEs must continually maintain and improve their skills throughout their careers."
Most PEs that I have worked with are very diligent. You don't get yahoos going that far. Generally, they are top-notch engineers. Think '10X' engineers, but for oil wells and churches. I mean, look at any large structure on Wikipedia, most of the time you'll find the engineer's name in the side placard, not just the firm's name. PEs are serious, dedicated, and smart people. I mean, their stamp is on those documents and they go to jail when the bridge fails.
So, for software, if you have an exam of the PE's caliber, you essentially guarantee '10X' engineers. The workforce is more highly paid, yes, but it is of a much higher quality. Bugs get solved faster, stand-ups don't last 30 minutes, the wiki is updated by everyone, etc. Employers know that their candidates really are good stuff, less whiteboards or take-homes are needed. Also, possibly when something messes up, the engineer is the one responsible, not the company. So offloading the insurance to the employees is not the worst thing for a company.
Honestly, it would not be a bad system and I think would be a much calmer and better one.
i'm seeing some sources that say it was discontinued after 2019 due to lack of interest, and the main "PE" page no longer links to the software PE exam.
That would be a horrible idea. Software developers should be encouraged to think about and discuss ethical issues by other software developers, employers, universities, and people in general.
We need increased awareness of ethics but we don't need more government involvement or any other form of coercion. Governments are already far less ethical then pretty much any other organisation or institution.
This "government involvement is bad" is a crock. Regulation is what makes a society. As an institution, they are no less "ethical" than corporations, in fact, neither ethics nor morals are relevant to corporate or government bodies.
Morals and ethics are about people. Laws and regulation are about how those morals and ethics are reduced to practice.
Voluntary cooperation and peaceful competition is what makes a society. A government is just a group of people who attempt to maintain a monopoly on violence and use it to enforce their will on others. I certainly think they are less ethical then people who do not attempt to enforce their will on others.
Government created laws and regulation are just threats. They tell you nothing about what is right or ethical. They only tell you how a particular group of people want you to behave and what they will try to do to you if you refuse to obey. Anyone with enough capacity for violence and willingness to use it can create a law but no amount of force can change what is right or wrong.
Copyright 1999. I was taught ethics in my college work, so clearly everyone else just ignores it like they do for the various other fields where there is an ethics code. People suck.
The issue is that there's no pressure on employers to require it, so there's no advantage on e.g. employability or salary from getting these codes.
I live in the UK and was considering going through the British Computer Society certification process years ago, but dropped it, because while I do care about the ethical side etc. the additional amount of effort it'd take to document skills I already had was not worth it when prospective employers in the fields I've worked in just did not care.
Agreed on taking a collective breath and evaluating.
For "real" engineers, is it fair to say their ethical questions are more easily constrained than those in software? I assume it usually reduces to "should I accept a lower-than-average safety margin to save money on a part/structure that might fail and hurt people?" Maybe "should I make this vehicle accommodate extremely tall people?" or "can a child hurt themselves with this device?"
Honestly lawyers feel like the closest analogue to software engineers.
> For "real" engineers, is it fair to say their ethical questions are more easily constrained than those in software?
For capital "E" Engineers (i.e., those with a State license), I would say yes. They are required to abide by specific ethics and some states require regular ethics training. If they certifying a design, they are putting their name/license/livelihood on the line that it meets a minimum standard. This standard is usually defined by specific codes.
The problem I often see in software is that many codes (if they even exist) are looked at as a "nice to have" not a "must have", unless they are contractually obligated. If I show a code-base doesn't adhere to a specific industry standard, the reaction is often "meh". This is in contrast to other engineering disciplines who tend to begrudgingly acknowledge they have to re-design to meet such a standard.
The issue with some of your examples is that much of engineering work happens under an "industrial exemption" meaning they don't have to be a licensed engineer.
The Free Software Foundation has been promoting a well-developed ethical code for software engineering since the 1980s: https://www.gnu.org/gnu/manifesto.html
It covers most of the widely debated issues in software today, including spam, malware, DRM, invasion of privacy, undue influence by one participant in the ecosystem, copyright infringement, and bait-and-switch terms of service. It even addresses the question of writing software that could be misused.
It isn't widely accepted because a lot of programmers don't like it. They don't like it because they make money by violating it.
This has improved so much since the last time I checked it.
It's interesting to see so high quality on image generation using neural networks, while text generation using gpt2, which seems a lot easier at first, is glorified nonsense.
Is part of it deliberate avoidance of women with both ears + earrings showing? That was one of the tells before...one earring or a mismatched pair. Now the women mostly seem to be in poses where you can only see one ear, or no ears (hair covering them).
There is, this time, much less craziness with hair sprouting from the wrong places.
In addition to the background, if you can see earrings, that is usually a giveaway. The generated faces seems to always have different earrings in each ear. Also glasses disappear into the skin, and ears are often different on each side and/or distorted.
It only took me a while to notice that this is an actual human face in the background, just photographed from a peculiar angle and bearing a peculiar expression - not a neural network artifact.
184 comments
[ 4.4 ms ] story [ 239 ms ] threadWhen generating a new person (or whatever it is that does not exist) can you know that it isn’t like any of the images that went into the training data set?
How likely is it for it to actually exist after all?
Even assuming only 16 bits of randomness for each parameter (to keep them small enough, so you don't get too wild, because the center of 0 is a pretty homogenized face), 16^9000 is a lot of permuations of faces.
They will, of course, borrow from all of the 70k faces on Flickr that power StyleGAN, in varying degrees.
Well, this is the question. How do you know that you can make that assumption? Also even though you have ~9000 parameters they could be highly dependent.
Is it not possible, or just incredibly unlikely?
I also wonder how similar to an original person it needs to be before you would believe it's them anyway.
https://news.ycombinator.com/item?id=19144280
(872 points, 242 comments)
The results are much better than when originally posted in HN.
[1] - https://arxiv.org/abs/1912.04958
https://www.youtube.com/watch?v=tnviGWO0wbU
It has a "machine learning" based projection mode where you can project a single photo of a face on the head of a character mesh and it will create somewhat believable set of 3D features from the photograph on the model. It was fun to take a couple of the generated faces from this site and apply them to the models.
I imagine you could do a lot with just those two things if you had the talent of the artist in the video I posted above. You could create fully rigged and clothed characters for games in a fraction of the time it would take to make them entirely from scratch, and for much less expense and cleanup (I would have thought) than full body scanning.
Really Cool! But also slightly weird.
Even more interestingly, you can interpolate between faces very smoothly. (For instance, I was able to give some face off the street a drag makeover, by interpolating that face plus or minus the face of a drag queen minus that performer's face out of drag: https://leebutterman.com/assets/lsb5-plus-trixie-minus-brian... )
But it can be tricky to encode faces if they look out of the ordinary, where "ordinary" is the 70k faces on Flickr that trained StyleGAN.
The persons whose images the net was trained with also might have, certainly if the network was trained with only a few images.
Another (darker) possibility is to generate copyrighted images. Once you have (say) a trillion photos, it will be hard to generate a new one that doesn't look (in essence) like one you already have.
Her eyes were melting into black and white streaks curling down over her sagging cheeks; her smile was perfectly white, and just a little too long.
I refreshed on instinct, and haven't seen anything like it since.
https://i.imgur.com/RhROFSf.jpg
https://i.imgur.com/NTC2ZiN.jpg
http://www.whichfaceisreal.com/fakeimages/image-2019-02-18_2...
Imagine seeing your lookalike in here.
It has produced some extraordinarily disturbing art, especially for those of us who were her subjects. Some of the images are so wild and out there, it really is an adventure into ones own psyche. I see so much in the generated images that creates an intense emotional response, and its very difficult to differentiate between the lines of the artist and the lines of the code, at least in a way that is easily discernible. Her art as the seed definitely amplifies things - the emotions of eyes, the despair of cheeks and lips - in a way that the generative programming enhances, 1000x ..
Very interesting stuff, and I'm looking forward to her exhibit of this work - especially the live demonstrations.
[1] https://github.com/NVIDIA/nvidia-docker
I got the feeling this could turn out the same as people using fake email addresses to later discover that the domain in fact exists (that's why you should always use example.com).
In reality, there are already real-world doppelgangers, where a typical observer would confuse two (unrelated) individuals. I'm guessing that could happen with a synthetic image too, but it's probably not any worse than a real-world matching error. (The bigger concern is the related one of deepfakes, which don't need any kind of synthesis, they just act directly on the real person's image.)
There's no imagination involved.
However, since the model only does faces centered in an image, you're restricted to different headshots: make them smile or frown, or wearing sunglasses, or different hair color, or animate lip flaps, yes, but not 'doing different things in different places', unless you can figure out some way to insert a headshot into a bigger photo.
To my mind there is a massive need for us in the software world to pause and think about the ethics of all of this. Engineers more broadly have well developed ethical codes, we should too.
As of now anyone and their dog can practice software development and I'm not sure what a nebulous "ethical code" would imply for them.
I feel like a “formalised ethical code” would result in more clever approaches of avoiding it. We don’t need to look far to find the examples (looking at ya ad tech)
I.e., contractually require someone licensed (where the license carries an ethic standard) or legally require it in areas that are 'in the public interest'
It's a bit shocking for me to know a mechanical, electrical, or civil design in, say, a power plant may require a Professional Engineer to officially stamp it but any ol' software engineer can write code on that system and fire it up. My understanding is that NCEES lists a controls systems license but I have yet to meet a single person with it or hear of a project that required it.
To a certain extent, I wonder if software development rigor/oversight is in an adolescence similar to what it must have been like for mechanical systems during the early parts of the industrial revolution.
1) programming is more like an art than a science, and so should not be regulated like engineering
2) the capital/labor requirements for programming are minimal - basically anyone could do it - so we should not worry about trying to license something that people could (and do) do at home.
3) there's no agreement about what would be required to get a license.
4) free software would die
Of course, some of these really do not hold up to close inspection. It's true that there is an "art" to programming, but not much more than there is in architecture for example, and there's no reason you couldn't require a license for cases where you do work for others rather than yourself. Since the 90s, it's become a _little_ more clear what the components of a s/w dev license might be, but perhaps still not enough to really pin it down. And yes, it would probably kill the free software world.
For example, as long as a certain percentage of principal engineers are licensed, others within the company don’t have to be. Similar with working under a licensed lead or specific article industry exemptions.
It would also give some power to the developers over management in key decisions. I’ve seen managers try to back engineers into a corner and when the engineer ultimately said they wouldn’t sign off on a design, management backed down because they didn’t have an equivalent to “just ship it now” and get it stamped later.
I understand the fear of it, but sometimes it seems like the sw development is run like the Wild West.
"To become licensed, engineers must complete a four-year college degree, work under a Professional Engineer for at least four years, pass two intensive competency exams and earn a license from their state's licensure board. Then, to retain their licenses, PEs must continually maintain and improve their skills throughout their careers."
Most PEs that I have worked with are very diligent. You don't get yahoos going that far. Generally, they are top-notch engineers. Think '10X' engineers, but for oil wells and churches. I mean, look at any large structure on Wikipedia, most of the time you'll find the engineer's name in the side placard, not just the firm's name. PEs are serious, dedicated, and smart people. I mean, their stamp is on those documents and they go to jail when the bridge fails.
So, for software, if you have an exam of the PE's caliber, you essentially guarantee '10X' engineers. The workforce is more highly paid, yes, but it is of a much higher quality. Bugs get solved faster, stand-ups don't last 30 minutes, the wiki is updated by everyone, etc. Employers know that their candidates really are good stuff, less whiteboards or take-homes are needed. Also, possibly when something messes up, the engineer is the one responsible, not the company. So offloading the insurance to the employees is not the worst thing for a company.
Honestly, it would not be a bad system and I think would be a much calmer and better one.
https://ncees.org/engineering/pe/software/
i'm seeing some sources that say it was discontinued after 2019 due to lack of interest, and the main "PE" page no longer links to the software PE exam.
We need increased awareness of ethics but we don't need more government involvement or any other form of coercion. Governments are already far less ethical then pretty much any other organisation or institution.
Morals and ethics are about people. Laws and regulation are about how those morals and ethics are reduced to practice.
Government created laws and regulation are just threats. They tell you nothing about what is right or ethical. They only tell you how a particular group of people want you to behave and what they will try to do to you if you refuse to obey. Anyone with enough capacity for violence and willingness to use it can create a law but no amount of force can change what is right or wrong.
https://www.computer.org/education/code-of-ethics
Copyright 1999. I was taught ethics in my college work, so clearly everyone else just ignores it like they do for the various other fields where there is an ethics code. People suck.
I live in the UK and was considering going through the British Computer Society certification process years ago, but dropped it, because while I do care about the ethical side etc. the additional amount of effort it'd take to document skills I already had was not worth it when prospective employers in the fields I've worked in just did not care.
For "real" engineers, is it fair to say their ethical questions are more easily constrained than those in software? I assume it usually reduces to "should I accept a lower-than-average safety margin to save money on a part/structure that might fail and hurt people?" Maybe "should I make this vehicle accommodate extremely tall people?" or "can a child hurt themselves with this device?"
Honestly lawyers feel like the closest analogue to software engineers.
For capital "E" Engineers (i.e., those with a State license), I would say yes. They are required to abide by specific ethics and some states require regular ethics training. If they certifying a design, they are putting their name/license/livelihood on the line that it meets a minimum standard. This standard is usually defined by specific codes.
The problem I often see in software is that many codes (if they even exist) are looked at as a "nice to have" not a "must have", unless they are contractually obligated. If I show a code-base doesn't adhere to a specific industry standard, the reaction is often "meh". This is in contrast to other engineering disciplines who tend to begrudgingly acknowledge they have to re-design to meet such a standard.
The issue with some of your examples is that much of engineering work happens under an "industrial exemption" meaning they don't have to be a licensed engineer.
Consider https://www.gnu.org/philosophy/the-root-of-this-problem.html and https://www.gnu.org/philosophy/freedom-or-copyright.html as recent entries.
It covers most of the widely debated issues in software today, including spam, malware, DRM, invasion of privacy, undue influence by one participant in the ecosystem, copyright infringement, and bait-and-switch terms of service. It even addresses the question of writing software that could be misused.
It isn't widely accepted because a lot of programmers don't like it. They don't like it because they make money by violating it.
It's interesting to see so high quality on image generation using neural networks, while text generation using gpt2, which seems a lot easier at first, is glorified nonsense.
There is, this time, much less craziness with hair sprouting from the wrong places.
http://whichfaceisreal.com/
The correct image was also on the left side all 5 times, which feels a little suspicious (although obviously not super unlikely).
It only took me a while to notice that this is an actual human face in the background, just photographed from a peculiar angle and bearing a peculiar expression - not a neural network artifact.