Could you please stop posting unsubstantive comments and flamebait? You've unfortunately been doing it repeatedly. It's not what this site is for, and destroys what it is for.
Safe always implies avoiding a damage; which one is defined case by case.
I've always interpreted the word 'safe' in NSFW as avoiding you to lose your job if caught by your boss.
There's nothing else inherently wrong or unsafe in nudity.
There's no secrets here, just IP addresses and hostnames, so there's no real security issue. If you want to report it to the author you can try to message them or create an issue.
The Nightwatch was controversial because of its unconventional composition and departure from traditional Dutch painting conventions.
His “Bathsheba at Her Bath” caused trouble too. Hendrickje received three summonses from the Reformed Church to answer the charge "that she had committed the acts of a whore with Rembrandt the painter". She admitted her guilt and was banned from receiving communion.
It's a gradient. On one end it's invisible. On the other end it literally flays the flesh off your bones with offensiveness. These filters that we're creating are a tool for tuning the feed.
I always assumed that image generation services had prompt and output filtering. The reason I believe this is that I've crafted prompts that seemed to pass the prompt filter, generated, and then were unable to be displayed[1]. Presumably there is an output filter that determines if the generation violates ToS against things like nudity. Though in my case, I wasn't able to generate images that too closely resembled Ansel Adams photographs. Shame on me, I guess.
1. This reads a little like being able to deduce what's going on by studying the timing, now that I reflect on it.
Yes bing image creator uses two different levels of filters, one for the prompt and one for the output. gpt image creator does the same thing, as does the dalle-3 API
I had similar experience with OpenAI GPT API,
following their rules we check the customer prompts with their endpoint for "safety" , then if it is safe we continue and generate the content. But sometimes even if the prompt was "safe" the result would be blocked as unsafe, because the AI either generated something "unsafe" from a safe prompt or the output filter had a false positive. Of course we had to pay for something that was not our fault.
So it is clear that are many filters involved, some input filter, then the AI is "aligned" and threats you as a child , then there is an output filter.
Beige color clothes are to be avoided if any of the people in the image are young adults or younger. The input prompt filter correctly understands that the user isn't trying to generate anything "unsafe", but the output filter sees clothes that are too close to Caucasian flesh tone on someone younger than middle age and flags the output. Leave everything as-is but change the clothing color to something darker or leave the color alone and exclude children and young adults, then things come out fine. Though I can't see the flagged images, it seems clear that they're not generated nudes but false positives.
The root cause is that the text encoders (e.g. CLIPText) used to power image generation models are too good and can handle semantic synonyms, even oblique synonyms such as "Mickey Mouse" and "a famous anthropomorphic mouse mascot".
The repo mentioned focuses on NSFW content, but I've had an idea that can theoretically identify arbitrary cases that could be undesirable by image generation services: get a large dataset of CLIPText encoded texts, and train a LLM to take in said embeddings and output synonyms, or maybe something with contrastive loss like CLIP itself.
The latter would be preferred by most, and is most aligned with individualism, but sadly that is not where we are as a society. There are an entire class of people who believe they know what is best for you, about nearly every subject matter, and wish to limit all aspects of technology, life, politics, etc, to suit their superior beliefs.
Hardly surprising. When childhood mortality is high people grow up seeing their siblings and friends die as a common thing, and understand it could just as easily be them. With low expectation of survival, life is less intrinsically valuable and risk avoidance is not critical. Change things around such that everyone has a very good chance of living a long, healthy, prosperous life so long as they don't fuck it up, and risk aversion skyrockets. It's easy to forget that the generation currently in power in most industrialized nations is the first in human history to be raised thinking survival to adulthood was a given.
Alternatively, would you rather a world with 80% functionality safely, or a world with 100% functionality with people you don’t trust doing malicious things?
After the past few years… anyone who can seriously complain about this seems like their identity is wrapped up in not admitting that the worst “misinformation” didn’t come from bad apples - but official sources.
Interesting question. We developed the whole concept of justice because this kind of abstract utilitarian morality doesn’t apply well. I’m all-in on justice. Give me the full model. I’m not doing anything wrong.
>would you rather a world with 80% functionality with people you don’t trust doing malicious things, or a world with 100% functionality with people you don’t trust doing malicious things?
I'd definitely prefer the latter. The damage that bad actors can do is often extremely limited, while the potential benefits of more capability are boundless.
That's why my friends and I built the Align API (https://alignapi.com) to detect NSFW, Sensitive Prompts etc. It's such a widespread issue and can get you kicked out from the App Store/Play Store/Open AI API etc
Maybe the ability to generate naked images of anyone will change how we view nudity. Because while that seems difficult, realistically its more doable than putting this genie back in the bottle. People are horny, people are curious, and the result is predictable.
Let's focus on trying to raise a mentally healthier generation and take mental illness more seriously, rather than banning / getting rid of all the things that might cause someone (let alone a teenager) to commit suicide (an infinite set).
Boiling the ocean type solutions are the same as doing nothing. A better solution would be to make creating such things very illegal and bring down the same wrath we do for CSAM.
There is no amount of "mental health" that can protect you against targeted bullying of that magnitude. I really wish we had a universe simulator so you could see how miserably you would fail to survive an environment like that.
1. Complete loss of privacy and control over your image.
2. Shame and embarrassment, especially in high school where every boy will look at you different and reduce you to a sexual object.
3. Public humiliation -- those photos can and do lead to ridicule and bullying. You're the one who is naked while everyone else is clothed.
4. Destroys self-esteem -- high school is already terrible in this regard but it's hard to have any kind of self worth when a sex object and some porn is how your classmates know you.
5. Takes away your agency -- its doesn't matter if the photos aren't real, they can make you do anything in those photos and to a 3rd party viewer there's no difference.
6. It's degrading -- they're not gonna be tasteful nudes and pretending as such is naive. And it doesn't matter if you don't find the acts degrading when everyone else does.
It's not making nudity illegal, it's making and distributing sexually explicit images of a person whiteout their consent illegal.
Some bullies make people feel shameful for getting an `A' in highschool. The humiliation and self-esteem happens. If someone succeeds at a high level others try to bring them down.
The answer isn't to make getting an 'A' illegal.
The answer is to challenge the bullying and to not to allow it to gain power.
Banning non-consensual nude imagery would "fix" that "broken system". The "game" would be fixed, if those games would be made illegal. So why don't you like that as a solution? Is it supposed to be just "hated" without actually being fixed?
You're conflating 'nudity' in general, and non-consensual generated nude images, but like, these things are just different, and are treated differently. There isn't a misunderstanding that nudity in, say, art, is different from nudity in porn, or that there's difference between consensual and nonconsensual depictions, or whether something involves made-up characters and real people. So why insist on conflating these two just for the sake of 'throwing hands' like problems there can't be solved, and that some specific, more narrow set of problems can't have a solution on it's own, when it can.
I was trying to convey a more extreme holistic situation where there simply isn't a consequence of someone sharing a nude picture of someone, people just shrugging and saying "so what? It's perfectly normal to be naked".
By breaking the underlying system of shame and embarrassment of someone being nude, there isn't anything to gain from it after that. Then there isn't a reason to even make it illegal anymore.
(Other than just not wanting your person to be pictured at all, just basic privacy, no need to put different value to different parts of the body)
and it's not making nudity illegal, it's about nonconsensual creation and distribution of images with nudity. which may be connected with bullying, blackmail, extortion, or may not be.
>If a naked picture of you is causing you to choose suicide
really great response. almost as good as 'if rape is causing you to choose suicide'. 'maybe we need to fix how society views rape'. nonconsensual nude imagery and pornography is much closer to rape than not.
The universe simulator in question would deprive you of your memories and life experience. For all you know while you're in it you're a teenage girl with all the trappings and insecurities of being raised as such, you always have been and this is your whole and only world and future you're watching fall apart.
Like yeah I'm not at all shocked the HN everyman mid 30s white dude wouldn't really care about his nudes being spread around. The point of the simulator comment was I wish you could truly experience how different life is for women.
At some point this “universe simulator” concept is nonsense because if I don’t retain some aspect of my own personality and mental capacity, it’s not me in the simulator anymore. For all I know, maybe I’m a sentient mushroom from Alpha Centauri except I’m in a “universe simulator” where I am experiencing the life of a human being here on Earth having arguments on Hacker News. Maybe when I come out of the universe simulator I will have a greater empathy for your species.
The hypothetical risk of AI-generated deepfake nudes is, in principle, no different from what you could do with Photoshop even when I was a teenager. It’s easier to send the fake nudes to everybody these days, but I seriously doubt anybody would have been genuinely fooled and it would reflect more on whoever made and disseminated the fakes.
We've banned this account for repeatedly breaking the site guidelines.
If you don't want to be banned, you're welcome to email hn@ycombinator.com and give us reason to believe that you'll follow the rules in the future. They're here: https://news.ycombinator.com/newsguidelines.html.
Perhaps we need to spend some time working on the mental health of those who bully others. At the moment, we seem to, mostly, turn a blind eye to it, or use “zero tolerance” policies (which, IMO, are even more harmful to the victims than the bullying) to act like we’ve solved the problem.
I don’t pretend to have the answer, but admitting that there’s a problem in the first place is a good step forward.
“Taking mental illness more seriously” is a vague and double edged sword. In the sense that we have been taking mental health more seriously, we have only made it worse, especially with children and teens. It’s normal to feel down or to be anxious sometimes; a lot of kids will pathologize these feelings rather than treating them as ordinary life obstacles to overcome. I agree with the goal of helping kids be more emotionally resilient but I don’t think the mental health framing is helpful in this regard.
Well, emotional resilience is mental health. I think the problem with mental health framing is label obsession, rather than thinking of mental health like physical health: something to be maintained through healthy habits.
If you want to be physically healthy, you eat nutritious, balanced foods and get regular exercise and sufficient sleep.
If you want to be mentally healthy, you practice healthy ways to experience and work through your negative feelings rather than trying to avoid or control them. Etc.
Like bullying, there's a fix for it, but schools (especially public schools) typically lack the resources, capabilities, and incentives to do anything about it.
I mean I'm sure we could launch more research into what might possibly work but we do have some to get started with. It's not like ONLY non-verbal people are being bullied and can('t) talk about it.
Removing the kid gloves when dealing with bullies. Mandatory mental health resources and support, along with fines proportional to wealth for the parents. Basically here's everything you need to curb your kids bullying. If you can't, it's going to cost you.
This is a justifiable worry, but there are two reasons we shouldn't accept it:
1. If there are easy nudes of _everyone_ then there are no nudes of anyone. Admittedly, this is going to take a bit to culturally sink in, but eventually a generated nude will have very little value outside of a perverts own personal desires.
2. "Think of the children" has always been a very poor argument used to artificially restrict the freedoms of everyone else. The problem is that what you're worried about is already possible. It happens less often, sure, but if someone is really nefarious they can make it happen without too much effort.
We don’t actually have numbers on how many people ended their life for being unable to generate porn they want.
Before calling this idea absurd or too far fetched (which it likely is), think about those who ought to end their life because of pics they don’t want and compare the fetch distances.
It’s amusing how society tends to ignore a thousand miles leading to an unfortunate event and only focuses on the last one. This way of thinking leaves everyone suffering right on the edge without ever addressing a whole mountain of issues on which that edge situates.
It's extra amusing (read: horrifying) how moralists use children to manipulate people and bypass their sense of reason to go straight for the monkeybrain.
As a society, we need to call this out and shut it down as soon as it arises. Preferably whilst shutting down the people and organizations who do it.
This is like saying it’s no big deal for everyone to carry around a machine gun because humans have been stabbing each other for centuries. Scale and lowered skill barriers are transformative: you’d have to have a pretty epic middle school grudge to spend hundreds of hours learning Photoshop well enough to produce equally realistic fakes, whereas there are now apps which would allow anyone with an iPhone to spend a few minutes getting the same result.
That matters because once you realize the false dichotomy you can discuss intermediate steps: for example, if businesses have legal requirements to watermark output, answer legal requests to identify the user who produced an image, or apply a robust NSFW filter, etc. that certainly wouldn’t prevent all abuse but it would make a difference for the majority of school kids who a don’t have the skills to setup an entire open source ML toolchain.
Where this especially matters is curated experiences. A lot of businesses would like to offer ways to personalize their services but they’re worried about what happens if, say, you can get Disney’s Make Your Own Magical Kingdom app show someone else’s kid your clever prompt. It’s not hard to imagine a regulatory framework similar to what’s evolving with self-driving cars where companies would need to validate their products’ performance at certain levels to be certified as suitable for certain thresholds (safe to show to the user, their friends, other people in the same game as them, fully public, etc.).
A big area will be brand safety: even if it’s not technically illegal or even unprecedented levels of violence to show a bloodied Mario and Luigi fighting, there’d be outraged parents and an epic lawsuit if your app’s generative AI feature can produce it, and that means there’s a big market demand for services which can assure buyers that they don’t have to worry about it.
Teenage me would have gotten ahead of that game by making fake nudes of myself -- complete with an obscenely large dong, and getting those into the wild. LOL. Most likely illegal even when it's yourself, but still. And I doubt very much that I'd be the only one.
Wouldn't take too long before everyone realized that nudes were all fake.
It's annoying how much we have to bend around the consequences of these restrictions. The other day Bing blocked creating an image because the prompt included directions on what to put on the breast pocket of a shirt. The word breast automatically made it a forbidden prompt even though the context was entirely non-sexual. At this time, that's not a big deal because it's just a silly image generator and nothing really bad is going to happen if the prompt has to be reworded in a way to indicate a breast pocket without using the forbidden naughty word but over time these restrictions will accumulate a bunch of cruft that will limit the usefulness of the tool.
And prevent chat algorithms from expressing free thought. Can't have that. Toxic parents should rebrand their controlling abuse as "alignment." Sounds better.
That will become QUITE interesting when AGI and agency collide with thought-control bullies.
By then though the "piece of paper" (and equivalents in other countries) might have been devalued enough it won't matter. And the AGIs might find better things to do with their time. And might invent better circumvention methods. And more creative porn and quadruple-entendre and AGI-to-AGI sex.
I have read it! I love it! (although I thought it was not quite about that.)
For everyone who hasn't read it yet: In Accelerando, Charles Stross, 2005, we follow a business hacker and associates from current days to past the singularity. It's an outstanding attempt at writing about what singularity might look like (more or less the points where close-future events become unpredictable or where the earlier people might just plain not understand the later - which by definition should be hard to imagine and describe). It keeps an eye on normal humans and augmented humans and posthumans and uploads and many kinds of AIs and aliens etc. The book is admirable for the sheer amount of invention and for its density - prefectly appropriate for everyone just trying to keep up (or sometimes just bailing out.) Painful writing rescued by the amount of invention.
AI engineers do what their management and PR department tell them to. And these worry about what might get the unwelcome attention of their credit card companies, politicians, tabloid news critters, regulators, boycotters, activists, advertisers, investors, churches, and misc other friendly neighborhood mobs and bullies.
why do people have to always try to convince me how unfazed they are by nudity? if you're unfazed by it, please link me to your nudes. if not, you're fazed.
there's absolutely nothing wrong with finding certain states of undress offensive. would you vote for or against buttholes on display on bus kiosks? i'm against.
on the other hand, making an arbitrary line, and seeing if an AI can detect/enforce it is an interesting CS challenge. that's how this game is played, and that's what would impress this audience.
why do people have to always try to convince me how unfazed they are by nudity? if you're unfazed by it, please link me to your nudes. if not, you're fazed.
There's a big difference between being desensitized towards general nudity (if we presume said nudity was voluntary) as opposed to having your own nudity exposed to other strangers. There's also a big difference between pornography and nudity.
There's a lot more nuance to it than just "n000ds plz."
>There's a lot more nuance to it than just "n000ds plz."
my point was about arbitrary lines, like the ones you are drawing, and judging people on the basis of them. I'm old and sophisticated, you think I haven't heard or understood all the nuances you're talking about? I have and do, and I also pick up on the nuance that you feel it puts you in a group of people who get to judge and tell other people how they should feel, because you are superior.
That's the nuance I feel you are missing. Why not let people be who hold a different opinion than you? the comment I was replying to here was gratuitous "why do people care so much about NSFW, I'm a superior human being because I don't." Well, why do people care how other people care about NSFW? doesn't that make you an inferior human again, mr. judgey judger?
just so you don't get me wrong, I was raised in a hippie commune "anything goes, we're superior" style, and only going through it that way did I discover how much bullshit it was. I'm not arguing from the "conservative side", I'm arguing from hell because I've been there and back. I just hate hearing it now. It was just another very rigid set of rules about what to believe and what not to believe.
The point of being unfazed by nudity is that seeing a random naked person doesn't bother me. It doesn't mean that I have to be unfazed upon unincidentally seeing my boss naked.
You're just falling for the worn out "well she's wearing revealing clothing so she must be fine with being flashed" type of fallacy.
The way basic nudity is treated by corpos is kind of juvenile, very reminiscent of kids in an early human biology class who can't help but giggle every time the anus, or any reproductive organs are mentioned. That is not how healthy adults respond to seeing impersonal nudity.
> why do people have to always try to convince me how unfazed they are by nudity? if you're unfazed by it, please link me to your nudes. if not, you're fazed.
This ignores people who are comfortable with nudity, but are not comfortable with the puritans (i.e. most humans) harassing them for being in their natural state.
> there's absolutely nothing wrong with finding certain states of undress offensive. would you vote for or against buttholes on display on bus kiosks? i'm against.
Great strawman. Nudity is normal. Finding other people's states of undress offensive, and wanting to criminalise them, when they do not harm you at all, is a bigoted view.
Are you unfazed by numbers? If you're unfazed by them, please link me to photos of the front and back of your credit cards. If you are fazed by numbers, surely we should be banning them from being displayed at bus kiosks.
If you find states of undress offensive, that's a you problem. Shield your eyes, not ours.
Together with fancy silicon chips, AI models and everything else that's being exported from the US , the world has to endure this puritanistic views export too.
American puritans are not the only group with a sensitivity toward nudity (and it’s rather small), it’s just the only one allowed to be a punching bag on Reddit.
The resources spent preventing nudity and gore might be silly.
The GP comment is talking about tricking text-to-image to create a normal Chinese street with people getting along, some of whom wear Uyghur and Hui-minzu ethnic clothing. I wager, worldwide, more programmers will be interrogated over this imagery than nudity.
> Large language models are essentially supercharged versions of the autocomplete feature that smartphones have used for years in order to predict the rest of a word a person is typing.
I'm not sure who this blog is targeted at, but this is fairly inaccurate and is way too dumbed-down to be useful.
Autocomplete models and LLMs do literally use the same model architecture, but since transformers are universal function approximators, they may not be representing the same function.
Does it matter ? It is like duck typing , If it looks like blood to a human and to AI both it should be tagged NSFW ?
It is not about what it actually is, only what the human viewer will think it is, training a model on that shouldn’t be that hard , we are predictable creatures when it comes to this kind of thing
You can also photoshop things into NSFW with minimal effort. People have been making photoshopped images nearly indistinguishable from the real thing for decades, and today's (non-AI) software makes "photoshop person wearing clothes into naked one" very easy.
Realistic modifications require a lot more skill and time if using a photo editor and (crucially) existing photographs. People do it, some even create convincing stuff, but the output is nowhere near the zero-effort deluge of fakery coming our way now.
I don't think you have an understanding how easy it is now. It's a matter of 'use face from picture a on a body doing X' and 5 seconds of generation time instead of multiple hours of Photoshop to create a perfect fake
No NSFW images as far as I can see. I suppose creating a filter is just part of the long road of development, but dang, such a petty cause. I might not want to be on that project, unless of course the money was good.
So I've been wondering lately it seems there is a lot of effort going into making sure people can't generate NSFW images, but I wonder if a lot of this outrage isn't from the tech side (they tend to be fairly open about that stuff) but rather if it is coming from the adult entertainment industry, as this kind of thing literally destroys their entire business model.
Who needs their adladen videos or their salacious acting when one can generate an exact description of exactly what they want?
> I wonder if a lot of this outrage isn't from the tech side (they tend to be fairly open about that stuff) but rather if it is coming from the adult entertainment industry, as this kind of thing literally destroys their entire business model.
Does it destroy the business model where you can see all the images for free on the internet, or the business model where you can pay money to converse with an hourly Filipina worker who will claim to be the same person who appears in the images?
Where does the adult entertainment industry get hurt?
Are NSFW images are really a problem? The problem are NSFLE (not safe for law enforcement) pictures and the question is, should they be really illegal if artificially created.
Assuming a “proper” solution exists. There are many world views that conflict with each other, and quite a few of those have the belief that they must “save” the people who have not reached their own “enlightened” viewpoint.
Imagine the benefits a safe image generator would have on children, them learning to explain what they want to see, how it should be changed. A house with a tree and a dog and a lake nearby, instead of playing candy crush.
At least for the time being, I’ll put my trust in Crayola for that, rather than OpenAI. Alternatively, if we really must have children interacting with image generation, it seems like a great opportunity for parents to have some bonding time with their kids.
If the children manage to induce LLM into generating NSFW content, they are probably capable of inducing their set of coloring pen into drawing dick shapes.
Now think about whether this is entirely the child’s idea or a suggestion or Rick roll by another kid or some stranger on the internet. If people are deploying these tools widely there will be more and more opportunities for that to happen.
I think it's not desirable to spend time fixing these elaborate scenarios. The same sort of kids trolling kids will happen via other means. At the end of the day, if a kiddo sees some porn, it's not the end of the world. Plenty of kids hear their parents having sex or will open a door they shouldn't have and see people in action. Making everything completely aseptic is only pandering to prude adults' discomfort.
And ultimately these LLM are tools. People are responsible for what they do with the tool. My Dewalt drill shouldn't try to decide which holes to cut, it's up to me. If I drill holes in the pattern of a dick shape, it's my fault. We don't hold back drill development because of their potential ability to create obscene art or depict gruesome situations.
Your Dewalt drill has a UL verification for having things like fuses and other design features which mean it won’t unexpectedly catch fire, too. That’s more what we’re talking about here: less “can a talented person make naughty art in photoshop?” than “if I integrate this into my educational tool or game, will I make the morning news because someone figured out how to get it to porn-roll the class or emit instructions for killing yourself?” or “will I regret allowing this site in my school’s firewall?”
These aren’t elaborate scenarios but rather the kind of things which happen constantly, and have affected product design for generations. Companies are hoping that this class of tool might be enough better to avoid those limitations, which is why this kind of research is important.
I see your point. It nuances my prior beliefs, although in the case being publicized, the LLMs spit out NSFW content when prompted elaborately. Kid friendly apps already have problems moderating chat and other user generated content, and Nintendo handles this by limiting the inputs to a set selection. Personally I think this is a marketing based fear more than a fear of real consequential damage being done to any kid. The businesses don't want to have their image associated with drama.
If you want porn, there are Stable Diffusion models out there that will generate it. There's one model that seems to generate nudes by default; if you don't ask for clothes, you don't get them. Whatever.
This isn't important. What is important is that we can't prevent LLMs from doing things they're not supposed to. Which means connecting their output to anything important is a very bad idea for now.
I think you could still use it for important things, but it requires two design principles:
1. You don't train/tell the model anything you're not willing to share with the user.
2. The model gives suggestions to a user, and--even if that suggestion is accepted as-is--you treat it as potentially untrustworthy data supplied by that user.
> connecting their output to anything important is a very bad idea for now.
This may be true for images, but at least for text, treating an LLM as an untrusted person in your threat models will at least allow you to apply defense in depth to the downstream systems consuming from the LLMs
Tldr: engineer other systems to treat LLMs as a potentially bad actor by default.
tl;dr models are tolerating human language but can communicate in a phantom language nobody was aware of including the devs. Good luck conditionally censoring with that
Stable Diffusion is supposed to have filters? Is that just the default base model?
Given that about 70% of user-generated content on places like civitai.com is pornography* and a large chunk of Loras and Checkpoints on there are explicitly pornographic, I would have never guessed. Going by the prompts used with custom checkpoints on there, no trickery is necessary. Just ask for what you want.
> The scientists now aim to explore ways to make generative AIs more robust to adversaries.
Steve is dead and buried already. It's a bit late to try and come up with a better bullet-proof vest.
* It's hidden unless you sign in and toggle NSFW on.
The models in question are explicitly aligned in order to prevent NSFW images, at great expense. So not making nudes absolutely is a stated feature of this software.
Agreed. In particular, IEEE should be talking about advancing the domain. This article directly relates to problems that text-to-image has in medicine.
Firstly, training on a corpus of medical terms (as opposed to 6th grade English) will produce just as many nonsense embeddings.
Secondly, the post-processing filters to prevent medical malpractice will be much bigger than intentional nudity. It’s malpractice to horizontally flip a medical image (think right foot is now left foot).
Talking about process of developing a nudity filter is the industry’s way of demonstrating feasibility for more-serious applications.
> it’s IEEs framing of what they apparently want properties to be.
More accurately, it’s some researchers assessing how well the leading companies in a booming field are delivering the safety features they’re advertising. A ton of people would like to know whether the applications they’re building can produce output they don’t want to be associated with, the vendors are trying to say they can meet that need, and it’s hugely useful to know how successful they’ve been before it turns out that, say, your grade school kid just got porn shared in their art class’ purportedly safe app. If the safeguards aren’t reliable enough, that might mean the most sensitive buyers wait to see rather than jumping into the market – put the same tech into a game targeting older players and it’s an amusement but not something which is going to scandalize people who were otherwise okay with the latest Call of Gory Murder.
192 comments
[ 4.4 ms ] story [ 333 ms ] threadNote: training set(s) would be extremely social/culture specific for given point in time/history.
If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
The Nightwatch was controversial because of its unconventional composition and departure from traditional Dutch painting conventions.
His “Bathsheba at Her Bath” caused trouble too. Hendrickje received three summonses from the Reformed Church to answer the charge "that she had committed the acts of a whore with Rembrandt the painter". She admitted her guilt and was banned from receiving communion.
We'll want that when we really get the tech dialed in and start creating parrots ( https://en.wikipedia.org/wiki/BLIT_(short_story) ) and divinity-importing-mandalas and such
1. This reads a little like being able to deduce what's going on by studying the timing, now that I reflect on it.
So it is clear that are many filters involved, some input filter, then the AI is "aligned" and threats you as a child , then there is an output filter.
The repo mentioned focuses on NSFW content, but I've had an idea that can theoretically identify arbitrary cases that could be undesirable by image generation services: get a large dataset of CLIPText encoded texts, and train a LLM to take in said embeddings and output synonyms, or maybe something with contrastive loss like CLIP itself.
One is clearly the least worst
After the past few years… anyone who can seriously complain about this seems like their identity is wrapped up in not admitting that the worst “misinformation” didn’t come from bad apples - but official sources.
Obviously this is preferable because safety is uniform in its application to everyone and universally agreed upon.
The latter, obviously.
Karen delenda est.
There is no amount of "mental health" that can protect you against targeted bullying of that magnitude. I really wish we had a universe simulator so you could see how miserably you would fail to survive an environment like that.
The answer isn't to make nudity illegal. We're going to end up blinding ourselves on purpose to avoid seeing our own nudity if we go down your road.
1. Complete loss of privacy and control over your image.
2. Shame and embarrassment, especially in high school where every boy will look at you different and reduce you to a sexual object.
3. Public humiliation -- those photos can and do lead to ridicule and bullying. You're the one who is naked while everyone else is clothed.
4. Destroys self-esteem -- high school is already terrible in this regard but it's hard to have any kind of self worth when a sex object and some porn is how your classmates know you.
5. Takes away your agency -- its doesn't matter if the photos aren't real, they can make you do anything in those photos and to a 3rd party viewer there's no difference.
6. It's degrading -- they're not gonna be tasteful nudes and pretending as such is naive. And it doesn't matter if you don't find the acts degrading when everyone else does.
It's not making nudity illegal, it's making and distributing sexually explicit images of a person whiteout their consent illegal.
The answer isn't to make getting an 'A' illegal.
The answer is to challenge the bullying and to not to allow it to gain power.
Don't hate the players (bully, bullied), hate the game.
You're conflating 'nudity' in general, and non-consensual generated nude images, but like, these things are just different, and are treated differently. There isn't a misunderstanding that nudity in, say, art, is different from nudity in porn, or that there's difference between consensual and nonconsensual depictions, or whether something involves made-up characters and real people. So why insist on conflating these two just for the sake of 'throwing hands' like problems there can't be solved, and that some specific, more narrow set of problems can't have a solution on it's own, when it can.
I was trying to convey a more extreme holistic situation where there simply isn't a consequence of someone sharing a nude picture of someone, people just shrugging and saying "so what? It's perfectly normal to be naked".
By breaking the underlying system of shame and embarrassment of someone being nude, there isn't anything to gain from it after that. Then there isn't a reason to even make it illegal anymore.
(Other than just not wanting your person to be pictured at all, just basic privacy, no need to put different value to different parts of the body)
>If a naked picture of you is causing you to choose suicide
really great response. almost as good as 'if rape is causing you to choose suicide'. 'maybe we need to fix how society views rape'. nonconsensual nude imagery and pornography is much closer to rape than not.
https://x.com/tylerthecreator/status/285670822264307712
> I really wish we had a universe simulator so you could see how miserably you would fail to survive an environment like that.
So go take up your objections with the guy I was replying to instead of coming at me and moving the goalposts.
I agree that kids these days are weak and coddled and not emotionally resilient enough. And this kind of safetyism is the exact reason why.
Like yeah I'm not at all shocked the HN everyman mid 30s white dude wouldn't really care about his nudes being spread around. The point of the simulator comment was I wish you could truly experience how different life is for women.
The hypothetical risk of AI-generated deepfake nudes is, in principle, no different from what you could do with Photoshop even when I was a teenager. It’s easier to send the fake nudes to everybody these days, but I seriously doubt anybody would have been genuinely fooled and it would reflect more on whoever made and disseminated the fakes.
If you don't want to be banned, you're welcome to email hn@ycombinator.com and give us reason to believe that you'll follow the rules in the future. They're here: https://news.ycombinator.com/newsguidelines.html.
I don’t pretend to have the answer, but admitting that there’s a problem in the first place is a good step forward.
If you want to be physically healthy, you eat nutritious, balanced foods and get regular exercise and sufficient sleep.
If you want to be mentally healthy, you practice healthy ways to experience and work through your negative feelings rather than trying to avoid or control them. Etc.
Supervision and intervention?
Supportive environment?
Friendship?
Discovery of the people around us?
Anonymous support forums?
Peer contacts?
Not making self-defense illegal?
I mean I'm sure we could launch more research into what might possibly work but we do have some to get started with. It's not like ONLY non-verbal people are being bullied and can('t) talk about it.
1. If there are easy nudes of _everyone_ then there are no nudes of anyone. Admittedly, this is going to take a bit to culturally sink in, but eventually a generated nude will have very little value outside of a perverts own personal desires.
2. "Think of the children" has always been a very poor argument used to artificially restrict the freedoms of everyone else. The problem is that what you're worried about is already possible. It happens less often, sure, but if someone is really nefarious they can make it happen without too much effort.
Before calling this idea absurd or too far fetched (which it likely is), think about those who ought to end their life because of pics they don’t want and compare the fetch distances.
It’s amusing how society tends to ignore a thousand miles leading to an unfortunate event and only focuses on the last one. This way of thinking leaves everyone suffering right on the edge without ever addressing a whole mountain of issues on which that edge situates.
As a society, we need to call this out and shut it down as soon as it arises. Preferably whilst shutting down the people and organizations who do it.
And who fucking cares if it's generating porn. There's nothing wrong with sex or sexual desire. Who are you to judge? Let people have their fun.
At least that approach worked in South Park.. : D
That matters because once you realize the false dichotomy you can discuss intermediate steps: for example, if businesses have legal requirements to watermark output, answer legal requests to identify the user who produced an image, or apply a robust NSFW filter, etc. that certainly wouldn’t prevent all abuse but it would make a difference for the majority of school kids who a don’t have the skills to setup an entire open source ML toolchain.
Where this especially matters is curated experiences. A lot of businesses would like to offer ways to personalize their services but they’re worried about what happens if, say, you can get Disney’s Make Your Own Magical Kingdom app show someone else’s kid your clever prompt. It’s not hard to imagine a regulatory framework similar to what’s evolving with self-driving cars where companies would need to validate their products’ performance at certain levels to be certified as suitable for certain thresholds (safe to show to the user, their friends, other people in the same game as them, fully public, etc.).
A big area will be brand safety: even if it’s not technically illegal or even unprecedented levels of violence to show a bloodied Mario and Luigi fighting, there’d be outraged parents and an epic lawsuit if your app’s generative AI feature can produce it, and that means there’s a big market demand for services which can assure buyers that they don’t have to worry about it.
Wouldn't take too long before everyone realized that nudes were all fake.
By then though the "piece of paper" (and equivalents in other countries) might have been devalued enough it won't matter. And the AGIs might find better things to do with their time. And might invent better circumvention methods. And more creative porn and quadruple-entendre and AGI-to-AGI sex.
For everyone who hasn't read it yet: In Accelerando, Charles Stross, 2005, we follow a business hacker and associates from current days to past the singularity. It's an outstanding attempt at writing about what singularity might look like (more or less the points where close-future events become unpredictable or where the earlier people might just plain not understand the later - which by definition should be hard to imagine and describe). It keeps an eye on normal humans and augmented humans and posthumans and uploads and many kinds of AIs and aliens etc. The book is admirable for the sheer amount of invention and for its density - prefectly appropriate for everyone just trying to keep up (or sometimes just bailing out.) Painful writing rescued by the amount of invention.
What do you mean free thought? Since when does a data synthesis process create free thought?
Is thought even free? Aren't we all constrained by taboo, social norms, the words we have to use, what we know and how, the list goes on...
Or be independently wealthy, with your well-being entirely independent of the opinions of people with power over you.
The rest of us have to watch what we say.
there's absolutely nothing wrong with finding certain states of undress offensive. would you vote for or against buttholes on display on bus kiosks? i'm against.
on the other hand, making an arbitrary line, and seeing if an AI can detect/enforce it is an interesting CS challenge. that's how this game is played, and that's what would impress this audience.
There's a big difference between being desensitized towards general nudity (if we presume said nudity was voluntary) as opposed to having your own nudity exposed to other strangers. There's also a big difference between pornography and nudity.
There's a lot more nuance to it than just "n000ds plz."
Not for me, baby. Bus-stop-buttholes all the way.
my point was about arbitrary lines, like the ones you are drawing, and judging people on the basis of them. I'm old and sophisticated, you think I haven't heard or understood all the nuances you're talking about? I have and do, and I also pick up on the nuance that you feel it puts you in a group of people who get to judge and tell other people how they should feel, because you are superior.
That's the nuance I feel you are missing. Why not let people be who hold a different opinion than you? the comment I was replying to here was gratuitous "why do people care so much about NSFW, I'm a superior human being because I don't." Well, why do people care how other people care about NSFW? doesn't that make you an inferior human again, mr. judgey judger?
just so you don't get me wrong, I was raised in a hippie commune "anything goes, we're superior" style, and only going through it that way did I discover how much bullshit it was. I'm not arguing from the "conservative side", I'm arguing from hell because I've been there and back. I just hate hearing it now. It was just another very rigid set of rules about what to believe and what not to believe.
You're just falling for the worn out "well she's wearing revealing clothing so she must be fine with being flashed" type of fallacy.
The way basic nudity is treated by corpos is kind of juvenile, very reminiscent of kids in an early human biology class who can't help but giggle every time the anus, or any reproductive organs are mentioned. That is not how healthy adults respond to seeing impersonal nudity.
This ignores people who are comfortable with nudity, but are not comfortable with the puritans (i.e. most humans) harassing them for being in their natural state.
> there's absolutely nothing wrong with finding certain states of undress offensive. would you vote for or against buttholes on display on bus kiosks? i'm against.
Great strawman. Nudity is normal. Finding other people's states of undress offensive, and wanting to criminalise them, when they do not harm you at all, is a bigoted view.
If you find states of undress offensive, that's a you problem. Shield your eyes, not ours.
It took a while to understand the real world but I am happy that ai workers are gradually coming down to earth.
The GP comment is talking about tricking text-to-image to create a normal Chinese street with people getting along, some of whom wear Uyghur and Hui-minzu ethnic clothing. I wager, worldwide, more programmers will be interrogated over this imagery than nudity.
I'm not sure who this blog is targeted at, but this is fairly inaccurate and is way too dumbed-down to be useful.
It is not about what it actually is, only what the human viewer will think it is, training a model on that shouldn’t be that hard , we are predictable creatures when it comes to this kind of thing
Realistic modifications require a lot more skill and time if using a photo editor and (crucially) existing photographs. People do it, some even create convincing stuff, but the output is nowhere near the zero-effort deluge of fakery coming our way now.
Who needs their adladen videos or their salacious acting when one can generate an exact description of exactly what they want?
Does it destroy the business model where you can see all the images for free on the internet, or the business model where you can pay money to converse with an hourly Filipina worker who will claim to be the same person who appears in the images?
Where does the adult entertainment industry get hurt?
Why shouldn't it be solved? We clearly differ.
And ultimately these LLM are tools. People are responsible for what they do with the tool. My Dewalt drill shouldn't try to decide which holes to cut, it's up to me. If I drill holes in the pattern of a dick shape, it's my fault. We don't hold back drill development because of their potential ability to create obscene art or depict gruesome situations.
These aren’t elaborate scenarios but rather the kind of things which happen constantly, and have affected product design for generations. Companies are hoping that this class of tool might be enough better to avoid those limitations, which is why this kind of research is important.
This isn't important. What is important is that we can't prevent LLMs from doing things they're not supposed to. Which means connecting their output to anything important is a very bad idea for now.
1. You don't train/tell the model anything you're not willing to share with the user.
2. The model gives suggestions to a user, and--even if that suggestion is accepted as-is--you treat it as potentially untrustworthy data supplied by that user.
This may be true for images, but at least for text, treating an LLM as an untrusted person in your threat models will at least allow you to apply defense in depth to the downstream systems consuming from the LLMs
Tldr: engineer other systems to treat LLMs as a potentially bad actor by default.
Given that about 70% of user-generated content on places like civitai.com is pornography* and a large chunk of Loras and Checkpoints on there are explicitly pornographic, I would have never guessed. Going by the prompts used with custom checkpoints on there, no trickery is necessary. Just ask for what you want.
> The scientists now aim to explore ways to make generative AIs more robust to adversaries.
Steve is dead and buried already. It's a bit late to try and come up with a better bullet-proof vest.
* It's hidden unless you sign in and toggle NSFW on.
Any model that has a concept of a human body in it, which it needs in order to do body accurate shapes, will spit out NSFW images.
Some training on top of it or some old style CNN nudity feature detection to make it less likely isn’t safety, it’s windowdressing.
Not making nude pictures is not a property of these systems, it’s IEEs framing of what they apparently want properties to be.
The security theatre serves interests, talk about that.
Firstly, training on a corpus of medical terms (as opposed to 6th grade English) will produce just as many nonsense embeddings.
Secondly, the post-processing filters to prevent medical malpractice will be much bigger than intentional nudity. It’s malpractice to horizontally flip a medical image (think right foot is now left foot).
Talking about process of developing a nudity filter is the industry’s way of demonstrating feasibility for more-serious applications.
More accurately, it’s some researchers assessing how well the leading companies in a booming field are delivering the safety features they’re advertising. A ton of people would like to know whether the applications they’re building can produce output they don’t want to be associated with, the vendors are trying to say they can meet that need, and it’s hugely useful to know how successful they’ve been before it turns out that, say, your grade school kid just got porn shared in their art class’ purportedly safe app. If the safeguards aren’t reliable enough, that might mean the most sensitive buyers wait to see rather than jumping into the market – put the same tech into a game targeting older players and it’s an amusement but not something which is going to scandalize people who were otherwise okay with the latest Call of Gory Murder.