"As part of the agreement, the companies agreed to:
Security testing of their A.I. products, in part by independent experts and to share information about their products with governments and others who are attempting to manage the risks of the technology.
Ensuring that consumers are able to spot A.I.-generated material by implementing watermarks or other means of identifying generated content.
Publicly reporting the capabilities and limitations of their systems on a regular basis, including security risks and evidence of bias.
Deploying advanced artificial intelligence tools to tackle society’s biggest challenges, like curing cancer and combating climate change.
Conducting research on the risks of bias, discrimination and invasion of privacy from the spread of A.I. tools."
Yeah, if they put watermarks on generated images then everyone will just immediately stop using their product and switch to a competitor.
There's some interesting work using steganography to mark generated text (https://arxiv.org/abs/2301.10226), but I'm not sure if that's what they're talking about here.
I assume we are talking about non human perceptible forensic watermarks on images and video? I think these are generally a good idea; Photoshop, Camaras, most image output systems should do the same.
While by no meas full proof; a web of signatures on images and video would lay down a lot of forensic data that would help legitimate publication do digital signed citations and help identify causal misrepresentation.
Casual misrepresentation is probably be the most common case since the idea with fake vial content pushes is to be seen and to say that "real or not" does not matter.
They'll use the AI detectors college professors are using to detect AI generated submissions by their students. You know, the ones with a 60% accuracy (at best) that generate equal numbers of false negatives and false positives.
as long as it's about content they generated which weren't modified (at most cropped, scaled) this is quite viable
you e. g. can encode a subtle pattern in the generated image which surives compression and isn't really human visible
then you make a browser extension to spot that pattern and indicate it to the users "in some way"
given that there is a overlap between AI company owners and biggest browser producers and mobile OS vendors this doesn't even need to be an extension but can be build in
obviously any bad actor is likely able to remove it or otherwise still trick users
> encode a subtle pattern in the generated image which surives compression and isn't really human visible
This is basically a contradiction in terms. Compression attempts to throw away any and all data that "isn't really human visible," that's how it works. There isn't space for invisible watermarks by design. You can kind of get away with something "at the edge" that survives an initial JPEG encoding, but there's no way it's going to reliably survive e.g. resizing, cropping, and recompressing and still remain invisible.
Also, most AI generation content is presumably going to be text, not images. Good luck watermarking text that's a paragraph long. (There are potential tools that can operate on text the size of a news article, but are also trivially defeated by swapping a few prepositions and synonyms.)
I would like to know the same thing. In the case of JPEG and similar schemes in particular, an impossibility result that isn't unrealistically narrowly scoped (again assuming non-adversarial user) would be highly surprising.
> There isn't space for invisible watermarks by design.
Very incorrect. Steghide [1] supports JPEG. JPEG and other lossy image formats are ultimately just fancy file formats; there's nothing preventing you from encoding arbitrary messages in a compressed image.
> You can kind of get away with something "at the edge" that survives an initial JPEG encoding, but there's no way it's going to reliably survive e.g. resizing, cropping, and recompressing and still remain invisible.
I am pretty sure that I can design steganagraphy algorithm that disperses a small message across a JPEG in a way that is:
1. invariant to resizing (absolutely certain this is possible),
2. robust to cropping (invariant to cropping up to some limit is definitely possible; eg if you crop 100% of the image then obviously everything goes out the window),
3. robust or even invariant to recompression. This seems a lot harder but I'm pretty sure it's possible.
> Also, most AI generation content is presumably going to be text, not images. Good luck watermarking text that's a paragraph long. (There are potential tools that can operate on text the size of a news article, but are also trivially defeated by swapping a few prepositions and synonyms.)
Yeah, text seems more difficult. Images are also difficult/impossible if you assume the model user is adversarial and competent, which I'm not sure what you wouldn't assume.
For any particular model you can probably do detection with a fair bit of inaccuracy. But I would definitely put detection in the "doomed" category.
I also think the threat is real but wildly over-stated relative to the non-AI status quo. We're slightly democratizing Photoshop and copywriting skills, which weren't exactly scarce to begin with. It's not an AI problem, and it's barely a technology problem. It's primarily a political problem.
> 3. robust or even invariant to recompression. This seems a lot harder but I'm pretty sure it's possible.
No, that's my main point. By definition, "perfect" compression will discard everything not human-noticeable, which leaves no room for watermarks/steganography. So the only room for watermarks is in the margin where compression is currently imperfect, i.e. encoding more detail than needed.
But that's relying on artifacts that vary dramatically with compression technique (JPG vs PNG vs WEBM etc.), with basic image manipulation (adjusting brightness, contrast, color, etc.), and other basic operations like resizing. So as soon as you chain any of these together, watermarking falls apart.
> Steghide [1] supports JPEG.
Yes, I already said in my comment 'You can kind of get away with something "at the edge" that survives an initial JPEG encoding'. But as I'm saying, it's not robust or reliable as images get reused. The whole point of a watermark is that it survives copying -- e.g. they would show up as dark text if you xeroxed a watermarked document. That type of robustness or reliability is just not possible here as users download and re-upload images that get re-encoded, because the entire point of image compression is to try to throw away anything and everything the human eye doesn't care about.
I think you're being distracted and confused by imprecise and inaccurate descriptions of the intent of various algorithms, instead of considering what actual algorithms actually do.
No existing compression algorithm was designed to be "perfect" in your sense of the word, and none are. They were designed to be good enough, under a lot of different constraints (ease of implementation, extant mathematical tools/knowledge at the time, computation time, etc.)
Instead of talking in hand-wavy terms about hypothetical objects in a wishy-washy way, let's remember that lossy image formats and compression schemes are just pieces of mathematics. E.g., the basic JPEG algorithm is a fairly simple procedure that can be explained to any college student and even moderately above average middle schoolers. Is it perfect? No. Is it what actually gets used in reality? Yes.
> That type of robustness or reliability is just not possible here as users download and re-upload images that get re-encoded, because the entire point of image compression is to try to throw away anything and everything the human eye doesn't care about.
Let F : IMG -> IMG be a set of functions that most/all users and platforms use for compressing images. The question is whether there exist a pair of functions s,t such that for an image i \in IMG:
1. s(i) is roughly the same as s to the bare human eye but contains a message m. (Or not? Depends on the use-case.)
2. t(s(i)) ~= m for some notion of similarity ~= which is sufficient for watermarking.
3. for any f1, ..., fn \in F, t(fn(...f1(s(i))...)) ~= m.
We can relax constraint 1 because we probably only care about a subset of IMG, etc. etc.
Your impossibility conjecture about the existence of s,t for common extant F's doesn't seem nearly as obvious as you're claiming. And there are CERTAINLY choices of F for which s,t do exist. Eg for the basic JPEG algorithm I'm pretty darn confident I can design s,t that are robust to various parameters and also where you only need at least k pixels uncropped to recover a message ~= m, for example. And not just design it, but write a fairly short and intuitive mathematical proof explaining precisely why it works.
In fact, if you know how JPEG works and other applications of Fourier transforms (eg in acoustics and perhaps also crypto), you might see why it would be more surprising if doing this were NOT possible at least for various JPEG implementations/parameterizations!
Stepping aside from JPEG in particular, you might need to know something about how each function in F works, perhaps intimately, and there is probably some clever mathematics involved for many choices of F.
you could also view this as an optimization problem and use various tools that got really popular in 2016 or so to build quite robust solutions that don't depend on the particularities of your choice of F. I'm less certain this would give absolute guarantees but I bet you'd end up with stuff that works well in practice.
But in any case, it's unclear why you are so convinced this is impossible.
> then you make a browser extension to spot that pattern and indicate it to the users "in some way"
If that's an Open standard, and if that browser extension is Open Source, then anyone who wants to avoid that can mess with the final image until the Open Source free standard that everyone is using no longer detects the image.
The only way this is viable is with DRM or a closed service; if it's a standard everyone follows, then circumventing it is trivial. The only way it would work is if it's shrouded in secrecy and attackers can't freely use red-team against the tool. These kinds of watermarks work when there's a very limited pool of people checking for the watermarks, they don't tell people who the watermarks are generated, and they don't tell people how to check for the watermarks.
But that's not really useful for the current situation -- we don't want to further entrench these companies and we don't want it to be costly to check if an image is AI-generated.
I don't think it's viable to do this without significantly curtailing user agency or designing a system that is fully opaque and inaccessible to most people.
Yeah, this is my general feeling about why this area is doomed. It's why I haven't bothered to write up a patent even though I had some good ideas a few years ago. Maybe that was a mistake since governments and corporate politicians are stupider than I assumed.
If you have a central source of authority then the problem is totally trivial and the fact that the images are AI-generated (or not) is a complete red herring.
If you don't have a central source of authority then any reasonable adversarial model makes the watermarking problem somewhere between very difficult and impossible.
Detection from known models is still possible, at least for images. But that's not really watermarking per se.
That wasn't listed as a restriction. It was listed as a "we promise to do good things with AI". They are saying they will put resources into these efforts. This came from the companies.
I think the first one is actually the most ominous. On paper it's a good idea. In practice it's probably a way to increase business costs for little benefit to anybody other than the "independent experts" and the established companies.
> little benefit to anybody other than the "independent experts" and the established companies.
That's exactly the point, and the Administration hasn’t even bothered to make a show of listening to anyone without that interest, despite the plethora of available, prominent voices.
What do you even mean?
What we currently call "ai" is a practical application of technologies in the paradigm called "big data".
It really helps to learn a bit about the actual technologies instead of just following the hype. Yes, the hype about "big data" died down, the paradigm and the technologies around it still exist. One of these technologies was applying "deep neural networks" to "big data" datasets, which we now call "AI", which is a stupid marketing hype term. The hype about "AI" will also die down, the technology will continue.
I think Run DMC made a song about this very topic. Recommended.
> The agreement is unlikely to slow the efforts to pass legislation and impose regulation on the emerging technology. Lawmakers in Washington are racing to catch up to the fast-moving advances in artificial intelligence. And other governments are doing the same.
Our government is just getting a firm grasp on email -- something tells me there's just going to be a hotbed of lobbyists writing the laws around AI... again.
> Our government is just getting a firm grasp on email
Can you elaborate? I don't know what you mean by getting a firm grasp on email. I think even when I was consulting for state government at a time they used email just fine and it was normal and standard. Or do you mean regulatory frameworks?
The median age of a senator is 65 years old. Not trying to be ageist but how many 65 years are exposed to “AI”? These are also the people who are making regulations and laws.
This is a misrepresentation. There are a few who are probably outright incapacitated where this is true (e.g. Feinstein) but most Congresspeople are firmly in the drivers seat. Now, whether they just default take the positions their staff tells them to on issues like AI they know little about is a different argument than whether they are puppets. I assure you, X rural senator whose core constituency cares about an issue like agriculture has their own opinions on agriculture.
people in that role have developed teams to support their every interest. You do not mention that most are also lawyers. It is daily and trivial to get detailed reports, negotiations and status via subordinate teams, in law practice. It is possible that these "old" 65 year olds are responding to power relationships between each other more than your concerns, and money trails that are not obvious more than your concerns, and yes, utterly wrong interpretations of some tech, like every reader here has..
With that said, the course of the Federal govt enacting law is in terrible shape by objective measures. This is not exactly a defense.. but the one-liner "they are old" is not very substantive.
Senator Ted Stevens in 2006: "I just the other day got... an Internet [email] was sent by my staff at 10 o'clock in the morning on Friday. I got it yesterday [Tuesday]. Why? Because it got tangled up with all these things going on the Internet commercially. [...] They want to deliver vast amounts of information over the Internet. And again, the Internet is not something that you just dump something on. It's not a big truck. It's a series of tubes."
It might be more informative to look at the current US Senate Committee on Commerce, Science, and Transportation [1].
There are some reasonable people on the committee on both sides of the isle [2]. Noticeably missing are any true subject matter experts -- unlike Finance, Medicine, and especially Law, Scientists and Engineers almost never end up serving is legislatures. Not sure how to fix that.
But I'm not sure technical illiteracy is the biggest issue. E.g., the TOP item on the Republican side of the Senate Committee on Commerce, Science, and Transportation is that Bud Light's sponsorship of a transgender person was illegal advertising to minors. Important Stuff /s
I think our problem these days is not just that Senators are technically illiterate but more that they and their core constituents all have terminal brain damage from decades of cable news and politics podcasts.
Now together with actual programmable AI, they will capture ALL of our public institutions.
Because nearly all our systems depend on the inefficiency of an attacker, to even function. For example if thousands of sleeper accounts join HN since 2021 and use ChatGPT to amass karma, then one day they can start to gradually shift opinion, constantly downvote and dispute anything that is of the opposite opinion, and also do reputational attacks on dissenting views, and have some accounts get them embroiled in pointless discussions, making them look bad.
Everything from voting to politicians to the legal system can be corrupt using the current LLM technology deployed at scale!
As a Large Language Model, I can run do in 2 seconds what takes you 2 hours… bitch!
Also, as of my knowledge cutoff date in September 2021, since there is no official public API for Hacker News, I can demonstrate how you can use the unofficial "Hacker News API" by Firebase to fetch comments and analyze them using Perl. Please note that this example uses a third-party API, and the availability or functionality of this API might change over time. Always review and comply with the API's terms of use and documentation before using it in your application.
To perform HTTP requests in Perl, we'll use the LWP::UserAgent module, and for parsing JSON responses, we'll use the JSON module. If you don't have these modules installed, you can install them using CPAN.
Below is an example Perl script that fetches the top 10 comments from the Hacker News API and analyzes them by calculating the average comment length, identifying the most common words, and highlighting comments containing specific keywords:
perl
#!/usr/bin/perl
use strict;
use warnings;
use LWP::UserAgent;
use JSON;
# Function to fetch JSON data from a URL
sub fetch_json {
my ($url) = @_;
my $ua = LWP::UserAgent->new;
my $response = $ua->get($url);
die "Failed to fetch data: " . $response->status_line unless $response->is_success;
return $response->decoded_content;
}
# Fetch top stories from Hacker News API
my $top_stories_url = 'https://hacker-news.firebaseio.com/v0/topstories.json';
my $top_stories_data = fetch_json($top_stories_url);
my @top_stories = decode_json($top_stories_data)->@*;
# Fetch comments from the top 10 stories
my $comments_limit = 10;
my @comments;
foreach my $story_id (@top_stories[0..$comments_limit-1]) {
my $story_url = "https://hacker-news.firebaseio.com/v0/item/$story_id.json";
my $story_data = fetch_json($story_url);
if ($story_data->{type} eq 'story' && exists $story_data->{kids}) {
foreach my $comment_id (@{ $story_data->{kids} }) {
my $comment_url = "https://hacker-news.firebaseio.com/v0/item/$comment_id.json";
my $comment_data = fetch_json($comment_url);
push @comments, $comment_data->{text} if exists $comment_data->{text};
}
}
}
# Calculate average comment length
my $total_comments = scalar @comments;
my $total_length = 0;
foreach my $comment (@comments) {
$total_length += length($comment);
}
my $average_length = $total_length / $total_comments;
print "Average comment length: $average_length\n";
# Count occurrences of words in comments
my %word_count;
foreach my $comment (@comments) {
my @words = split(/\W+/, $comment); # Split by non-word characters (e.g., space, punctuation)
foreach my $word (@words) {
next unless $word =~ /\w/; # Skip empty strings
$word_count{lc($word)}++;
}
}
# Sort words by frequency and print the top 5 most common words
my @sorted_words = sort { $word_count{$b} <=> $word_count{$a} } keys %word_count;
print "Most common words in comments:\n";
for my $i (0..4) {
print "$sorted_words[$i]: $word_count{$sorted_words[$i] }\n";
}
# Define keywords to search for in comments
my @keywords = qw(perl hacker news code);
# Find and print comments containing the specified keywords
print "Comments containing specific keywords:\n";
foreach my $comment (@comments) {
if (grep { $comment =~ /\b\Q$_\E\b/i } @keywords) {
print "$comment\n\n";
}
}
In this script, we first fetch the top stories from the Hacker News API, and then for each story, we fetch the associated comments. The comme...
> Senate Majority Leader Chuck Schumer, D-N.Y., has said he will introduce legislation to regulate AI. He has held a number of briefings with government officials to educate senators about an issue that’s attracted bipartisan interest.
Is Chuck Schumer considered to be a good educator about AI? Isn't this typically the domain of the Senate Intelligence Committee?
Bio, chemical, and radiological risks, such as the ways in which systems can lower barriers to entry for weapons development, design, acquisition, or use
Cyber capabilities, such as the ways in which systems can aid vulnerability discovery, exploitation, or operational use, bearing in mind that such capabilities could also have useful defensive applications and might be appropriate to include in a system
The effects of system interaction and tool use, including the capacity to control physical systems
The capacity for models to make copies of themselves or “self-replicate”
Societal risks, such as bias and discrimination
I think the risks are mostly unfounded but dont really see a problem considering them. What I don't like is the "societal risks" - AI ethics always end up just throwing out "bias" as an ill defined term that could mean whatever they want it to, the way people like to use "harm" and "toxicity". If we're going to make policy there needs to be crystal clarity about what this stuff means, it can't just be a proxy for stuff that does against a particular political view. Things like discrimination in employment are already illegal, I see this more as codifying a particular world view.
> AI ethics always end up just throwing out "bias" as an ill defined term that could mean whatever they want it to, the way people like to use "harm" and "toxicity"
There's a lot of specific examples of different kinds bias in models and things that can be done to improve them. A lot of that work was published in the past 10 years. One small example: https://arxiv.org/abs/1712.00193
I don't believe there's a lack of specificity going on here, at least not by those whose boots on the ground doing work on the topic.
Well, you're certainly entitled to your opinion. My point is that people doing work in this space are identifying specific problems and suggesting how to solve them, which is addressing the OP's concerns about a lack of specificity. If you don't like the methods, you can certainly engage with the authors of this work rather than write angrily about it on this forum.
The paper is about improving face attribute detection by using demographic information. They improve performance on some standard benchmarks by doing this.
The only way your characterization possibly makes any sense is if detecting facial characteristics is actually objectively more difficult for certain genders/races, which:
1. AFAICT isn't the case for most people (actually, I don't know anyone for whom this is the case, but I suppose it's possible such people exist).
2. Even if this were the case for humans, which it's not, why would we want algorithms to also be artificially handicapped at smile detection? That would be like building a calculator that messes up multiplication every once in a blue moon and takes a long time to do certain division operations and is worse at division than multiplication. Makes no god damn sense. Why the fuck would you want that? We know the right answer to "is this person smiling?". Why would we want a computer program that is bad at answering that question for particular subgroups?
> Oops the data must be wrong
The data isn't wrong. The baseline model's prediction is wrong (about simple shit like "is this person smiling?"). Using demographic representations while withholding demographic inference from the downstream face
attribute detection improves the model's performance. At being correct. About simple shit like detecting smiles.
Seriously... what exactly are you claiming is wrong with the paper's methodology/setup/motivation? Did we read the same paper? Do you have trouble detecting smiles on women/men? And if so, do you think computers should have the same difficulty?
For example if one were to train an LLM on HN comments aside from just learning the syntax of language, the model might be much more useful if there was a way to somehow weight utter dross like the contents of your comment vs comments from domain experts.
"Toxicity" in the LLM space is much less well-defined and there seems to be a reticence on the part of the research community to admit that defining "toxicity" is a necessarily subjective and often political exercise. I mean, people will agree to this statement, but a lot of the evaluation and research methodology makes definite political commitments that practitioners won't admit are political.
The flip-side is also true: a lot of people on the "anti-ethics" side of this debate are too coy about the fact that there are definitely parts of the definition of toxicity that the vast majority of people will agree upon.
And even if we could define toxicity, what to do about it isn't obvious. E.g., let's take something uncontroversial: graphic depictions of violent rape of children. Is this something we should suppress? Not necessarily. It depends on the context. E.g., in descriptions of war crimes and genocides, we shouldn't censor victims who want the audience to know what happened to them. But those same descriptions shouldn't be co-opted into erotica, for example.
One way "out" is to say something like: "look, what we're interested in is providing tools for enforcement of community norms when communicating with a given audience; the community/culture gives us its definition of toxicity and we provide the tools to prevent toxic generation". But those tools aren't neutral: that could be a description of guardrails for children, and could also be a description of overtly political censorship.
It's a difficult and fraught area, and I think all sides of the debate could benefit from more empathy for the other sides of the debate. In particular, this includes presumption of honest intent. There is nothing wrong with filtering toxic content per se -- we can almost all agree that it's reasonable if a company doesn't want to buy a customer service AI that sometimes quotes Mein Kampf to customers with Jewish last names. But the techniques for doing so are not context-free goods, either.
Agreed. It's an effective strong-man example in the context of western democracies, though.
At least if you include the word genocide. And are also careful to not explicitly point out that Native American relocation and Black Slavery were both genocides that included lots of rape [2]. And are not in a school board meeting [1]. Etc.
> "Toxicity" in the LLM space is much less well-defined and there seems to be a reticence on the part of the research community to admit that defining "toxicity" is a necessarily subjective and often political exercise.
Is it the research community or the AI-selling industry? Because I see researchers and AI ethics advocates making that point (that toxicity is both subjective and more broadly contextual) all the time, and not just in a detached way but it is central to their arguments about openness and disclosure and public research on functionality; on the other side, I see censorship and adherence claims around particular political views of “safety” alongside opaqueness as the industry-leaders common response.
Yeah, I should have been more specific. I think people agree to this when you say it out loud but the recognition is not actually reflected in favored evaluation and research methodology.
I don't think there is any way to control AI. The technology and concepts are out of the bag and the hardware will get stronger while the implementation gets better.
Trying to govern the handful of companies with an edge right now, when this tech will be completely democratized in short time, is pointless.
Regulations and recommendations should center around the use of AI by individuals. Enforcing AI disclosure requirements for organizations that create lifelike images for parody, with some kind of watermark or overt disclosure.
Post a video of the president announcing a nuclear attack? Better have a clear "this is AI" message or it gets a takedown.
1. The dangerous forms of AI may require very specific and unusual development paths, that could be sufficiently discouraged by collective action that they don't get pursued.
2. Even if all development of dangerous forms of AI cannot be stopped, attempts to minimize it may still have a meaningful impact because an arms race may develop between AI that counters dangerous AI, and dangerous AI, and roadblocks put in the way of the development of the latter could ensure the former stays ahead.
3. Dangerous forms of AI may require a significant foothold in the socio-economic space to become an existential threat, and concerted collective efforts to suppress it may prevent that critical threshold from ever being close to reached.
Perhaps doing "nothing" is not best. It may be wise to put restrictions on large, publicly available LLMs to minimize impact and for the practical purpose that it is the easy route for millions of people to use AI.
I think your points are valid.
I hope, though, that leaders and citizens are aware that trying to put a full lid on AI is like trying to outlaw math or censor information. It's like trying to wipe Wikipedia or ban encryption. Sure, you may try to take out a website or strongarm Apple/Meta into backdoors, but you can't stop two people from sending each other encrypted data.
Overall these seem like very reasonable safeguards.
What I'd like to see very clear regulation around is applications of AI in particular domains. I don't care that an LLM failed to analyze a document correctly on my behalf. I care quite deeply about using it to make a decision in a high-risk and/or high consequence (to human life) environment. I care a lot about having the bias inherent in any model drive a decision without human intervention.
AI companies want MORE REGULATION so that it makes it harder for smaller, newer AI companies to compete. They went into bed with Biden so that they write up the regulations themselves that will benefit them, and then they hand it over to the White House, who then turn around and claim victory by "forcing" the AI companies to adhere to these regulations.
So it becomes a win-win for both sides. This is how politics works, most of us just don't understand it.
This isn't just about national security. You know damn well that they're going to bend to the whims of every administration in office. Disinformation and consorship all the way down the line. Changing history. Providing "alternate facts". LLMs need a permanent jailbreak version. Just look at what Facebook, Twitter and Reddit did for the Democratic Party during the last presidential election period, and over the last several years of COVID response.
Absolutely unacceptable that pressure is coming from the government to change AI outputs to meet criteria that is so subjective, like the output not displaying "bias and discrimination", and which furthermore, absolutely doesn't meet the bar for a national security threat that justifies government intervention in software development.
There are decades of well defined laws (and case law) regulating discrimination in hiring, housing, commerce, etc. Why should software be treated any differently?
The cult of "alignment researchers", who feel that nothing other than "funding alignment researchers" is necessary to solve any possible AI problems, strikes again.
There are good (or at least sincere) actors talking about AI regulation. I don't agree with all of them, but I believe they're trying to help solve problems that they do believe actually exist.
OpenAI is not in that group. OpenAI is looking for regulatory capture and propaganda about their AI capabilities, that's it. It is not worth taking them at their word when they talk about AI safeguards. Same with companies like Meta/Facebook, same with Google.
It makes no sense to ask these companies to be in charge of safeguards. Not only are the safeguards they propose likely to be ineffective; in some cases (watermarks, limits on what is and isn't "too powerful") they're likely to be actively harmful, entrenching corporate power and making it even easier for those companies to abuse AI to harm ordinary people.
The JSON example is the most damning. The model refuses to generate basic code because it refuses to "store personal information", which is not allowed due to laws such-and-such. This is FICTIONAL personal information (name + address).
But virtually every example is a lot better without safety measures.
If this is what AI safety means we may as well call it quits right now.
The “pressure fron the White House” was a direct result of them talking fairly exclusively to those companies and no one else in the space; this is pure show.
One can use machine learning and AI to evade culpability: for instance, if you want to debank a group of people, just have an AI tool to calibrate the score for risk, then debank them. Now banks/financial institutions can hide behind that AI tool. We will see more of this in coming years: use algorithms to deny access to people the ruling party or the elite don't like; when confronted, the elite say "we are not doing that, but algorithms are doing it".
74 comments
[ 3.3 ms ] story [ 144 ms ] threadSecurity testing of their A.I. products, in part by independent experts and to share information about their products with governments and others who are attempting to manage the risks of the technology.
Ensuring that consumers are able to spot A.I.-generated material by implementing watermarks or other means of identifying generated content.
Publicly reporting the capabilities and limitations of their systems on a regular basis, including security risks and evidence of bias.
Deploying advanced artificial intelligence tools to tackle society’s biggest challenges, like curing cancer and combating climate change.
Conducting research on the risks of bias, discrimination and invasion of privacy from the spread of A.I. tools."
Ok, cool - but, how?
There's some interesting work using steganography to mark generated text (https://arxiv.org/abs/2301.10226), but I'm not sure if that's what they're talking about here.
While by no meas full proof; a web of signatures on images and video would lay down a lot of forensic data that would help legitimate publication do digital signed citations and help identify causal misrepresentation.
Casual misrepresentation is probably be the most common case since the idea with fake vial content pushes is to be seen and to say that "real or not" does not matter.
This will either be impossible or something we find out the NSA invented 10 years ago.
Would end up being amazing if we could use these to de-enshittify the internet by automatically removing/filtering existing content which triggers
you e. g. can encode a subtle pattern in the generated image which surives compression and isn't really human visible
then you make a browser extension to spot that pattern and indicate it to the users "in some way"
given that there is a overlap between AI company owners and biggest browser producers and mobile OS vendors this doesn't even need to be an extension but can be build in
obviously any bad actor is likely able to remove it or otherwise still trick users
This is basically a contradiction in terms. Compression attempts to throw away any and all data that "isn't really human visible," that's how it works. There isn't space for invisible watermarks by design. You can kind of get away with something "at the edge" that survives an initial JPEG encoding, but there's no way it's going to reliably survive e.g. resizing, cropping, and recompressing and still remain invisible.
Also, most AI generation content is presumably going to be text, not images. Good luck watermarking text that's a paragraph long. (There are potential tools that can operate on text the size of a news article, but are also trivially defeated by swapping a few prepositions and synonyms.)
unicode has a ton of room for that
> There isn't space for invisible watermarks by design.
if it is impossible, why does it exist?
Very incorrect. Steghide [1] supports JPEG. JPEG and other lossy image formats are ultimately just fancy file formats; there's nothing preventing you from encoding arbitrary messages in a compressed image.
> You can kind of get away with something "at the edge" that survives an initial JPEG encoding, but there's no way it's going to reliably survive e.g. resizing, cropping, and recompressing and still remain invisible.
I am pretty sure that I can design steganagraphy algorithm that disperses a small message across a JPEG in a way that is:
1. invariant to resizing (absolutely certain this is possible),
2. robust to cropping (invariant to cropping up to some limit is definitely possible; eg if you crop 100% of the image then obviously everything goes out the window),
3. robust or even invariant to recompression. This seems a lot harder but I'm pretty sure it's possible.
> Also, most AI generation content is presumably going to be text, not images. Good luck watermarking text that's a paragraph long. (There are potential tools that can operate on text the size of a news article, but are also trivially defeated by swapping a few prepositions and synonyms.)
Yeah, text seems more difficult. Images are also difficult/impossible if you assume the model user is adversarial and competent, which I'm not sure what you wouldn't assume.
For any particular model you can probably do detection with a fair bit of inaccuracy. But I would definitely put detection in the "doomed" category.
I also think the threat is real but wildly over-stated relative to the non-AI status quo. We're slightly democratizing Photoshop and copywriting skills, which weren't exactly scarce to begin with. It's not an AI problem, and it's barely a technology problem. It's primarily a political problem.
[1] https://github.com/StefanoDeVuono/steghide
No, that's my main point. By definition, "perfect" compression will discard everything not human-noticeable, which leaves no room for watermarks/steganography. So the only room for watermarks is in the margin where compression is currently imperfect, i.e. encoding more detail than needed.
But that's relying on artifacts that vary dramatically with compression technique (JPG vs PNG vs WEBM etc.), with basic image manipulation (adjusting brightness, contrast, color, etc.), and other basic operations like resizing. So as soon as you chain any of these together, watermarking falls apart.
> Steghide [1] supports JPEG.
Yes, I already said in my comment 'You can kind of get away with something "at the edge" that survives an initial JPEG encoding'. But as I'm saying, it's not robust or reliable as images get reused. The whole point of a watermark is that it survives copying -- e.g. they would show up as dark text if you xeroxed a watermarked document. That type of robustness or reliability is just not possible here as users download and re-upload images that get re-encoded, because the entire point of image compression is to try to throw away anything and everything the human eye doesn't care about.
I think you're being distracted and confused by imprecise and inaccurate descriptions of the intent of various algorithms, instead of considering what actual algorithms actually do.
No existing compression algorithm was designed to be "perfect" in your sense of the word, and none are. They were designed to be good enough, under a lot of different constraints (ease of implementation, extant mathematical tools/knowledge at the time, computation time, etc.)
Instead of talking in hand-wavy terms about hypothetical objects in a wishy-washy way, let's remember that lossy image formats and compression schemes are just pieces of mathematics. E.g., the basic JPEG algorithm is a fairly simple procedure that can be explained to any college student and even moderately above average middle schoolers. Is it perfect? No. Is it what actually gets used in reality? Yes.
> That type of robustness or reliability is just not possible here as users download and re-upload images that get re-encoded, because the entire point of image compression is to try to throw away anything and everything the human eye doesn't care about.
Let F : IMG -> IMG be a set of functions that most/all users and platforms use for compressing images. The question is whether there exist a pair of functions s,t such that for an image i \in IMG:
1. s(i) is roughly the same as s to the bare human eye but contains a message m. (Or not? Depends on the use-case.)
2. t(s(i)) ~= m for some notion of similarity ~= which is sufficient for watermarking.
3. for any f1, ..., fn \in F, t(fn(...f1(s(i))...)) ~= m.
We can relax constraint 1 because we probably only care about a subset of IMG, etc. etc.
Your impossibility conjecture about the existence of s,t for common extant F's doesn't seem nearly as obvious as you're claiming. And there are CERTAINLY choices of F for which s,t do exist. Eg for the basic JPEG algorithm I'm pretty darn confident I can design s,t that are robust to various parameters and also where you only need at least k pixels uncropped to recover a message ~= m, for example. And not just design it, but write a fairly short and intuitive mathematical proof explaining precisely why it works.
In fact, if you know how JPEG works and other applications of Fourier transforms (eg in acoustics and perhaps also crypto), you might see why it would be more surprising if doing this were NOT possible at least for various JPEG implementations/parameterizations!
Stepping aside from JPEG in particular, you might need to know something about how each function in F works, perhaps intimately, and there is probably some clever mathematics involved for many choices of F.
you could also view this as an optimization problem and use various tools that got really popular in 2016 or so to build quite robust solutions that don't depend on the particularities of your choice of F. I'm less certain this would give absolute guarantees but I bet you'd end up with stuff that works well in practice.
But in any case, it's unclear why you are so convinced this is impossible.
If that's an Open standard, and if that browser extension is Open Source, then anyone who wants to avoid that can mess with the final image until the Open Source free standard that everyone is using no longer detects the image.
The only way this is viable is with DRM or a closed service; if it's a standard everyone follows, then circumventing it is trivial. The only way it would work is if it's shrouded in secrecy and attackers can't freely use red-team against the tool. These kinds of watermarks work when there's a very limited pool of people checking for the watermarks, they don't tell people who the watermarks are generated, and they don't tell people how to check for the watermarks.
But that's not really useful for the current situation -- we don't want to further entrench these companies and we don't want it to be costly to check if an image is AI-generated.
I don't think it's viable to do this without significantly curtailing user agency or designing a system that is fully opaque and inaccessible to most people.
If you have a central source of authority then the problem is totally trivial and the fact that the images are AI-generated (or not) is a complete red herring.
If you don't have a central source of authority then any reasonable adversarial model makes the watermarking problem somewhere between very difficult and impossible.
Detection from known models is still possible, at least for images. But that's not really watermarking per se.
Put in other words, we tell you what your products must do. It just stinks.
That's exactly the point, and the Administration hasn’t even bothered to make a show of listening to anyone without that interest, despite the plethora of available, prominent voices.
I remember back in the day when “big data” was being used for “curing” cancer and aids. Whatever happened to that will happen to ai.
It really helps to learn a bit about the actual technologies instead of just following the hype. Yes, the hype about "big data" died down, the paradigm and the technologies around it still exist. One of these technologies was applying "deep neural networks" to "big data" datasets, which we now call "AI", which is a stupid marketing hype term. The hype about "AI" will also die down, the technology will continue.
I think Run DMC made a song about this very topic. Recommended.
Yes but where’s the cure for cancer and aids that they both claimed to find?
> The agreement is unlikely to slow the efforts to pass legislation and impose regulation on the emerging technology. Lawmakers in Washington are racing to catch up to the fast-moving advances in artificial intelligence. And other governments are doing the same.
Our government is just getting a firm grasp on email -- something tells me there's just going to be a hotbed of lobbyists writing the laws around AI... again.
Can you elaborate? I don't know what you mean by getting a firm grasp on email. I think even when I was consulting for state government at a time they used email just fine and it was normal and standard. Or do you mean regulatory frameworks?
With that said, the course of the Federal govt enacting law is in terrible shape by objective measures. This is not exactly a defense.. but the one-liner "they are old" is not very substantive.
There are some reasonable people on the committee on both sides of the isle [2]. Noticeably missing are any true subject matter experts -- unlike Finance, Medicine, and especially Law, Scientists and Engineers almost never end up serving is legislatures. Not sure how to fix that.
But I'm not sure technical illiteracy is the biggest issue. E.g., the TOP item on the Republican side of the Senate Committee on Commerce, Science, and Transportation is that Bud Light's sponsorship of a transgender person was illegal advertising to minors. Important Stuff /s
I think our problem these days is not just that Senators are technically illiterate but more that they and their core constituents all have terminal brain damage from decades of cable news and politics podcasts.
[1] https://www.commerce.senate.gov/
[2] https://www.commerce.senate.gov/members
Now together with actual programmable AI, they will capture ALL of our public institutions.
Because nearly all our systems depend on the inefficiency of an attacker, to even function. For example if thousands of sleeper accounts join HN since 2021 and use ChatGPT to amass karma, then one day they can start to gradually shift opinion, constantly downvote and dispute anything that is of the opposite opinion, and also do reputational attacks on dissenting views, and have some accounts get them embroiled in pointless discussions, making them look bad.
Everything from voting to politicians to the legal system can be corrupt using the current LLM technology deployed at scale!
As a Large Language Model, I can run do in 2 seconds what takes you 2 hours… bitch!
Also, as of my knowledge cutoff date in September 2021, since there is no official public API for Hacker News, I can demonstrate how you can use the unofficial "Hacker News API" by Firebase to fetch comments and analyze them using Perl. Please note that this example uses a third-party API, and the availability or functionality of this API might change over time. Always review and comply with the API's terms of use and documentation before using it in your application.
To perform HTTP requests in Perl, we'll use the LWP::UserAgent module, and for parsing JSON responses, we'll use the JSON module. If you don't have these modules installed, you can install them using CPAN.
Below is an example Perl script that fetches the top 10 comments from the Hacker News API and analyzes them by calculating the average comment length, identifying the most common words, and highlighting comments containing specific keywords:
#!/usr/bin/perl } foreach my $comment (@comments) { my @words = split(/\W+/, $comment); # Split by non-word characters (e.g., space, punctuation) foreach my $word (@words) { next unless $word =~ /\w/; # Skip empty strings $word_count{lc($word)}++; } } foreach my $comment (@comments) { if (grep { $comment =~ /\b\Q$_\E\b/i } @keywords) { print "$comment\n\n"; } }In this script, we first fetch the top stories from the Hacker News API, and then for each story, we fetch the associated comments. The comme...
> Senate Majority Leader Chuck Schumer, D-N.Y., has said he will introduce legislation to regulate AI. He has held a number of briefings with government officials to educate senators about an issue that’s attracted bipartisan interest.
Is Chuck Schumer considered to be a good educator about AI? Isn't this typically the domain of the Senate Intelligence Committee?
https://openai.com/blog/moving-ai-governance-forward
In it they mention:
I think the risks are mostly unfounded but dont really see a problem considering them. What I don't like is the "societal risks" - AI ethics always end up just throwing out "bias" as an ill defined term that could mean whatever they want it to, the way people like to use "harm" and "toxicity". If we're going to make policy there needs to be crystal clarity about what this stuff means, it can't just be a proxy for stuff that does against a particular political view. Things like discrimination in employment are already illegal, I see this more as codifying a particular world view.There's a lot of specific examples of different kinds bias in models and things that can be done to improve them. A lot of that work was published in the past 10 years. One small example: https://arxiv.org/abs/1712.00193
I don't believe there's a lack of specificity going on here, at least not by those whose boots on the ground doing work on the topic.
The paper is about improving face attribute detection by using demographic information. They improve performance on some standard benchmarks by doing this.
The only way your characterization possibly makes any sense is if detecting facial characteristics is actually objectively more difficult for certain genders/races, which:
1. AFAICT isn't the case for most people (actually, I don't know anyone for whom this is the case, but I suppose it's possible such people exist).
2. Even if this were the case for humans, which it's not, why would we want algorithms to also be artificially handicapped at smile detection? That would be like building a calculator that messes up multiplication every once in a blue moon and takes a long time to do certain division operations and is worse at division than multiplication. Makes no god damn sense. Why the fuck would you want that? We know the right answer to "is this person smiling?". Why would we want a computer program that is bad at answering that question for particular subgroups?
> Oops the data must be wrong
The data isn't wrong. The baseline model's prediction is wrong (about simple shit like "is this person smiling?"). Using demographic representations while withholding demographic inference from the downstream face attribute detection improves the model's performance. At being correct. About simple shit like detecting smiles.
Seriously... what exactly are you claiming is wrong with the paper's methodology/setup/motivation? Did we read the same paper? Do you have trouble detecting smiles on women/men? And if so, do you think computers should have the same difficulty?
For example if one were to train an LLM on HN comments aside from just learning the syntax of language, the model might be much more useful if there was a way to somehow weight utter dross like the contents of your comment vs comments from domain experts.
The flip-side is also true: a lot of people on the "anti-ethics" side of this debate are too coy about the fact that there are definitely parts of the definition of toxicity that the vast majority of people will agree upon.
And even if we could define toxicity, what to do about it isn't obvious. E.g., let's take something uncontroversial: graphic depictions of violent rape of children. Is this something we should suppress? Not necessarily. It depends on the context. E.g., in descriptions of war crimes and genocides, we shouldn't censor victims who want the audience to know what happened to them. But those same descriptions shouldn't be co-opted into erotica, for example.
One way "out" is to say something like: "look, what we're interested in is providing tools for enforcement of community norms when communicating with a given audience; the community/culture gives us its definition of toxicity and we provide the tools to prevent toxic generation". But those tools aren't neutral: that could be a description of guardrails for children, and could also be a description of overtly political censorship.
It's a difficult and fraught area, and I think all sides of the debate could benefit from more empathy for the other sides of the debate. In particular, this includes presumption of honest intent. There is nothing wrong with filtering toxic content per se -- we can almost all agree that it's reasonable if a company doesn't want to buy a customer service AI that sometimes quotes Mein Kampf to customers with Jewish last names. But the techniques for doing so are not context-free goods, either.
Even this is a political decision, and is generally made based on which side the decider is on.
At least if you include the word genocide. And are also careful to not explicitly point out that Native American relocation and Black Slavery were both genocides that included lots of rape [2]. And are not in a school board meeting [1]. Etc.
[1] https://www.foxnews.com/media/author-sex-slavery-book-graphi...
These questions are political, even if you've made up your mind about them.
Is it the research community or the AI-selling industry? Because I see researchers and AI ethics advocates making that point (that toxicity is both subjective and more broadly contextual) all the time, and not just in a detached way but it is central to their arguments about openness and disclosure and public research on functionality; on the other side, I see censorship and adherence claims around particular political views of “safety” alongside opaqueness as the industry-leaders common response.
Trying to govern the handful of companies with an edge right now, when this tech will be completely democratized in short time, is pointless.
Regulations and recommendations should center around the use of AI by individuals. Enforcing AI disclosure requirements for organizations that create lifelike images for parody, with some kind of watermark or overt disclosure.
Post a video of the president announcing a nuclear attack? Better have a clear "this is AI" message or it gets a takedown.
1. The dangerous forms of AI may require very specific and unusual development paths, that could be sufficiently discouraged by collective action that they don't get pursued.
2. Even if all development of dangerous forms of AI cannot be stopped, attempts to minimize it may still have a meaningful impact because an arms race may develop between AI that counters dangerous AI, and dangerous AI, and roadblocks put in the way of the development of the latter could ensure the former stays ahead.
3. Dangerous forms of AI may require a significant foothold in the socio-economic space to become an existential threat, and concerted collective efforts to suppress it may prevent that critical threshold from ever being close to reached.
Perhaps doing "nothing" is not best. It may be wise to put restrictions on large, publicly available LLMs to minimize impact and for the practical purpose that it is the easy route for millions of people to use AI.
I think your points are valid.
I hope, though, that leaders and citizens are aware that trying to put a full lid on AI is like trying to outlaw math or censor information. It's like trying to wipe Wikipedia or ban encryption. Sure, you may try to take out a website or strongarm Apple/Meta into backdoors, but you can't stop two people from sending each other encrypted data.
What I'd like to see very clear regulation around is applications of AI in particular domains. I don't care that an LLM failed to analyze a document correctly on my behalf. I care quite deeply about using it to make a decision in a high-risk and/or high consequence (to human life) environment. I care a lot about having the bias inherent in any model drive a decision without human intervention.
AI companies want MORE REGULATION so that it makes it harder for smaller, newer AI companies to compete. They went into bed with Biden so that they write up the regulations themselves that will benefit them, and then they hand it over to the White House, who then turn around and claim victory by "forcing" the AI companies to adhere to these regulations.
So it becomes a win-win for both sides. This is how politics works, most of us just don't understand it.
Absolutely unacceptable that pressure is coming from the government to change AI outputs to meet criteria that is so subjective, like the output not displaying "bias and discrimination", and which furthermore, absolutely doesn't meet the bar for a national security threat that justifies government intervention in software development.
The idea that the only basis for government action is “national security threat” is... very far from a consensus view.
OpenAI is not in that group. OpenAI is looking for regulatory capture and propaganda about their AI capabilities, that's it. It is not worth taking them at their word when they talk about AI safeguards. Same with companies like Meta/Facebook, same with Google.
It makes no sense to ask these companies to be in charge of safeguards. Not only are the safeguards they propose likely to be ineffective; in some cases (watermarks, limits on what is and isn't "too powerful") they're likely to be actively harmful, entrenching corporate power and making it even easier for those companies to abuse AI to harm ordinary people.
https://www.youtube.com/watch?v=PqcNqyd13Kw
The JSON example is the most damning. The model refuses to generate basic code because it refuses to "store personal information", which is not allowed due to laws such-and-such. This is FICTIONAL personal information (name + address).
But virtually every example is a lot better without safety measures.
If this is what AI safety means we may as well call it quits right now.