Apparently NIST employees dislike Christiano's beliefs and associations. It doesn't seem justified to label their or Christiano's beliefs as "religious". If any disagreement with a common position is "religious war", then all such disagreements are arbitrary and religion itself is just as arbitrary.
There are immediate cybercrime etc activities happening around AI -- eg, content abuse is rampant rn -- so someone advising on active standards that explicitly doesn't (relatively) care about these, especially at the federal standards leadership level, is insane. That's the job!
Internally to gov, depts deploying AI are terrible at data security even before LLMs hoover & regurgitate everything. Most senior leaders don't understand this stuff, like why secure RAG indexing is hard. (Some are sharp, just too few.) That's even before other basics like ai vs human decision making for diff tasks. Getting a future-ethics person asleep at the wheel on the on-fire problems right as deployments & ATOs are getting figured out in every agency.. ouch.
At least for our work, it just takes 1 hat out of the ring for those that we would be otherwise paying attention to, but afaict doesn't really change much otherwise. There are parallel efforts across the DOD etc to set AI compliance standards for internal policies. Likewise, as usual, legal innovators at the state level are also doing legislation to govern commerce, like the recent California thing. Precedent will still get set... just not by NIST?
Ex: Sounds like this person would not be focused on basic efforts to update practical aspects of NIST SP 800-53 for telling teams how it is safe to do AI in the cloud. So either teams will default to NO -- we've seen this as a common reaction in different govs + banks -- or need to point to something like the IC/DOD AI policies. Or it'll continue business as usual for the folks doing the work.
the particular positions christiano is being criticized for are positions on questions such as the following:
- what are the correct standards of right and wrong, if any exist?
- what is the ultimate purpose of human existence?
- what, if anything, will bring about the extinguishment of the human race?
- what sorts of immaterial more-than-human consciousnesses exist, or can exist, and what are the relationships between them and humanity?
it doesn't seem especially controversial or metaphorical to me to describe beliefs about such topics as 'religious', particularly when they are coupled with associations organized around those beliefs. buddhism is routinely described as a 'religion' even though it doesn't involve the worship of any deities and is often described by buddhists as entailing atheism
Ok I see, I interpreted the phrase "association with" to be more about the people he's associating with than the beliefs, but the article is incredibly vague so there's no way for us to know what these criticisms actually are.
the criticisms are that other people have different positions on those questions; it should hardly be surprising that people with conflicting positions on questions like the correct standards of right and wrong will criticize one another
oh and also some bad people are effective altruists, which is something everyone agrees on, including the other effective altruists
It seems concerning that he would be in charge of determining AI policy when he was a high-level OpenAI employee for 4 years and presumably has a huge financial interest in their success (https://www.levels.fyi/blog/openai-compensation.html).
Maybe; hopefully he can sell whatever stake he has in OpenAI to avoid conflict-of-interest concerns. Personally I can hardly think of any better person for the role.
That would be great, but I don't think federal appointees are ever forced or even asked to divest anything. At most he'd just have to disclose the conflict of interest.
Yeah being appointed to such a body should come with a mandatory “you must completely divest from the industry you will be regulating, include and aggregate fund or similar”, alongside “any compensation, hiring, or rehiring contracts and agreements offered prior to or during tenure are explicitly invalid”, and a “you cannot work on the board or executive of any business you I. The relevant industries for five years post departure, or at or with any previous employers for 10 years”
A policy as likely to stick in our marvelous environment of endemic corruption as banning trading by members of the legislative branch and their families.
OpenAI doesn't give stock, they give "Profit Participation Units" that are directly tied to how much profit the company makes. If they aren't making a profit and you can't find a buyer who thinks they will in the future, then they are worthless.
> SAMUEL ALTMAN, an individual, GREGORY
BROCKMAN, an individual, OPENAI, INC., a
corporation, OPENAI, L.P., a limited
partnership, OPENAI, L.L.C., a limited liability
company, OPENAI GP, L.L.C., a limited
liability company, OPENAI OPCO, LLC, a
limited liability company, OPENAI GLOBAL,
LLC, a limited liability company, OAI
CORPORATION, LLC, a limited liability
company, OPENAI HOLDINGS, LLC, a limited
liability company, and DOES 1 through 100,
inclusive
> develop science-based and empirically backed guidelines and standards for AI measurement and policy, laying the foundation for AI safety across the world.
At first glance, terms like "long-term" and "altruism" resonate with me, as I care much more whether we can avoid the heat death in 500 million years than about what happens tomorrow.
Also the main "doctrine" seems to be one I can agree with, namely to find "unusually effective ways" of helping so that with the same ressources, more positive impact can be achieved. (I did the same when I had to decide which charity to support - since I am not a millionaire, but experienced at research, I focused on identifying highly effective charities to avoid wasting my humble charitable contributions.)
However, where things get problematic is when utilitarian ethics get adopted, a paradigm considered problematic in philosophy because it tries to quantify good in order to measure how to maximize it, and the most good to the most people can often result in outcomes that people of good will agree are unethical, namely the suppression or extermination of minorities. For example, the link above compares pandemic deaths with terrorism deaths and concludes that pandemic deaths are the big ticket numbers and in comparison terrorism deaths are small.
While the numbers reported are possibly statistically correct, I would personally object if one drew the conclusion from them that a small number of deaths does not matter, or that they mattered less than larger number of casualties.
Utilitarian ethics, which is often appealing to engineering minds who are used to think in terms of optimizing a variable, has issues with protecting minorities and with justice [1].
From a recent article in the WSJ, it appears that NIST have larger problems than a potentially strange leader: the building substance of the Gaithersburg campus is rotten to the core, to the extent that equipment got destroyed from dampness or water damage etc. A complete overhaul seem due, yet plans exist to cut -- rather than expand -- its funding.
> as I care much more whether we can avoid the heat death in 500 million years than about what happens tomorrow.
Isn't this just an egotistical mechanism that allows the bearer to freely justify any means they please by the supposed loftiness of their stated ends?
It has to be, considering anyone who took such a nonsense philosophy remotely seriously would not be so many orders of magnitude wrong about when heat death is expected to occur. 500My isn't even in the right order for solar expansion!
This article would be better if it reported on the topic of the article, rather than stopping at stating the topic. Seriously, the whole thing could easily fit in 240 characters.
NIST staffers are revolting against his appointment... why, exactly? Ok, it also says he's associated with EA, and EA people have done bad things. Is that what the NIST staffers are saying? What are the staffers saying, anyway? If you're going to claim to be reporting on them, maybe at least try talking to them.
I got pretty much nothing from this article other than the headline. It both piqued and frustrated my curiousity. Perhaps I need to read more about how EA is being problematic in practice. The high-level description seems fine, but then again, every single religion, cult, movement, rebellion, revolution, etc. could be described as "do good stuff" at a high level (or at least "don't do bad stuff"—Don't Be Evil, anyone?) The part between "do good stuff" and "therefore we must commit genocide" is kinda important.
36 comments
[ 4.0 ms ] story [ 87.1 ms ] threadInternally to gov, depts deploying AI are terrible at data security even before LLMs hoover & regurgitate everything. Most senior leaders don't understand this stuff, like why secure RAG indexing is hard. (Some are sharp, just too few.) That's even before other basics like ai vs human decision making for diff tasks. Getting a future-ethics person asleep at the wheel on the on-fire problems right as deployments & ATOs are getting figured out in every agency.. ouch.
At least for our work, it just takes 1 hat out of the ring for those that we would be otherwise paying attention to, but afaict doesn't really change much otherwise. There are parallel efforts across the DOD etc to set AI compliance standards for internal policies. Likewise, as usual, legal innovators at the state level are also doing legislation to govern commerce, like the recent California thing. Precedent will still get set... just not by NIST?
Ex: Sounds like this person would not be focused on basic efforts to update practical aspects of NIST SP 800-53 for telling teams how it is safe to do AI in the cloud. So either teams will default to NO -- we've seen this as a common reaction in different govs + banks -- or need to point to something like the IC/DOD AI policies. Or it'll continue business as usual for the folks doing the work.
- what are the correct standards of right and wrong, if any exist?
- what is the ultimate purpose of human existence?
- what, if anything, will bring about the extinguishment of the human race?
- what sorts of immaterial more-than-human consciousnesses exist, or can exist, and what are the relationships between them and humanity?
it doesn't seem especially controversial or metaphorical to me to describe beliefs about such topics as 'religious', particularly when they are coupled with associations organized around those beliefs. buddhism is routinely described as a 'religion' even though it doesn't involve the worship of any deities and is often described by buddhists as entailing atheism
oh and also some bad people are effective altruists, which is something everyone agrees on, including the other effective altruists
https://www.politico.com/news/2024/02/23/ai-safety-washingto...
"Rules for thee, not for me."
I never understood this. There is money to be made whichever direction the stock moves.
Kinda seems like it's not
https://www.courthousenews.com/wp-content/uploads/2024/02/mu...
(It's also extremely unclear who and how many staffers are 'revolting' against this prospective appointment.)
> develop science-based and empirically backed guidelines and standards for AI measurement and policy, laying the foundation for AI safety across the world.
At first glance, terms like "long-term" and "altruism" resonate with me, as I care much more whether we can avoid the heat death in 500 million years than about what happens tomorrow.
Also the main "doctrine" seems to be one I can agree with, namely to find "unusually effective ways" of helping so that with the same ressources, more positive impact can be achieved. (I did the same when I had to decide which charity to support - since I am not a millionaire, but experienced at research, I focused on identifying highly effective charities to avoid wasting my humble charitable contributions.)
However, where things get problematic is when utilitarian ethics get adopted, a paradigm considered problematic in philosophy because it tries to quantify good in order to measure how to maximize it, and the most good to the most people can often result in outcomes that people of good will agree are unethical, namely the suppression or extermination of minorities. For example, the link above compares pandemic deaths with terrorism deaths and concludes that pandemic deaths are the big ticket numbers and in comparison terrorism deaths are small.
While the numbers reported are possibly statistically correct, I would personally object if one drew the conclusion from them that a small number of deaths does not matter, or that they mattered less than larger number of casualties. Utilitarian ethics, which is often appealing to engineering minds who are used to think in terms of optimizing a variable, has issues with protecting minorities and with justice [1].
From a recent article in the WSJ, it appears that NIST have larger problems than a potentially strange leader: the building substance of the Gaithersburg campus is rotten to the core, to the extent that equipment got destroyed from dampness or water damage etc. A complete overhaul seem due, yet plans exist to cut -- rather than expand -- its funding.
[1] https://www.scu.edu/ethics/ethics-resources/ethical-decision...
Isn't this just an egotistical mechanism that allows the bearer to freely justify any means they please by the supposed loftiness of their stated ends?
(1) If you rank charities by effectiveness, how do you avoid making moral comparisons?
(2) If you are experienced at research into charity efficacy, how is it possible you have not ever encountered effective altruism before?
NIST staffers are revolting against his appointment... why, exactly? Ok, it also says he's associated with EA, and EA people have done bad things. Is that what the NIST staffers are saying? What are the staffers saying, anyway? If you're going to claim to be reporting on them, maybe at least try talking to them.
I got pretty much nothing from this article other than the headline. It both piqued and frustrated my curiousity. Perhaps I need to read more about how EA is being problematic in practice. The high-level description seems fine, but then again, every single religion, cult, movement, rebellion, revolution, etc. could be described as "do good stuff" at a high level (or at least "don't do bad stuff"—Don't Be Evil, anyone?) The part between "do good stuff" and "therefore we must commit genocide" is kinda important.