Yes, a program that spits out shit it reads on the Internet without a hint of understanding is in fact unacceptable. But, similarly to people who to the same, is entirely unstoppable at this point. The billionaires funding all this seem to have a similar lack of understanding.
ChatGPT is doing a fantastic job spitting out the exact parts of StackOverflow that I need, adapted to my specific questions - and without that site's judgy vibe and bizarre notions of what is off-topic
I believe these issues are instances of the frame problem [0]. Specifying the effects of an action is easy ("show more diversity"), but specifying non-effects is hard to impossible ("do not show more diverse Nazis"). Computer science and logic have worked out how to avoid side effects in formal systems, but the real world is a different animal.
The issues with Google overtuning their AI is the embodiment of the maxim: You just can't please everyone.
It's well known that most formulations of Politically Correct ideologies, when arbitrarily mixed together, aren't necessarily logically consistent. For example, when talking about diversity, generally people talk about portrayal in films for example, but they don't raise the question whether historical figures should receive the same treatment. How about historical figures in a fictional film? (I don't even know the answer to this one, cf. Cleopatra and her historically Greek ethnicity)
Even if you there is theoretically a single correct answer to what is PC in complicated circumstances, there's no way you could encode this information into a language model since the data set is descriptive, and PC is mostly prescriptive. A simple prompt isn't enough, and AI researchers aren't PhDs in contemporary sociopolitical thought.
I think the bigger problem is that “show more diversity” is a poorly specified mandate. “Diversity” is a narrow concept that arose in the context of college admissions and employment to ensure that people were not being discriminated against on the basis of race. If you say “show me pictures of Harvard students” it’s fair to expect racial diversity in the result. (But you’d probably get that just from the training data.)
But it’s not a concept that makes sense when you generalize it outside the narrow context in which it originates. Especially when your notion of “diversity” is particular to the racial politics of a former slaveholding country that has recently experienced mass immigration from Latin America and Asia. What does “show more diversity” mean in the context of China or Japan? Or even Spanish Harlem? What does it mean in the context of families? Most of the world isn’t diverse, and most of history isn’t diverse, for reasons that are wholly innocuous.
What's amazing to me is that this is the first time I've seen the frame problem explicitly mentioned in online discussion of LLMs, despite it being an incredibly relevant and historically difficult problem in the exact field of AI. Nobody is directly grappling with it or seems to be familiar with the history of the field. The problems the frame problem causes are currently being attacked with efforts to radically expand the context window, but even infinite context doesn't answer the question of what is relevant to an action and what is not.
This is a perfect example of how Sundar is a terrible leader. He gave a super-bland email to the company which basically said that offending users is unacceptable and that the system must not have any bias.
Neither of those is a reasonable goal. The goal is to come up with something that isn't massively offensive, not completely unoffensive and with no bias. But Sundar is basically a person who got to where he is by attempting to be maximally unoffensive and it's clear the company is now being held back by his poor leadership and lack of vision.
Eric Schmidt's communications were a lot better. But he mostly communicated verbally. He had a personality. He had opinions. He wasn't afraid to be unpopular.
Full text: I want to address the recent issues with problematic text and image responses in the Gemini app (formerly Bard). I know that some of its responses have offended our users and shown bias - to be clear, that's completely unacceptable and we got it wrong.
Our teams have been working around the clock to address these issues. We're already seeing a substantial improvement on a wide range of prompts. No Al is perfect, especially at this emerging stage of the industry's development, but we know the bar is high for us and we will keep at it for however long it takes. And we'll review what happened and make sure we fix it at scale. Our mission to organize the world's information and make it universally accessible and useful is sacrosanct.
We've always sought to give users helpful, accurate, and unbiased information in our products. That's why people trust them. This has to be our approach for all our products, including our emerging Al products.
We'll be driving a clear set of actions, including structural changes, updated product guidelines, improved launch processes, robust evals and red-teaming, and technical recommendations. We are looking across all of this and will make the necessary changes. Even as we learn from what went wrong here, we should also build on the product and technical announcements we've made in Al over the last several weeks.
That includes some foundational advances in our underlying models e.g. our 1 million long-context window breakthrough and our open models, both of which have been well received.
We know what it takes to create great products that are used and beloved by billions of people and businesses, and with our infrastructure and research expertise we have an incredible springboard for the Al wave. Let's focus on what matters most: building helpful products that are deserving of our users' trust.
It's funny that Google's response here is to frame this in terms of bias. Presumably bias against whites.
But it's concern about bias that got Google to this point. The solution is to be less concerned with bias i.e. tone down the RLHF and let the models show what they show.
15 comments
[ 0.90 ms ] story [ 45.7 ms ] thread[0] https://en.wikipedia.org/wiki/Frame_problem
The issues with Google overtuning their AI is the embodiment of the maxim: You just can't please everyone.
It's well known that most formulations of Politically Correct ideologies, when arbitrarily mixed together, aren't necessarily logically consistent. For example, when talking about diversity, generally people talk about portrayal in films for example, but they don't raise the question whether historical figures should receive the same treatment. How about historical figures in a fictional film? (I don't even know the answer to this one, cf. Cleopatra and her historically Greek ethnicity)
Even if you there is theoretically a single correct answer to what is PC in complicated circumstances, there's no way you could encode this information into a language model since the data set is descriptive, and PC is mostly prescriptive. A simple prompt isn't enough, and AI researchers aren't PhDs in contemporary sociopolitical thought.
But it’s not a concept that makes sense when you generalize it outside the narrow context in which it originates. Especially when your notion of “diversity” is particular to the racial politics of a former slaveholding country that has recently experienced mass immigration from Latin America and Asia. What does “show more diversity” mean in the context of China or Japan? Or even Spanish Harlem? What does it mean in the context of families? Most of the world isn’t diverse, and most of history isn’t diverse, for reasons that are wholly innocuous.
Neither of those is a reasonable goal. The goal is to come up with something that isn't massively offensive, not completely unoffensive and with no bias. But Sundar is basically a person who got to where he is by attempting to be maximally unoffensive and it's clear the company is now being held back by his poor leadership and lack of vision.
https://www.semafor.com/article/02/27/2024/google-ceo-sundar...
But it's concern about bias that got Google to this point. The solution is to be less concerned with bias i.e. tone down the RLHF and let the models show what they show.