Shared Attention Layer Issues in ChatGPT and DALL-E
Both ChatGPT and DALL-E and pretty much any current text AI models internally use an embedding space where the meaning of text is encoded. And then they decode that embedding with attention layers. The "attention" here is like a fuzzy key-value lookup to determine which words go together.
And both ChatGPT and DALL-E make the same mistakes here. Mistakes that no human would make.
If you ask ChatGPT to "write a funny story about the medieval princess miyu" it'll actually do "write a story about the funny medieval princess miyu". If you ask DALL-E to generate a "black and white photograph of a green orange in the jungle" it'll generate pictures of a "green orange on top of black and white jungle". What that shows is that the attention layer cannot distinguish between "funny" as an attribute of the story vs. "funny" as an attribute of the person inside the story. Or similarly, between "black and white" as an attribute of the image vs. as an attribute of the jungle.
Both AIs are lacking the ability to distinguish between multiple objects. My guess would be because the entire text is temporarily merged into one embedding space.
In line with that, both ChatGPT and DALL-E also cannot count. DALL-E "photo of twelve oranges" will almost never show 12 oranges. And ChatGPT "count the number of A in "BIKLDJJ34c2424v2bv43v2c3xwbves4b5bLSKJ"" will return random numbers if you ask it multiple times.
5 comments
[ 48.0 ms ] story [ 403 ms ] threadSo then I guess what I'm saying is that ChatGPT has the same composability issues that DALL-E is known to have.
In this case, I'm not sure the mistake you're claiming is a mistake. A reasonable way to produce a funny story is to make the title character funny. Making the title character funny doesn't mean that it misunderstood the directions.
"Once upon a time in the medieval kingdom of Gigglesworth, there was a princess named Miyu. Princess Miyu was known throughout the kingdom for her infectious laugh and love of practical jokes.
One day, the kingdom was thrown into chaos when the royal jester decided to quit his job. The king and queen were beside themselves with worry, knowing that they needed someone to fill the jester's shoes before the upcoming royal ball.
Princess Miyu, always eager to help, stepped up to the plate. "I can be the jester!" she exclaimed with a giggle.
The king and queen were skeptical at first, but Princess Miyu was persistent. She spent the next few days practicing her jokes and tricks, determined to make the royal ball the most entertaining event the kingdom had ever seen.
Finally, the day of the ball arrived. Princess Miyu donned her jester's hat and entered the grand hall, ready to perform.
She started with a classic joke, pulling a fake flower from her sleeve and presenting it to the king. But instead of a flower, a live frog jumped out, much to the king's surprise and the queen's horror.
The crowd erupted in laughter, but the king and queen were not amused. Princess Miyu quickly realized that perhaps she was not cut out for the job of the royal jester after all.
As she slunk out of the hall, she couldn't help but laugh at herself. "Maybe I should stick to being a princess," she thought to herself.
And so, the kingdom of Gigglesworth went on without a jester, but with the memory of Princess Miyu's hilarious attempt to fill the role."