Shared Attention Layer Issues in ChatGPT and DALL-E

5 points by fxtentacle ↗ HN
I just thought this might be interesting for all the discussions about what ChatGPT can or cannot do.

Both ChatGPT and DALL-E and pretty much any current text AI models internally use an embedding space where the meaning of text is encoded. And then they decode that embedding with attention layers. The "attention" here is like a fuzzy key-value lookup to determine which words go together.

And both ChatGPT and DALL-E make the same mistakes here. Mistakes that no human would make.

If you ask ChatGPT to "write a funny story about the medieval princess miyu" it'll actually do "write a story about the funny medieval princess miyu". If you ask DALL-E to generate a "black and white photograph of a green orange in the jungle" it'll generate pictures of a "green orange on top of black and white jungle". What that shows is that the attention layer cannot distinguish between "funny" as an attribute of the story vs. "funny" as an attribute of the person inside the story. Or similarly, between "black and white" as an attribute of the image vs. as an attribute of the jungle.

Both AIs are lacking the ability to distinguish between multiple objects. My guess would be because the entire text is temporarily merged into one embedding space.

In line with that, both ChatGPT and DALL-E also cannot count. DALL-E "photo of twelve oranges" will almost never show 12 oranges. And ChatGPT "count the number of A in "BIKLDJJ34c2424v2bv43v2c3xwbves4b5bLSKJ"" will return random numbers if you ask it multiple times.

5 comments

[ 48.0 ms ] story [ 403 ms ] thread
There were some articles re how generative ML doesn't understand composition (it doesn't understand anything, but it's good at pretending for common stuff) that came out when the vision models were popular last year, e.g: https://garymarcus.substack.com/p/did-googleai-just-snooker-...
great article :)

So then I guess what I'm saying is that ChatGPT has the same composability issues that DALL-E is known to have.

GPT does encode position information in its training data (an idea going back to Vaswani's 2017 paper). But indeed, it can get it wrong if it hasn't seen contrasting examples where different order means something different.

In this case, I'm not sure the mistake you're claiming is a mistake. A reasonable way to produce a funny story is to make the title character funny. Making the title character funny doesn't mean that it misunderstood the directions.

The result is a story that describes the princess as being funny, without actually saying anything funny:

"Once upon a time in the medieval kingdom of Gigglesworth, there was a princess named Miyu. Princess Miyu was known throughout the kingdom for her infectious laugh and love of practical jokes.

One day, the kingdom was thrown into chaos when the royal jester decided to quit his job. The king and queen were beside themselves with worry, knowing that they needed someone to fill the jester's shoes before the upcoming royal ball.

Princess Miyu, always eager to help, stepped up to the plate. "I can be the jester!" she exclaimed with a giggle.

The king and queen were skeptical at first, but Princess Miyu was persistent. She spent the next few days practicing her jokes and tricks, determined to make the royal ball the most entertaining event the kingdom had ever seen.

Finally, the day of the ball arrived. Princess Miyu donned her jester's hat and entered the grand hall, ready to perform.

She started with a classic joke, pulling a fake flower from her sleeve and presenting it to the king. But instead of a flower, a live frog jumped out, much to the king's surprise and the queen's horror.

The crowd erupted in laughter, but the king and queen were not amused. Princess Miyu quickly realized that perhaps she was not cut out for the job of the royal jester after all.

As she slunk out of the hall, she couldn't help but laugh at herself. "Maybe I should stick to being a princess," she thought to herself.

And so, the kingdom of Gigglesworth went on without a jester, but with the memory of Princess Miyu's hilarious attempt to fill the role."

gpt3.5 fails terribly. even when I tried to guide it to reflect and correct, it couldn't. I had it list the characters in a numeric order and it couldn't, but gpt4 seems to be doing good.