> It isn’t “wrong.” Wolfram defines Binomial[n,m] at negative integers by a symmetric limiting rule that enforces Binomial[n,m] = Binomial[n,n−m]. With n = −1, m = −1 this forces Binomial[−1,−1] = Binomial[−1,0] = 1. The gamma-formula has poles at nonpositive integers, so values there depend on which limit you adopt. Wolfram chooses the symmetry-preserving limit; it breaks Pascal’s identity at a few points but keeps symmetry. If you want the convention that preserves Pascal’s rule and makes all cases with both arguments negative zero, use PascalBinomial[−1,−1] = 0. Wolfram added this explicitly to support that alternative definition.
Of course this particular question might have been in the training set.
Honestly 2.5 years feel like infinity when it comes to AI development. I'm using ChatGPT very regularly, and while it's far from perfect, recently it gave obviously wrong answers very rarely. Can't say anything about ChatGPT 5, I feel like in my conversations with AI, I've reached my limit, so I'd hardly notice AI getting smarter, because it's already smart enough for my questions.
I was reading yesterday about a Buddhist concept (albeit quite popular in the west) called Begginer's Mind. I think this post represents it perfectly.
We are presented with a first reaction to chatgpt, we must never forget how incredible this technology is, and not become accustomed to it.
Donald knuth approached several of the questions from the absence of knowledge, asking questions as basic as "12. Write a sentence that contains only 5-letter words.", and being amazed not only by correct answers, but incorrect answers parsed effectively and with semantic understanding.
It's sad that we've made the internet so disorganized and crammed with advertising and crap that we now need tools to find actual information and summarize it for us.
2023 was a crazy and exciting year for AI research. LLMs have come a long way, but clearly still have a long way to go. They should do much better on most of these questions.
The discussion at the end also reminded me of how a lot of us took Gary Marcus' prose more seriously at the time before many of his short-term predictions started failing spectacularly.
Pretty interesting - some contamination, some better answers, and it failed to write a sentence with all 5-letter-words. I’d have expected it to pass this one!
The Haj novel by Leon Uris, by the way, is a disturbing Zionist book that depicts Arabs as backward, violent, and incapable of progress without outside control, and that justifies the taking of Palestinian land by Jewish settlers.
Tokenization. Before text is fed into an LLM, it's processed into tokens typically consisting of multiple characters. GPT-4o would see the word "jokes" as the tokens [73, 17349]. That's much more efficient than processing individual characters, but it means that LLMs can't count letters or words without some additional trickery. LLMs have also struggled with arithmetic for the same reason - they can't "see" the numbers in the text, just tokens representing groups of characters.
> I myself shall certainly continue to leave such research to others, and to devote my time to developing concepts that are authentic and trustworthy. And I hope you do the same.
In a way taming these stochastic beasts into reliable and trustworthy software components is more like (quantitative) social science than computer science.
17 comments
[ 4.1 ms ] story [ 38.8 ms ] thread> It isn’t “wrong.” Wolfram defines Binomial[n,m] at negative integers by a symmetric limiting rule that enforces Binomial[n,m] = Binomial[n,n−m]. With n = −1, m = −1 this forces Binomial[−1,−1] = Binomial[−1,0] = 1. The gamma-formula has poles at nonpositive integers, so values there depend on which limit you adopt. Wolfram chooses the symmetry-preserving limit; it breaks Pascal’s identity at a few points but keeps symmetry. If you want the convention that preserves Pascal’s rule and makes all cases with both arguments negative zero, use PascalBinomial[−1,−1] = 0. Wolfram added this explicitly to support that alternative definition.
Of course this particular question might have been in the training set.
Honestly 2.5 years feel like infinity when it comes to AI development. I'm using ChatGPT very regularly, and while it's far from perfect, recently it gave obviously wrong answers very rarely. Can't say anything about ChatGPT 5, I feel like in my conversations with AI, I've reached my limit, so I'd hardly notice AI getting smarter, because it's already smart enough for my questions.
We are presented with a first reaction to chatgpt, we must never forget how incredible this technology is, and not become accustomed to it.
Donald knuth approached several of the questions from the absence of knowledge, asking questions as basic as "12. Write a sentence that contains only 5-letter words.", and being amazed not only by correct answers, but incorrect answers parsed effectively and with semantic understanding.
The discussion at the end also reminded me of how a lot of us took Gary Marcus' prose more seriously at the time before many of his short-term predictions started failing spectacularly.
https://chatgpt.com/share/6897a21b-25c0-8011-a10a-85850870da...
Pretty interesting - some contamination, some better answers, and it failed to write a sentence with all 5-letter-words. I’d have expected it to pass this one!
Simple example: “Every night, dreams swirl swiftly.
Anyone have an idea how this happened? Supposed to be a sentence of only 5 letter words.
In a way taming these stochastic beasts into reliable and trustworthy software components is more like (quantitative) social science than computer science.
https://espeed.dev/Knuth's-ChatGPT-Experiment-Insights
https://espeed.dev/Unable-to-Access-URL-Content