I usually find myself lurking around when Americans discuss this topic. I like to imagine I'm an Anthropologist/Time-traveler that travelled back in time to the witch trials and is trying to blend in not to get caught…
The Navier-Stokes proof cost ~10mio USD in tokens at consumer prices, and it was something that the two people working on it were allegedly weeks or months away from solving. You can buy ~200 Math PhD Years for 10Mio…
You can also just draw lots.
Robots that replace auto industry factory workers exist; the CEO of GM didn't imagine them as part of some sort of business media induced psychotic episode. The same is not true for the software industry execs.
Amazon lost a cumulative 2.8 billion over their first 17 quarters. So if you're asking about time, then amazon stopped a lot faster. OpenAI is 40 quarters old. If you are asking about money, then amazon... also stopped…
If you want to play games like that, you could also flip it around and ask if the AI would have been eventually fired (assuming no one knew they were talking to a computer). Not sure what that proves.
If you can make it 800 you can claim to be a 100x engineer!
Say what you want about nazis, but they are good at rockets.
But then go right back to being unprofitable again afterwards, which is a little weird.
They don't want to build trust. They want to build a trust wedge between the people making the buying decisions and the people with hands on experience of the product. When an employee says AI isn't speeding up his…
So, it's like if they were a pharma company that was barely profitable if you didn't take into account R&D costs?
[dead]
Sounds like a healthy industry, selling tokens at 1000x below cost.
Yeah, I use them all the time. I just don't see any good argument that it's anything other than statistical pattern matching plus some sort of logic encoded in language. My overfitted LLM obviously didn't arrive at…
I don't know why people are so impressed by 8h. I trained an LLM to write the whole Harry Potter series, and that took JK Rowling like 17 years. For my next point on the graph, I'll train the LLM to write the Bible,…
Karım Kahn at the International Criminal Court would like a word about that.
How do you propose to do a Turing test on a human (in a sense that is different from a machine simply passing the Turing test)? Like failing to pick out all the motorcycles in a captcha, or a turing test where you have…
Same question I have for all these benchmarks: What's going to stop e.g. OpenAI from hiring a bunch of teenagers to play these games non-stop for a month and annotate the game with their logic for deriving the rules,…
That pretty much describes shape up : https://basecamp.com/shapeup I have a mixed relationship to it, but the scope cutting part of it works extremely well. The focus it brings on focusing on the problem solved rather…
In my experience the last answer it gives is usually the right one
It's an interesting parallel to, especially right wingers, want project intelligence into 1 dimension so things all humans can be ordered from inferior to superior. That logic was already strained with humans, but with…
Now they have agents. People need to understand that code is a liability. LLMs hasn't changed that at all. You LLM will get every bit as confused when you have a bug somewhere in the backend and you then work around it…
Your developers were so preoccupied with whether or not they could, they didn't stop to think if they should (add 250kloc)
I love the total lack of humility on that site. "What if the METR study turns out not to capture anything relevant? We just add a constant gap to be conservative!". But I guess these guys aren't really scientist, so…
I rushed so much that I didn't have time to do it right. It could have been the AST tree of my PIN number validation algorithm for ATM machines. :-P
I usually find myself lurking around when Americans discuss this topic. I like to imagine I'm an Anthropologist/Time-traveler that travelled back in time to the witch trials and is trying to blend in not to get caught…
The Navier-Stokes proof cost ~10mio USD in tokens at consumer prices, and it was something that the two people working on it were allegedly weeks or months away from solving. You can buy ~200 Math PhD Years for 10Mio…
You can also just draw lots.
Robots that replace auto industry factory workers exist; the CEO of GM didn't imagine them as part of some sort of business media induced psychotic episode. The same is not true for the software industry execs.
Amazon lost a cumulative 2.8 billion over their first 17 quarters. So if you're asking about time, then amazon stopped a lot faster. OpenAI is 40 quarters old. If you are asking about money, then amazon... also stopped…
If you want to play games like that, you could also flip it around and ask if the AI would have been eventually fired (assuming no one knew they were talking to a computer). Not sure what that proves.
If you can make it 800 you can claim to be a 100x engineer!
Say what you want about nazis, but they are good at rockets.
But then go right back to being unprofitable again afterwards, which is a little weird.
They don't want to build trust. They want to build a trust wedge between the people making the buying decisions and the people with hands on experience of the product. When an employee says AI isn't speeding up his…
So, it's like if they were a pharma company that was barely profitable if you didn't take into account R&D costs?
[dead]
Sounds like a healthy industry, selling tokens at 1000x below cost.
Yeah, I use them all the time. I just don't see any good argument that it's anything other than statistical pattern matching plus some sort of logic encoded in language. My overfitted LLM obviously didn't arrive at…
I don't know why people are so impressed by 8h. I trained an LLM to write the whole Harry Potter series, and that took JK Rowling like 17 years. For my next point on the graph, I'll train the LLM to write the Bible,…
Karım Kahn at the International Criminal Court would like a word about that.
How do you propose to do a Turing test on a human (in a sense that is different from a machine simply passing the Turing test)? Like failing to pick out all the motorcycles in a captcha, or a turing test where you have…
Same question I have for all these benchmarks: What's going to stop e.g. OpenAI from hiring a bunch of teenagers to play these games non-stop for a month and annotate the game with their logic for deriving the rules,…
That pretty much describes shape up : https://basecamp.com/shapeup I have a mixed relationship to it, but the scope cutting part of it works extremely well. The focus it brings on focusing on the problem solved rather…
In my experience the last answer it gives is usually the right one
It's an interesting parallel to, especially right wingers, want project intelligence into 1 dimension so things all humans can be ordered from inferior to superior. That logic was already strained with humans, but with…
Now they have agents. People need to understand that code is a liability. LLMs hasn't changed that at all. You LLM will get every bit as confused when you have a bug somewhere in the backend and you then work around it…
Your developers were so preoccupied with whether or not they could, they didn't stop to think if they should (add 250kloc)
I love the total lack of humility on that site. "What if the METR study turns out not to capture anything relevant? We just add a constant gap to be conservative!". But I guess these guys aren't really scientist, so…
I rushed so much that I didn't have time to do it right. It could have been the AST tree of my PIN number validation algorithm for ATM machines. :-P