RL environment (instruction, stateful container, reward function) is the training data product being bought
No, it isn't. The private data is largely private data, created by highly-specialized, highly-paid contracted teams of experts for domains finance, swe, consulting, etc. Reddit data is just not that interesting, that…
labs invest multiple billion dollars a year each in private data, and that number is growing. internet training data is not where frontier capabilities come from, this view is outdated
I don't condone the practice, and I don't want to run afoul of anthropic or risk my reputation for it
it is certainly possible and being done all over the place. there's a black market that chinese labs use to buy frontier american llm trajectories by the millions through US intermediaries. they're not even particularly…
The benchmarks are now the equivalents of SAT/ACT/other standardized exams for humans. They are directionally quite predictive, but with plenty of outcome variance on the margins
Yes, they did. You could also find this information easily. A company like Andon creates value by exposing interesting AI failure modes, so it makes perfect sense for them to move on to harder problems when the previous…
You could just look it up on their website leaderboard? The newest Claude model makes over $10k profit over a simulated year of operation, after starting with $500
I don't think this was a simple assumption. LLMs used to be much dumber! GPT-3 era LLMS were not good at grep, they were not that good at recovering from errors, and they were not good at making followup queries over…
Kinda funny how we went full circle with you calling me ignorant and illiterate on the basis of not using your preferred terminology, as opposed to the actual, obvious meaning of the subject
It's far more telling that you'd imagine this is a literary tool so interesting or complex that other people haven't seen it or thought of it https://www.google.com/search?q=it%27s+not+x+its+y
Say more, I'd love to know how this is a demonstration of illiteracy
this has ai writing smell all over it. entire paragraphs that just say it's-not-this-it's-that over and over again
one underrated approach more and more people are finding success with: apple watch ultra as a primary device (optionally with a case for a more phone-like factor) you can do most things an iphone does, but you can't…
> In nearly every case of RTO, there was no recorded dip in productivity associated with the move to remote work. I am extremely skeptical of this. On the contrary there is a mountain of direct evidence that people…
the funny part about the backlash is that the travel budgets are likely somewhat pareto distributed and under the not-personalized pricing strategy it is the little guy who is probably overpaying for plane tickets, not…
I find the very popular response of "you're just not using it right" to be big copout for LLMs, especially at the scale we see today. It's hard to think of any other major tech product where it's acceptable to shift so…
would just caching llm responses work here?
The tech industry unfortunately screwed up a basic social contract It is well-understood in every other industry - if you want to be at a prestigious firm, make top compensation, sit in a nice office, work with top-tier…
I'm thinking a lot about these tools, and the biggest challenge in the space seems to be figuring out the right unit of work for AI to take on. A signle class/function is too small to be that helpful, a whole app is too…
this is very cool, have you tried DPO?
Hey guys, it's been over two months since I've been in the weeds with SCIP so I'm not going to be able to write very detailed issues, most of my experiences were with scip python and some in typescript. 1. roles…
My, and my friends experiences with SCIP indexers built by Sourcegraph have been less than stellar. They are buggy and sparsely maintained
I've found embeddings to perform quite poorly on code because 1) user queries are not semantically similar to target code in most cases 2) often times two very concretely related pieces of code are not at all…
One of the most interesting approaches to code search I've seen recently (no affiliation) https://github.com/pyjarrett/septum The hardest part about getting code search right imo is grabbing the right amount of…
RL environment (instruction, stateful container, reward function) is the training data product being bought
No, it isn't. The private data is largely private data, created by highly-specialized, highly-paid contracted teams of experts for domains finance, swe, consulting, etc. Reddit data is just not that interesting, that…
labs invest multiple billion dollars a year each in private data, and that number is growing. internet training data is not where frontier capabilities come from, this view is outdated
I don't condone the practice, and I don't want to run afoul of anthropic or risk my reputation for it
it is certainly possible and being done all over the place. there's a black market that chinese labs use to buy frontier american llm trajectories by the millions through US intermediaries. they're not even particularly…
The benchmarks are now the equivalents of SAT/ACT/other standardized exams for humans. They are directionally quite predictive, but with plenty of outcome variance on the margins
Yes, they did. You could also find this information easily. A company like Andon creates value by exposing interesting AI failure modes, so it makes perfect sense for them to move on to harder problems when the previous…
You could just look it up on their website leaderboard? The newest Claude model makes over $10k profit over a simulated year of operation, after starting with $500
I don't think this was a simple assumption. LLMs used to be much dumber! GPT-3 era LLMS were not good at grep, they were not that good at recovering from errors, and they were not good at making followup queries over…
Kinda funny how we went full circle with you calling me ignorant and illiterate on the basis of not using your preferred terminology, as opposed to the actual, obvious meaning of the subject
It's far more telling that you'd imagine this is a literary tool so interesting or complex that other people haven't seen it or thought of it https://www.google.com/search?q=it%27s+not+x+its+y
Say more, I'd love to know how this is a demonstration of illiteracy
this has ai writing smell all over it. entire paragraphs that just say it's-not-this-it's-that over and over again
one underrated approach more and more people are finding success with: apple watch ultra as a primary device (optionally with a case for a more phone-like factor) you can do most things an iphone does, but you can't…
> In nearly every case of RTO, there was no recorded dip in productivity associated with the move to remote work. I am extremely skeptical of this. On the contrary there is a mountain of direct evidence that people…
the funny part about the backlash is that the travel budgets are likely somewhat pareto distributed and under the not-personalized pricing strategy it is the little guy who is probably overpaying for plane tickets, not…
I find the very popular response of "you're just not using it right" to be big copout for LLMs, especially at the scale we see today. It's hard to think of any other major tech product where it's acceptable to shift so…
would just caching llm responses work here?
The tech industry unfortunately screwed up a basic social contract It is well-understood in every other industry - if you want to be at a prestigious firm, make top compensation, sit in a nice office, work with top-tier…
I'm thinking a lot about these tools, and the biggest challenge in the space seems to be figuring out the right unit of work for AI to take on. A signle class/function is too small to be that helpful, a whole app is too…
this is very cool, have you tried DPO?
Hey guys, it's been over two months since I've been in the weeds with SCIP so I'm not going to be able to write very detailed issues, most of my experiences were with scip python and some in typescript. 1. roles…
My, and my friends experiences with SCIP indexers built by Sourcegraph have been less than stellar. They are buggy and sparsely maintained
I've found embeddings to perform quite poorly on code because 1) user queries are not semantically similar to target code in most cases 2) often times two very concretely related pieces of code are not at all…
One of the most interesting approaches to code search I've seen recently (no affiliation) https://github.com/pyjarrett/septum The hardest part about getting code search right imo is grabbing the right amount of…