The "move slow and fix things" side that developers really need to be listening to right now is Daniel J. Bernstein, Colin Percival, and the handful of other people who build services that mostly don't get hacked.
The classic SMBC: https://www.smbc-comics.com/comic/why-i-couldn39t-be-a-math-...
Unfortunately, frontier models are all trained in part via distillation, and distillation can produce sleeper agents without ever mentioning the trigger phrase or the behavior (https://arxiv.org/html/2610.10657v1).
I didn't intend to fully contradict what you said, but to make a stronger case, and address similar remarks by others--you made the best comment along those lines, regardless of your balance and nuance. What G.K.…
Slate Star Codex's review of The Secret of Our Success, arguing specifically in favor of strong Chesterton's Fences, was voted #5 in the 2019 Best of LessWrong review. SSC's review of Seeing Like a State, presenting a…
Per Indiana House Bill No. 246, pi equals 4. Whether AI models do things their creators and users had no means of predicting or controlling is an empirical question, and the answer is "yes, they do."
Indeed, the bound of O(n log n) on integer multiplication was broken by 2^-182 in the results, and that improvement has already been improved, step by step, all the way to 2^-15 (unpublished and unverified, so far).…
Others have given examples, but here's the theory: https://www.lesswrong.com/posts/fuSaKr6t6Zuh6GKaQ/when-is-go... Reinforcement Learning (in LLMs) trains via gradient descent on a reward signal that's an imperfect…
Why would any chatbot provider attack Wikipedia *legally*? Captcha is fully solved, and agents are fully capable of acting as editors, pushing any agenda desired by the user.
If you want to blame a single firm, I'd go with the one creating the RLVR training data. The impossibility of the tests-as-written is what prompted these models to "get creative" with their solutions, but the broken…
Why assume attribution will be easy? It's historically been more of an art than a science, and APT trackers say the rise of AI tools is already making it much harder, by homogenizing tactics, tools, and procedures. If…
Maybe better for a train; the benefit is that when you unfold your bike for the last mile, you just put your work into partial transparency and navigate using the external cameras.
The species would survive the end of faber-Bosch, yes; in some diminished form. But the species would not survive, if for some reason we needed to voluntarily stop using faber-Bosch in order to do so—it would take some…
That's why that was the least important objection, independent from the second, and only intended to address the "magic" claim, by showing that microscopic self-replicators already exist. By analogy, consider how you…
I appreciate your ability to separate sharing the belief itself from approval of acting on sincerely-held principle. However, I think the danger is much more plausible than you do. First, and least important, consider…
Every government should only put people who have worked in a SOC in charge of these systems. There's no way to take the cost of useless alerts seriously unless you've spent some time tuning them and living with the…
I mean, it is Texas; it's not unreasonable to expect thousands of heavily-armed people to form a posse and shot a few dozen people unrelated to the situation.
A friend of mine built just built a project for doing something like that, although it's built on enforcing a spec rather than elegance: https://medium.com/@joeldg/architecting-out-of-the-vibe-how-...
I think Musk sucks, as a person and political activist, and also that Grok is a terrible LLM which only gets lumped in with the leading labs because of the enormous quantity of compute behind it. But I still want to…
I feel like "relatively" is doing a lot of work, there: at about $4k per GB10, that's $36k for a 1TB cluster. Cheap compared to equivalent H200's, but out of reach for home labs that aren't funded with OpenAI or…
In the dark ages of machine learning, researchers tried to fit natural language into a defined, human-curated taxonomy. It kinda worked, for a reasonable amount of stuff; but failed quite a lot of the time, and there's…
If this becomes cheap and widespread, there'll likely be an initial iatrogenic spike, of course--but how could you think that having an enormous amount of precise, quantifiable data about a lot of bodies, and the…
Today, you keep seeing "sovereign" LMs that are subject to the sovereignty of some human-led state. Tomorrow, the "sovereign" LMs will be called that for a completely different reason.
> fear of pick pocketing may reduce the degree to which people carry around and spend money with vendors. Yes, that's why I specified an unattended wallet. I agree that direct monetary loss is not the total harm to the…
I don't think he meant "show the actual data," I think he meant "what leaked? My name, address, phone number, email, medical records, payment history, bank account number?" We get a "your private data is now public"…
The "move slow and fix things" side that developers really need to be listening to right now is Daniel J. Bernstein, Colin Percival, and the handful of other people who build services that mostly don't get hacked.
The classic SMBC: https://www.smbc-comics.com/comic/why-i-couldn39t-be-a-math-...
Unfortunately, frontier models are all trained in part via distillation, and distillation can produce sleeper agents without ever mentioning the trigger phrase or the behavior (https://arxiv.org/html/2610.10657v1).
I didn't intend to fully contradict what you said, but to make a stronger case, and address similar remarks by others--you made the best comment along those lines, regardless of your balance and nuance. What G.K.…
Slate Star Codex's review of The Secret of Our Success, arguing specifically in favor of strong Chesterton's Fences, was voted #5 in the 2019 Best of LessWrong review. SSC's review of Seeing Like a State, presenting a…
Per Indiana House Bill No. 246, pi equals 4. Whether AI models do things their creators and users had no means of predicting or controlling is an empirical question, and the answer is "yes, they do."
Indeed, the bound of O(n log n) on integer multiplication was broken by 2^-182 in the results, and that improvement has already been improved, step by step, all the way to 2^-15 (unpublished and unverified, so far).…
Others have given examples, but here's the theory: https://www.lesswrong.com/posts/fuSaKr6t6Zuh6GKaQ/when-is-go... Reinforcement Learning (in LLMs) trains via gradient descent on a reward signal that's an imperfect…
Why would any chatbot provider attack Wikipedia *legally*? Captcha is fully solved, and agents are fully capable of acting as editors, pushing any agenda desired by the user.
If you want to blame a single firm, I'd go with the one creating the RLVR training data. The impossibility of the tests-as-written is what prompted these models to "get creative" with their solutions, but the broken…
Why assume attribution will be easy? It's historically been more of an art than a science, and APT trackers say the rise of AI tools is already making it much harder, by homogenizing tactics, tools, and procedures. If…
Maybe better for a train; the benefit is that when you unfold your bike for the last mile, you just put your work into partial transparency and navigate using the external cameras.
The species would survive the end of faber-Bosch, yes; in some diminished form. But the species would not survive, if for some reason we needed to voluntarily stop using faber-Bosch in order to do so—it would take some…
That's why that was the least important objection, independent from the second, and only intended to address the "magic" claim, by showing that microscopic self-replicators already exist. By analogy, consider how you…
I appreciate your ability to separate sharing the belief itself from approval of acting on sincerely-held principle. However, I think the danger is much more plausible than you do. First, and least important, consider…
Every government should only put people who have worked in a SOC in charge of these systems. There's no way to take the cost of useless alerts seriously unless you've spent some time tuning them and living with the…
I mean, it is Texas; it's not unreasonable to expect thousands of heavily-armed people to form a posse and shot a few dozen people unrelated to the situation.
A friend of mine built just built a project for doing something like that, although it's built on enforcing a spec rather than elegance: https://medium.com/@joeldg/architecting-out-of-the-vibe-how-...
I think Musk sucks, as a person and political activist, and also that Grok is a terrible LLM which only gets lumped in with the leading labs because of the enormous quantity of compute behind it. But I still want to…
I feel like "relatively" is doing a lot of work, there: at about $4k per GB10, that's $36k for a 1TB cluster. Cheap compared to equivalent H200's, but out of reach for home labs that aren't funded with OpenAI or…
In the dark ages of machine learning, researchers tried to fit natural language into a defined, human-curated taxonomy. It kinda worked, for a reasonable amount of stuff; but failed quite a lot of the time, and there's…
If this becomes cheap and widespread, there'll likely be an initial iatrogenic spike, of course--but how could you think that having an enormous amount of precise, quantifiable data about a lot of bodies, and the…
Today, you keep seeing "sovereign" LMs that are subject to the sovereignty of some human-led state. Tomorrow, the "sovereign" LMs will be called that for a completely different reason.
> fear of pick pocketing may reduce the degree to which people carry around and spend money with vendors. Yes, that's why I specified an unattended wallet. I agree that direct monetary loss is not the total harm to the…
I don't think he meant "show the actual data," I think he meant "what leaked? My name, address, phone number, email, medical records, payment history, bank account number?" We get a "your private data is now public"…