AI coding isn’t an abstraction, though. You can’t treat a prompt like source code because it will give you a different output every time you use it. An abstraction lets you offload cognitive capacity while retaining…
At what point do LLMs enable bad engineering practices, if instead of working to abstract or encapsulate toilsome programming tasks we point an expensive slot machine at them and generate a bunch of verbose code and…
It's easy to get this way with enough scrolling, try to focus on the things around you in real life. If you aren't reading LinkedIn or HN, how much do you actually hear about AI in day-to-day life? If someone at work…
It's disheartening that a potentially worthwhile discussion — should we invest engineering resources in LLMs as a normal technology rather than as a millenarian fantasy? — has been hijacked by a (at this writing)…
Going to a popular restaurant that accepts app delivery orders (or a grocery store in a neighborhood where people prefer to pay for delivery) is an objectively bad experience. The kitchen or checkout line is backed up…
I'm sure it's nearly an academic distinction, but: > Basically, for any given region, we find its highest point and assume that there is a perfectly placed sibling peak of the same height that is mutually visible.…
A world model itself, in its particulars, isn't as important as the tacit understanding that the "world model" is necessarily incomplete and subordinate to the world itself, that there are sensory inputs from the world…
I think the wide variance in responses here is explainable by tool preference and the circumstance of what you want to work on. You might also have felt "behind" not knowing or wanting to use Dreamweaver, or React, or…
The example given for inverting an embedding back to text doesn't help the idea that this effect is reflecting some "shared statistical model of reality": What would be the plausible whalesong mapping of "Mage (foaled…
Typically debugging, e.g., a tricky race condition in an unfamiliar code base would require adding logging, refactoring library calls, inspecting existing logs, and even rewriting parts of your program to be more…
To make this more concrete: ImageNet enabled computer "vision" by providing images + labels, enabling the computer to take an image and spit out a label. LLM training sets enable text completion by providing text +…
This seems simplistic, tech and infrastructure play a huge part here. A short and incomplete list of things that contributed: - Moore's law petering out, steering hardware advancements towards parallelism - Fast-enough…
The new icon looks weird not just because it's a change, but because if you think of the face as being 3D and illuminated by a light source, the new icon doesn't scan as easily as the old one. You'd have to assume the…
On the one hand, I actually thought it was pretty refreshing to see a tech company keynote that wasn't wall-to-wall with just-so futuristic AI demos that will, if they ever arrive, inevitably be unreliable and finicky…
One thought technology for understanding "hallucination" is that LLMs can only predict a fact statistically using all of the syntax available in its training data. This means that when you ask for a fact, you are really…
I feel like people in the comments are misunderstanding the findings in the article. It’s not that people save time with AI and then turn that time to novel tasks; it’s that perceived savings from using AI are nullified…
This is unnecessarily self-congratulating. The problem is that vulnerabilities are found in cars after they are on the market for a while and already purchased, so existing owners get their rates hiked, but the…
In addition to Kamshak's note about parallel inference accumulating float errors differently due to order of operations, which makes LLMs theoretically non-deterministic at temperature 0, there is the issue of them…
Going back to the original "Bitter Lesson" article, I think the analogy to chess computers could be instructive here. A lot of institutional resources were spent trying to achieve "superhuman" chess performance, it was…
For CS-related books, I think the original 1979 Hopcroft and Ullman is better than its successor editions.
I don't know, the expiration of low interest rates seems to fully explain the slump in the software industry, without considering the advent of chatbot-assisted coding. The job market is just much smaller and new…
Adopting this perspective would improve the quality of efforts around this technology. Instead of thinking of it as somehow creating an "intelligence", seeing it as a complex lens on the training data that is controlled…
Research skills involve not just combining multiple pieces of data, but also being able to apply very subtle skills to determine whether a source is trustworthy, to cross-check numbers where their accuracy is important…
It's hinted at in the article, but so far, it seems that 95%+ of use cases are satisfied by free-tier models, and will imminently be satisfied by local, open-weight models. It's not clear that people or businesses will…
Something like this seems expected, right? If you tune a statistical model to very high accuracy in "addition" over tokens, then the resulting structure of the model must correspond to some structure in the training…
AI coding isn’t an abstraction, though. You can’t treat a prompt like source code because it will give you a different output every time you use it. An abstraction lets you offload cognitive capacity while retaining…
At what point do LLMs enable bad engineering practices, if instead of working to abstract or encapsulate toilsome programming tasks we point an expensive slot machine at them and generate a bunch of verbose code and…
It's easy to get this way with enough scrolling, try to focus on the things around you in real life. If you aren't reading LinkedIn or HN, how much do you actually hear about AI in day-to-day life? If someone at work…
It's disheartening that a potentially worthwhile discussion — should we invest engineering resources in LLMs as a normal technology rather than as a millenarian fantasy? — has been hijacked by a (at this writing)…
Going to a popular restaurant that accepts app delivery orders (or a grocery store in a neighborhood where people prefer to pay for delivery) is an objectively bad experience. The kitchen or checkout line is backed up…
I'm sure it's nearly an academic distinction, but: > Basically, for any given region, we find its highest point and assume that there is a perfectly placed sibling peak of the same height that is mutually visible.…
A world model itself, in its particulars, isn't as important as the tacit understanding that the "world model" is necessarily incomplete and subordinate to the world itself, that there are sensory inputs from the world…
I think the wide variance in responses here is explainable by tool preference and the circumstance of what you want to work on. You might also have felt "behind" not knowing or wanting to use Dreamweaver, or React, or…
The example given for inverting an embedding back to text doesn't help the idea that this effect is reflecting some "shared statistical model of reality": What would be the plausible whalesong mapping of "Mage (foaled…
Typically debugging, e.g., a tricky race condition in an unfamiliar code base would require adding logging, refactoring library calls, inspecting existing logs, and even rewriting parts of your program to be more…
To make this more concrete: ImageNet enabled computer "vision" by providing images + labels, enabling the computer to take an image and spit out a label. LLM training sets enable text completion by providing text +…
This seems simplistic, tech and infrastructure play a huge part here. A short and incomplete list of things that contributed: - Moore's law petering out, steering hardware advancements towards parallelism - Fast-enough…
The new icon looks weird not just because it's a change, but because if you think of the face as being 3D and illuminated by a light source, the new icon doesn't scan as easily as the old one. You'd have to assume the…
On the one hand, I actually thought it was pretty refreshing to see a tech company keynote that wasn't wall-to-wall with just-so futuristic AI demos that will, if they ever arrive, inevitably be unreliable and finicky…
One thought technology for understanding "hallucination" is that LLMs can only predict a fact statistically using all of the syntax available in its training data. This means that when you ask for a fact, you are really…
I feel like people in the comments are misunderstanding the findings in the article. It’s not that people save time with AI and then turn that time to novel tasks; it’s that perceived savings from using AI are nullified…
This is unnecessarily self-congratulating. The problem is that vulnerabilities are found in cars after they are on the market for a while and already purchased, so existing owners get their rates hiked, but the…
In addition to Kamshak's note about parallel inference accumulating float errors differently due to order of operations, which makes LLMs theoretically non-deterministic at temperature 0, there is the issue of them…
Going back to the original "Bitter Lesson" article, I think the analogy to chess computers could be instructive here. A lot of institutional resources were spent trying to achieve "superhuman" chess performance, it was…
For CS-related books, I think the original 1979 Hopcroft and Ullman is better than its successor editions.
I don't know, the expiration of low interest rates seems to fully explain the slump in the software industry, without considering the advent of chatbot-assisted coding. The job market is just much smaller and new…
Adopting this perspective would improve the quality of efforts around this technology. Instead of thinking of it as somehow creating an "intelligence", seeing it as a complex lens on the training data that is controlled…
Research skills involve not just combining multiple pieces of data, but also being able to apply very subtle skills to determine whether a source is trustworthy, to cross-check numbers where their accuracy is important…
It's hinted at in the article, but so far, it seems that 95%+ of use cases are satisfied by free-tier models, and will imminently be satisfied by local, open-weight models. It's not clear that people or businesses will…
Something like this seems expected, right? If you tune a statistical model to very high accuracy in "addition" over tokens, then the resulting structure of the model must correspond to some structure in the training…