Interesting. Upshot - right to left eval means you generally must start at the end, or at least hold an expression in working memory - LLMs - not so good at this.
I wonder if diffusion models would be better at this; most start out as sequential token generators and then get finetuned.
Same reason the same models don't fundamentally understand all languages. They're not trained to. Frankly the design changes to get this to work in training is minimal but this isn't the way English works so expect most of the corporate LLM to struggle because that's where the interest and money is.
Give it time until we have true globally multi lingual models for superior context awareness.
> And it’s not the same as translation to Arabic or Hebrew; direction here refers to the temporal order in which the tokens are produced; even for right-to-left languages, the order in which the tokens get produced remains unchanged; rather, a thin display layer handles the visual presentation.
The popularity of a convention has no relationship with its usability.
Everybody learns in school the traditional convention for writing mathematical expressions.
It appears that for most people it is difficult or impossible to unlearn later such a convention, even if they encounter a superior convention.
On the other hand, I am among those fewer for which this is not true, so when I have first read the book "A Programming Language" of K. Iverson, on which the later APL language and its successors have been based, I have immediately recognized that the Iverson convention is much better than the school convention, and I have no trouble in using it.
When reading a program written with the Iverson convention, you still read from left to right, but you typically do not read until the end of the line, but only as much of the left part as necessary to understand the purpose of the line. (Because the right operand of any operator is everything that follows it until the end of the line, and the details of that computation may be irrelevant. With school notation, when searching where a variable has been modified and how, you must jump between the beginning of the line and the end of the line, to find the last operations that have generated the stored value, when reading and understanding the complete expression would be a waste of time.)
The original motivation of the Iverson convention, which remains very important, was to give a useful meaning for a sequence of identical non-commutative operators, e.g. subtraction and division. This is particularly desirable when the operators are used in vector reductions.
(With school notation, a0 - a1 - a2 - ... - an is seldom a useful expression, but with the Iverson convention it becomes alternate sum, which is needed very frequently. Similarly for division.)
"Claude is aware of that, but it struggled to write correct code based on those rules"
It's actually not, and unless they in some way run a rule engine on top of their LLM SaaS stuff it seems far fetched to believe it adheres to rule sets in any way.
Local models confuse Python, Elixir, PHP and Bash when I've tried to use them for coding. They seem more stable for JS, but sometimes they slip out of that too.
Seems pretty contrived and desperate to invent transpilers from quasi-Python to other languages to try and find a software development use for LLM SaaS. Warnings about Lisp macros and other code rewrite tools ought to apply here as well. Plus, of course, the loss of 'notation as a tool of thought'.
I always thought APL was written in the wrong direction. It writes like a concatenative language that's backwards--you tack things onto the front. NumPy fixes it by making the verbs all dotted function calls, effectively mirroring the order. e.g. in APL you write "10 10 ⍴ ⍳100" but in NumPy you write "np.arange(1, 101).reshape(10, 10)". Even if you don't know either language, you can tell that the APL version is the reverse of the Python version.
My hot take is that Iverson was simply wrong about this. He couldn't be expected to predict code completion and then LLMs both wanting later tokens to depend on earlier tokens. SQL messed it up, too, with "from" not coming first. If APL were developed today, I think left-to-right evaluation would have been preferred. The popularity of dotted function calls in various languages makes it reasonably clear that people like tacking things onto the end and seeing a "pipeline" form from left to right.
There is something deep in this observation. When I reflect on how I write code, sometimes it’s backwards. Sometimes I start with the data and work back through to the outer functions, unnesting as I go. Sometimes I start with the final return and work back to the inputs. I notice sometimes LLMs should work this way, but can’t. So they end up rewriting from the start.
Makes me wonder if future llms will be composing nonlinear things and be able to work in non-token-order spaces temporarily, or will have a way to map their output back to linear token order. I know nonlinear thinking is common while writing code though. current llms might be hiding a deficit by having a large and perfect context window.
> Sometimes I start with the final return and work back to the inputs.
Shouldn't be hard to train a coding LLM to do this too by doubling the training time: train the LLM both forwards and backwards across the training data.
The process of developing software involves this kind of non-linear code editing. When you learn to do something (and the same should go for code, even if sometimes people don't get this critical level of instruction), you don't just look at the final result: you watch people construct the result. The process of constructing code involves a temporarily linear sequence of operations on a text file, but your cursor is bouncing around as you put in commands that move your cursor through the file. We don't have the same kind of copious training data for it, but thereby what we really need to do is to train models not on code, but on all of the input that goes into a text editor. (If we concentrate on software developers that are used to do doing work entirely in a terminal this can be a bit easier, as we can then just essentially train the model on all of the keystrokes they press.)
This is something that diffusion based models would capable of. For example diffusion-coder https://arxiv.org/abs/2506.20639
Could be trained on right to left, but it doesn't seem like they did.
no, it wasn't your choice how you were taught to read and write something like this:
1|2*3>>4+5
in C and k, this expression should hopefully evaluate to 1, but this is just a lucky coincidence: reading and writing these two expressions are wildly different in complexity in those two languages. if you're not sure what i mean, ask your local LLM to explain why that is, but make sure you're sitting down. what you'll discover is that what you think you "simply chose to do" is not what you're actually doing.
while it is true that you can write code anyway you deem fit, i'm afraid you're a bit confused about the actual direction you're forced to think you chose to write it.
but once you're there, it suddenly gets a lot less complicated, and - miraculosly - doesn't cancel out or mess up your previous beliefs and habits.
k/q, of apl heritage, are beautiful - first and foremost because they're simple to write and simple to read.
Another example of this is Claude placing unnecessary imports when writing Python, because it's hedge-importing modules that it suspects it might need later.
It's not because of the left of right evaluation. If the difference was that simple, most humans, let alone LLMs, wouldn't struggle with picking up q when they come from the common languages.
Usually when someone solves problems with q, they don't use the way one would for Python/Java/C/C++/C#/etc.
This is probably a poor example, if I asked someone to write a function to create an nxn identity matrix for a given number the non-q solution would probably involve some kind of nested loop that checks if i==j and assigns 1, otherwise assigns 0.
In q you'd still check equivalence, but instead of looping, you generate a list of numbers as long as the given dimension and then compare each item of the list to itself:
{x=/:x:til x}3
An LLM that's been so heavily trained on an imperative style will likely struggle to solve similar (and often more complex) problems in a standard q manner.
most mainstream models are decoders vs. encoders-decoders, diffusers, etc. and lack reversible causal reasoning, which of course can be counter-intuitive since it doesn’t feel that way when models can regenerate prior content
some hacks for time / position/ space flipping the models:
- test spate of diffusion models emerging. pro is faster, con is smaller context, ymmv is if trained on that language &/or context large enough to ICL lang booster info
- exploit known LTL tricks that may work there’s bunch of these
- e.g., tell model to gen drafts in some sort RPN variant of lang, if tests tell it to simulate creating such a fork of this and then gen clean standard form at end
- have it be explicit about leapfrogging recall and reasoning, eg be excessively verbose with comments can regex strip later
- have it build a stack / combo of the RPN & COT & bootstrapping its own ICL
- exploit causal markers - think tags that can splinter time - this can really boost any of the above methods - eg give each instance of things disjoint time tags, A1 vs K37 for numbered instances of things that share a given space - like a time GUID
- use orthogonal groups of such tags to splinter time and space recall and reasoning in model, to include seemingly naive things like pass 1 etc
- our recent arXiv paper on HDRAM / hypertokens pushes causal markers to classic-quantum holographic extreme and was built for this, next version will be more accessible
- the motivators are simple - models fork on prefix-free modulo embedding noise, so the more you make prefix-free, the better the performance, there’s some massive caveats on how to do this perfectly which is exactly our precise work - think 2x to 10x gain on model and similar on reasoning, again ymmv as we update preprint, post second paper that makes baseline better, prep git release etc to make it tons easier to get better recall and exploit same to get better reasoning by making it possible for any model to do the equivalent of arbitrary RPN
- our future state is exactly this a prompt compiler for exactly this use case - explainable time-independent computation in any model
Languages that are difficult for LLM to read & write are also difficult for the general public. These languages have always had poor uptake and never reach critical mass, or are eventually replaced by better languages.
Language designers would be smart to recognize this fact and favor making their languages more LLM friendly. This should also make them more human friendly.
don't plan on it staying that way. I used to toss wads of my own forth-like language into LLMs to see what kinds of horrible failure modes the latest model would have in parsing and generating such code.
at first they were hilariously bad, then just bad, then kind of okay, and now anthropic's claude4opus reads and writes it just fine.
Cognitive load in LLMs: When LLMs are faced with syntactic complexity (Lisp/J parentheses/RL-NOP), distractors (cat facts), or unfamiliar paradigms (right-to-left evaluation), the model’s performance degrades because its "attention bandwidth" is split or overwhelmed. This mirrors human cognitive overload.
My question: is there a way to reduce cognitive load in LLMs?, one solution seems to be process the input and output format so that the LLM can use a more common format. I don't know if there is a more general solution.
42 comments
[ 3.7 ms ] story [ 110 ms ] threadI wonder if diffusion models would be better at this; most start out as sequential token generators and then get finetuned.
Give it time until we have true globally multi lingual models for superior context awareness.
I'd love to see some quantification of errors in q/kdb+ (or hebrew) vs. languages of similar size that are left-to-right.
W.r.t. natural languages, TFA clarifies it a bit:
> And it’s not the same as translation to Arabic or Hebrew; direction here refers to the temporal order in which the tokens are produced; even for right-to-left languages, the order in which the tokens get produced remains unchanged; rather, a thin display layer handles the visual presentation.
Everybody learns in school the traditional convention for writing mathematical expressions.
It appears that for most people it is difficult or impossible to unlearn later such a convention, even if they encounter a superior convention.
On the other hand, I am among those fewer for which this is not true, so when I have first read the book "A Programming Language" of K. Iverson, on which the later APL language and its successors have been based, I have immediately recognized that the Iverson convention is much better than the school convention, and I have no trouble in using it.
When reading a program written with the Iverson convention, you still read from left to right, but you typically do not read until the end of the line, but only as much of the left part as necessary to understand the purpose of the line. (Because the right operand of any operator is everything that follows it until the end of the line, and the details of that computation may be irrelevant. With school notation, when searching where a variable has been modified and how, you must jump between the beginning of the line and the end of the line, to find the last operations that have generated the stored value, when reading and understanding the complete expression would be a waste of time.)
The original motivation of the Iverson convention, which remains very important, was to give a useful meaning for a sequence of identical non-commutative operators, e.g. subtraction and division. This is particularly desirable when the operators are used in vector reductions.
(With school notation, a0 - a1 - a2 - ... - an is seldom a useful expression, but with the Iverson convention it becomes alternate sum, which is needed very frequently. Similarly for division.)
It's actually not, and unless they in some way run a rule engine on top of their LLM SaaS stuff it seems far fetched to believe it adheres to rule sets in any way.
Local models confuse Python, Elixir, PHP and Bash when I've tried to use them for coding. They seem more stable for JS, but sometimes they slip out of that too.
Seems pretty contrived and desperate to invent transpilers from quasi-Python to other languages to try and find a software development use for LLM SaaS. Warnings about Lisp macros and other code rewrite tools ought to apply here as well. Plus, of course, the loss of 'notation as a tool of thought'.
My hot take is that Iverson was simply wrong about this. He couldn't be expected to predict code completion and then LLMs both wanting later tokens to depend on earlier tokens. SQL messed it up, too, with "from" not coming first. If APL were developed today, I think left-to-right evaluation would have been preferred. The popularity of dotted function calls in various languages makes it reasonably clear that people like tacking things onto the end and seeing a "pipeline" form from left to right.
Makes me wonder if future llms will be composing nonlinear things and be able to work in non-token-order spaces temporarily, or will have a way to map their output back to linear token order. I know nonlinear thinking is common while writing code though. current llms might be hiding a deficit by having a large and perfect context window.
Shouldn't be hard to train a coding LLM to do this too by doubling the training time: train the LLM both forwards and backwards across the training data.
1|2*3>>4+5
in C and k, this expression should hopefully evaluate to 1, but this is just a lucky coincidence: reading and writing these two expressions are wildly different in complexity in those two languages. if you're not sure what i mean, ask your local LLM to explain why that is, but make sure you're sitting down. what you'll discover is that what you think you "simply chose to do" is not what you're actually doing.
while it is true that you can write code anyway you deem fit, i'm afraid you're a bit confused about the actual direction you're forced to think you chose to write it.
but once you're there, it suddenly gets a lot less complicated, and - miraculosly - doesn't cancel out or mess up your previous beliefs and habits.
k/q, of apl heritage, are beautiful - first and foremost because they're simple to write and simple to read.
Usually when someone solves problems with q, they don't use the way one would for Python/Java/C/C++/C#/etc.
This is probably a poor example, if I asked someone to write a function to create an nxn identity matrix for a given number the non-q solution would probably involve some kind of nested loop that checks if i==j and assigns 1, otherwise assigns 0.
In q you'd still check equivalence, but instead of looping, you generate a list of numbers as long as the given dimension and then compare each item of the list to itself:
An LLM that's been so heavily trained on an imperative style will likely struggle to solve similar (and often more complex) problems in a standard q manner.some hacks for time / position/ space flipping the models:
- test spate of diffusion models emerging. pro is faster, con is smaller context, ymmv is if trained on that language &/or context large enough to ICL lang booster info
- exploit known LTL tricks that may work there’s bunch of these
- e.g., tell model to gen drafts in some sort RPN variant of lang, if tests tell it to simulate creating such a fork of this and then gen clean standard form at end
- have it be explicit about leapfrogging recall and reasoning, eg be excessively verbose with comments can regex strip later
- have it build a stack / combo of the RPN & COT & bootstrapping its own ICL
- exploit causal markers - think tags that can splinter time - this can really boost any of the above methods - eg give each instance of things disjoint time tags, A1 vs K37 for numbered instances of things that share a given space - like a time GUID
- use orthogonal groups of such tags to splinter time and space recall and reasoning in model, to include seemingly naive things like pass 1 etc
- our recent arXiv paper on HDRAM / hypertokens pushes causal markers to classic-quantum holographic extreme and was built for this, next version will be more accessible
- the motivators are simple - models fork on prefix-free modulo embedding noise, so the more you make prefix-free, the better the performance, there’s some massive caveats on how to do this perfectly which is exactly our precise work - think 2x to 10x gain on model and similar on reasoning, again ymmv as we update preprint, post second paper that makes baseline better, prep git release etc to make it tons easier to get better recall and exploit same to get better reasoning by making it possible for any model to do the equivalent of arbitrary RPN
- our future state is exactly this a prompt compiler for exactly this use case - explainable time-independent computation in any model
Language designers would be smart to recognize this fact and favor making their languages more LLM friendly. This should also make them more human friendly.
at first they were hilariously bad, then just bad, then kind of okay, and now anthropic's claude4opus reads and writes it just fine.
My question: is there a way to reduce cognitive load in LLMs?, one solution seems to be process the input and output format so that the LLM can use a more common format. I don't know if there is a more general solution.
Edit: Cat attack https://the-decoder.com/cat-attack-on-reasoning-model-shows-...