> Trinary didn’t make any headway in the 20th century; binary’s direct mapping to the “on”/”off” states of electric current was just too effective, or seductive; but remember that electric current isn’t actually “on” or “off”. It has taken a ton of engineering to “simulate” those abstract states in real, physical circuits, especially as they have gotten smaller and smaller.
But, I think things are actually trending the other way, right? You just slam the voltage to “on” or “off” nowadays—as things get smaller, voltages get lower, and clock times get faster, it gets harder to resolve the tiny voltage differences.
Maybe you can slam to -1. OTOH, just using 2 bits instead of one... trit(?) seems easier.
Same reason the “close window” button is in the corner. Hitting a particular spot requires precision in 1 or 2 dimensions. Smacking into the boundary is easy.
I've always thought we could put a bit of general purpose TCAM into general purpose computers instead of just routers and switches, and see what people can do with it.
I know (T)CAM's are used in CPU's, but I am nore thinking of the kind of research being done with TCAM's in SSD like products, so maybe we will get there some day.
There's a ton of places in modern silicon where a voltage represents far more than just on or off. From the 16 levels of QLC to the various PAM technologies used by modern interconnects
And the first thing you do to them is to convert them to binary.
Look at any Ethernet PHY for example - you have that nice 5-level signal coming on.. and the first thing you do is feed it into AFE/ADC so you get digital signals that you can actually work with.
So yes, in some specific situations, like Flash memory or interconnects, there are multi-level signals. But the computing itself is always binary.
Transistors are generally at their lowest static power dissipation if the are either fully on or off. The analog middle is great if you're trying to process continuous values, but then you're going to be forced to use a bias current to hold on in the middle, which is ok if that's the nature of the circuit.
A chip with billions of transistors can't reasonably work if most of them are in the analog mode, it'll just melt to slag, unless you have an amazing cooling system.
Also consider that there is only one threshold between values on a binary system. With a trinary system you would likely have to double the power supply voltage, and thus quadruple the power required just to maintain noise margins.
> Trinary is philosophically appealing because its ground-floor vocabulary isn’t “yes” and “no”, but rather: “yes”, “no”, and “maybe”. It’s probably a bit much to imagine that this architectural difference could cascade up through the layers of abstraction and tend to produce software with subtler, richer values … yet I do imagine it.
You can just have a struct { case yes; case no; case maybe; } data structure and pepper it throughout your code wherever you think it’d lead to subtler, richer software… sure, it’s not “at the hardware level” (whatever that means given today’s hardware abstractions) but that should let you demonstrate whatever proof of utility you want to demonstrate.
Trinary is an efficient way of storing lots of -1/0/1 machine learning model weights. But as soon as you load it into memory, you need RAM that can store the same thing (or you're effectively losing the benefits: storage is cheap). So now you need trinary RAM, which as it turns out, isn't great for doing normal general purpose computation with. Integers and floats and boolean values don't get stored efficiently in trinary unless you toss out power of two sized values. CPU circuitry becomes more complicated to add/subtract/multiply those values. Bitwise operators in trinary become essentially impossible for the average IQ engineer to reason about. We need all new IAs, assembly languages, compilers, languages that can run efficiently without the operations that trinary machines can't perform well, etc.
So do we have special memory and CPU instructions for trinary data that lives in a special trinary address space, separate from traditional data that lives in binary address space? No, the juice isn't worth the squeeze. There's no compelling evidence this would make anything better overall: faster, smaller, more energy efficient. Every improvement that trinary potentially offers results in having to throw babies out with the bathwater. It's fun to think about I guess, but I'd bet real money that in 50 years we're still having the same conversation about trinary.
Ternary is indeed an enticing, yet ultimately flawed dream.
Quaternary allows for:
True, “Yes”
False, “No”
Undetermined, “Maybe”, “Either”, True or False
And:
Contradiction, “Invalid”, “Both”, True and False
For logical arithmetic, I.e. reducing tree expressions, True and False are enough.
But in algebraic logic, where more general constraint topologies are possible, the other two values are required.
What is the logical value of the isolated expression “(x)”? I.e. “x” unconstrained?
Or the value of the expression “(x = not x)”?
None of 4-valued logic’s values are optional or spurious for logical algebra.
—-
Many people don’t know this, but all modern computers are quaternary, with 4 quaternit bytes. We don’t just let anyone in on that. Too much power, too much footgun jeopardy, for the unwashed masses and Python “programmers”.
The tricky thicket of web standards can’t be upgraded without introducing mayhem. But Apple’s internal-only docs reveal macOS and Swift have been fully quaternary compliant on their ARM since the M1.
On other systems you can replicate this functionality, at your own risk and effort, by accessing each quaternit with their two bit legacy isomorphic abstraction. Until Rust ships safe direct support.
—-
It will revolutionize computing, from the foundations up, when widely supported.
Russell’s paradox in math is resolved. Given a set S = “The set of all sets that don’t contain themselves”, the truth value of “Is S in S” in quaternary logic, reduces to Contradiction, which indeed it is. I.e. True and False. Making S a well formed, consistent entity, and achieving full set and logical completeness with total closure. So consistency is returned to Set theory and Russell’s quest for a unification of mathematics with just sets and logic becomes possible again. He would have been ecstatic. Gödel be damned! [0]
Turing’s Incompleteness Theorem demonstrates that 2-valued bit machines are inherently inconsistent or incomplete.
Given a machine M, applied to the statement S = “M will say this statement is False”, or “M(S) = False”, it has to fail.
If M(S) returns True, we can see that S is actually False. If M(S) returns False, we can see that actually S is True.
But for a quaternary Machine M4 evaluating S4 = “M4(S4) = False”, M4(S4) returns Contradiction. True and False. Which indeed we can see S4 is. If it is either True or False, we know it is the other as well.
Due to the equivalence of Undecidability and the Turing Halting Problem, resolving one resolves the other. And so quaternary machines are profoundly more powerful and well characterized than binary machines. Far better suited for the hardest and deepest problems in computing.
It’s easy to see why the developers of Rust and Haskell are so adamant about getting this right.
I've never understood the fascination here. Apparently some expression relating the number of possible symbols and the length of a message is closer to euler's number. I don't see why the product of those things is worth optimizing for. The alphabet size that works best is dictated by the storage technology, more symbols usually means it's harder to disambiguate.
2 is the smallest amount of symbols needed to encode information, and makes it the easiest to disambiguate symbols in any implementation, good enough for me.
Ternary Spike: Learning Ternary Spikes for Spiking Neural Networks
> The Spiking Neural Network (SNN), as one of the biologically
inspired neural network infrastructures, has drawn increas-
ing attention recently. It adopts binary spike activations to
transmit information, thus the multiplications of activations
and weights can be substituted by additions, which brings
high energy efficiency. However, in the paper, we theoret-
ically and experimentally prove that the binary spike acti-
vation map cannot carry enough information, thus causing
information loss and resulting in accuracy decreasing. To
handle the problem, we propose a ternary spike neuron to
transmit information. The ternary spike neuron can also enjoy
the event-driven and multiplication-free operation advantages of the binary spike neuron but will boost the information ca-
pacity. Furthermore, we also embed a trainable factor in the
ternary spike neuron to learn the suitable spike amplitude, thus
our SNN will adopt different spike amplitudes along layers,
which can better suit the phenomenon that the membrane po-
tential distributions are different along layers. To retain the
efficiency of the vanilla ternary spike, the trainable ternary
spike SNN will be converted to a standard one again via a re-
parameterization technique in the inference. Extensive experi-
ments with several popular network structures over static and
dynamic datasets show that the ternary spike can consistently
outperform state-of-the-art methods.
I remember reading somewhere that because Ternary computing is inherently reversible, that from an information theoretic point of view that ternary computations have a lower theoretical bound on energy usage, and as such could be a way to bypass heat dissipation problems in chips built with ultra-high density, large size, and high computational load.
I wasn't knowledgeable enough to evaluate that claim at the time, and I'm still not.
I once tried to start enumerating gate types for Trinary.
In binary, with two inputs, there are 2^2 = 4 total possible inputs (00, 01, 10, 11). Different gate types can give different outputs for each of those four inputs: each output can be 0 or 1, so that's 2^4 == 16 different possible gate types. (0, 1, A, B, not A, not B, AND, OR, NAND, NOR, XOR, XNOR, A and not B, B and not A, A or not B, B or not A)
In ternary, with two inputs, there are 3^2 = 9 total possible inputs, so 3^9 = 19,683. I'm sure there are some really sensible ones in there, but damn that's a huge search space. That's where I gave up that time around! :-)
27 comments
[ 3.5 ms ] story [ 38.1 ms ] threadBut, I think things are actually trending the other way, right? You just slam the voltage to “on” or “off” nowadays—as things get smaller, voltages get lower, and clock times get faster, it gets harder to resolve the tiny voltage differences.
Maybe you can slam to -1. OTOH, just using 2 bits instead of one... trit(?) seems easier.
Same reason the “close window” button is in the corner. Hitting a particular spot requires precision in 1 or 2 dimensions. Smacking into the boundary is easy.
I know (T)CAM's are used in CPU's, but I am nore thinking of the kind of research being done with TCAM's in SSD like products, so maybe we will get there some day.
Look at any Ethernet PHY for example - you have that nice 5-level signal coming on.. and the first thing you do is feed it into AFE/ADC so you get digital signals that you can actually work with.
So yes, in some specific situations, like Flash memory or interconnects, there are multi-level signals. But the computing itself is always binary.
A chip with billions of transistors can't reasonably work if most of them are in the analog mode, it'll just melt to slag, unless you have an amazing cooling system.
Also consider that there is only one threshold between values on a binary system. With a trinary system you would likely have to double the power supply voltage, and thus quadruple the power required just to maintain noise margins.
You can just have a struct { case yes; case no; case maybe; } data structure and pepper it throughout your code wherever you think it’d lead to subtler, richer software… sure, it’s not “at the hardware level” (whatever that means given today’s hardware abstractions) but that should let you demonstrate whatever proof of utility you want to demonstrate.
So do we have special memory and CPU instructions for trinary data that lives in a special trinary address space, separate from traditional data that lives in binary address space? No, the juice isn't worth the squeeze. There's no compelling evidence this would make anything better overall: faster, smaller, more energy efficient. Every improvement that trinary potentially offers results in having to throw babies out with the bathwater. It's fun to think about I guess, but I'd bet real money that in 50 years we're still having the same conversation about trinary.
Quaternary allows for:
And: For logical arithmetic, I.e. reducing tree expressions, True and False are enough.But in algebraic logic, where more general constraint topologies are possible, the other two values are required.
What is the logical value of the isolated expression “(x)”? I.e. “x” unconstrained?
Or the value of the expression “(x = not x)”?
None of 4-valued logic’s values are optional or spurious for logical algebra.
—-
Many people don’t know this, but all modern computers are quaternary, with 4 quaternit bytes. We don’t just let anyone in on that. Too much power, too much footgun jeopardy, for the unwashed masses and Python “programmers”.
The tricky thicket of web standards can’t be upgraded without introducing mayhem. But Apple’s internal-only docs reveal macOS and Swift have been fully quaternary compliant on their ARM since the M1.
On other systems you can replicate this functionality, at your own risk and effort, by accessing each quaternit with their two bit legacy isomorphic abstraction. Until Rust ships safe direct support.
—-
It will revolutionize computing, from the foundations up, when widely supported.
Russell’s paradox in math is resolved. Given a set S = “The set of all sets that don’t contain themselves”, the truth value of “Is S in S” in quaternary logic, reduces to Contradiction, which indeed it is. I.e. True and False. Making S a well formed, consistent entity, and achieving full set and logical completeness with total closure. So consistency is returned to Set theory and Russell’s quest for a unification of mathematics with just sets and logic becomes possible again. He would have been ecstatic. Gödel be damned! [0]
Turing’s Incompleteness Theorem demonstrates that 2-valued bit machines are inherently inconsistent or incomplete.
Given a machine M, applied to the statement S = “M will say this statement is False”, or “M(S) = False”, it has to fail.
If M(S) returns True, we can see that S is actually False. If M(S) returns False, we can see that actually S is True.
But for a quaternary Machine M4 evaluating S4 = “M4(S4) = False”, M4(S4) returns Contradiction. True and False. Which indeed we can see S4 is. If it is either True or False, we know it is the other as well.
Due to the equivalence of Undecidability and the Turing Halting Problem, resolving one resolves the other. And so quaternary machines are profoundly more powerful and well characterized than binary machines. Far better suited for the hardest and deepest problems in computing.
It’s easy to see why the developers of Rust and Haskell are so adamant about getting this right.
[0] https://tinyurl.com/godelbedamned
The quantum dream is also the trinary dream.
2 is the smallest amount of symbols needed to encode information, and makes it the easiest to disambiguate symbols in any implementation, good enough for me.
[1]: https://arxiv.org/abs/2402.17764
[2]: https://huggingface.co/tiiuae/Falcon-E-3B-Instruct
[3]: https://huggingface.co/microsoft/bitnet-b1.58-2B-4T
https://github.com/yfguo91/Ternary-Spike
Ternary Spike: Learning Ternary Spikes for Spiking Neural Networks
> The Spiking Neural Network (SNN), as one of the biologically inspired neural network infrastructures, has drawn increas- ing attention recently. It adopts binary spike activations to transmit information, thus the multiplications of activations and weights can be substituted by additions, which brings high energy efficiency. However, in the paper, we theoret- ically and experimentally prove that the binary spike acti- vation map cannot carry enough information, thus causing information loss and resulting in accuracy decreasing. To handle the problem, we propose a ternary spike neuron to transmit information. The ternary spike neuron can also enjoy the event-driven and multiplication-free operation advantages of the binary spike neuron but will boost the information ca- pacity. Furthermore, we also embed a trainable factor in the ternary spike neuron to learn the suitable spike amplitude, thus our SNN will adopt different spike amplitudes along layers, which can better suit the phenomenon that the membrane po- tential distributions are different along layers. To retain the efficiency of the vanilla ternary spike, the trainable ternary spike SNN will be converted to a standard one again via a re- parameterization technique in the inference. Extensive experi- ments with several popular network structures over static and dynamic datasets show that the ternary spike can consistently outperform state-of-the-art methods.
I wasn't knowledgeable enough to evaluate that claim at the time, and I'm still not.
In binary, with two inputs, there are 2^2 = 4 total possible inputs (00, 01, 10, 11). Different gate types can give different outputs for each of those four inputs: each output can be 0 or 1, so that's 2^4 == 16 different possible gate types. (0, 1, A, B, not A, not B, AND, OR, NAND, NOR, XOR, XNOR, A and not B, B and not A, A or not B, B or not A)
In ternary, with two inputs, there are 3^2 = 9 total possible inputs, so 3^9 = 19,683. I'm sure there are some really sensible ones in there, but damn that's a huge search space. That's where I gave up that time around! :-)