Yeah, I'm aware, but you need to accurately compute abs/magnitude, which presents its own complexities.
Including exponentials, it's highly related to the non-trivial accurate computation of trigonometric functions of floats.
And there are algorithms that rely on this, like this[1] branchless axis-aligned bounding box intersection test.
It exploits the division by zero sign logic of IEEE-754, so that the min/max operations used also handles the edge cases where the ray is parallel to an axis.
+/- Infinity seems to be correct according to IEEE754.
IEEE754 section 7.3:
The divideByZero exception shall be signaled if and only if an exact infinite result is defined for an
operation on finite operands. The default result of divideByZero shall be an ∞ correctly signed according to
the operation:
― For division, when the divisor is zero and the dividend is a finite non-zero number, the sign of the
infinity is the exclusive OR of the operands’ signs (see 6.3).
― For logB(0) when logBFormat is a floating-point format, the sign of the infinity is minus (−∞).
Haha, -0. After spending literal years of my life dealing with that little encoding nuisance, in everything from the V8 TurboFan JIT, to WebAssembly, to adding floating point to Virgil, I just gotta say, that little turd is so not worth it. I've read a lot of different justifications for it (it's a limit, it represents approaching 0 from the negatives, some algorithms need it, etc), I have to admit, I just don't buy it. It's an encoding artifact that's produced so much accidental complexity that, in my book, has been far more trouble than it's been worth.
We cannot represent "correct math" on a computer, because the set of reals is uncountable, but in 64 bits we can only represent finitely many numbers, not infinitely many, let alone uncountably many.
And given that we have to implement some sort of approximation to the usual math on a closed subset of the reals (rationals, even), why not extend it with some "numbers" (negative zero, plus/minus infinity, NaNs) that, yes, enable programming convenience?
There is a difference between 2.4987665678 as 2.5 and undefined as ±infinity especially because 1.0/+0.0 is +infinity and 1/0 is just an exception in Java.
Shouldn't it be the same result?
But, bottom line, Kahan (not only the father of IEEE 754, but also, with Golub, father of the SVD!) has thought about it a lot, and though he called the signed zero "a pain in the ass", he also said "There really wasn’t a way around that and you were stuck with it."
Which languages did you try? I found that dividing 1.0 by 0.0 gives Infinity in Java, C, Julia, JavaScript, Ruby, Clojure, Haskell and Rust. I get division errors in Python and Perl.
Curiously, Go gives me a compiler error if I divide constant literals, but assigning 0.0 to a variable and dividing by it does work. On top of that, assigning -0.0 to a float variable leads to it having the value +0.0, which I would call a borderline compiler bug. Both are probably a consequence of Go constant literals not being IEEE-754 floats but some arbitrary precision number types, but it's very unexpected behavior.
> In most languages I've used you get a division by zero error or NaN
Or it raises an exception, which once disabled a navy warship for hours:
> Now decommissioned, the USS Yorktown was among the first warhips extensively computerized to reduce crew (by 10% to 374) and costs (by $2.8 million per year).
> On 21 Sept. 1997, the Yorktown was maneuvering off the coast of Cape Charles, VA, when a crewman accidentally ENTERed a blank field into a data base. The blank was treated as a zero and caused a Divide-by-Zero Exception which the data-base program could not handle. It aborted to the operating system, Microsoft Windows NT 4.0, which crashed, bringing down all the ship’s LAN consoles and miniature remote terminals.
> The Yorktown was paralyzed for 2 3/4 hours,
unable to control steering, engines or weapons, until the operating system had been re-booted. Fortunately the Yorktown was not in combat nor in crowded shipping lanes.
> If IEEE 754’s default had been in force, the division by zero would have insinuated into the data-base an ∞ and/or NaN, which would have been detected afterwards without a crash.
Eh, there's like six separate issues that led to this.
1. Interface accepted blank value for a form when a numeric value is required.
2. Application and database permitted a null or coerced a zero when the value is required and cannot be zero. You've got to sanitize your input.
3. Application did not check for or gracefully handle something as common as a division by zero exception. Instead it proceeded to a buffer overrun and crashed.
4. Application crashing apparently caused an OS crash? Yes, okay, it's NT 4.0 and a buffer overrun, but that still should not happen. Protected mode is a thing.
5. There is no redundancy in a system that manages the steering, weapons, and engines of a warship.
6. A warship was unable to restore the single CNC system to working order for 23 hours.
Furthermore... there's no evidence that the software would have behaved more sensibly or more controllably if it had a value of infinity instead of an error. It shouldn't have had a zero in the first place.
> Does this correctly handle all the different NaN values, weird corner cases, etc. properly?
Of course the answer to "have you covered all weird corner cases" is always 'no', but the article addresses the issue of different NaNs:
> […] However, it turns out that doubleToLongBits is also not entirely trivial, because it canonicalizes NaNs. There are many ways to encode a not-a-number as a double, but only one of them is canonical. These different NaNs are even more similar twins, they can be distinguished neither via Double.compare, nor via any operation. Even string representation is the same. But they look different in computer memory. To avoid surprises, doubleToLongBits converts any NaN to the canonical form, which is encoded in long as 0x7ff8000000000000L. Of course, this procedure adds more conditions, which we do not need here.
> What can we do? It appears that there's another method doubleToRawLongBits. It doesn't do any smart conversion of NaN and just returns exactly the same bit representation: […]
It's the required semantics of the IEEE floating point spec's absolute-value operation. This does mean that abs() doesn't behave like eg addition in that it won't set fp exception flags for signaling NaNs, or canonicalize NaNs, or set a flag to note that this was a denormal number. But at the language level "follows IEEE semantics" is probably overall less surprising.
chs() ("flip the sign bit") is the other operation that's defined as effectively just an operation on the representation rather than a "do a real floating point computation".
For double it would be harder to check but for float (that should have the same corner cases) it's easy to just check all possible values if you have a gold standard. I've been doing that for image processing functions and it's amazing how many corner cases you fix and keep fixed by writing unit tests that iterate over all possible values or a good subset of them when that's not possible.
I second "just try all of them" when testing single precision floats. It's really easy to do, and gives you a lot of confidence in the correctness. For example, I had a float-to-bytes-to-floats conversion in some GPU shaders and "just test that every single possible float survives a round trip" made it trivial to be sure it was correct. (Also I found out my video card drivers were losing an ulp when parsing float constants in shaders.)
Roundtrip tests are great when you don't have a gold standard. It allows for at least consistency across operations. I've done a bunch and found quite a few corner cases. I also did a few roundtrip tests in RGB spaces where being exhaustive was no longer feasible. For those I ended up iterating over the whole RGB space but skipping values in each component by different prime values, to try and cover as many of the possible weird cases as possible:
It does that because of the way they implement arithmetic, not about an inherent property of the mathematical function `max`. The problem has to be solved once.
But if x is -0.0, you're back to the same problem that -0.0 >= +0.0 actually turns out to be true and not false, hence the return value will be -0.0.
Looking at the actual implementation[0], doubleToRawLongBits is used again here to take care of this edge case. So it would work, but a lot slower than the version given in the article.
Positive and negative zero cannot be distinguished by normal equality checks, meaning +0.0 == -0.0. Therefore, for consistency's sake +0.0 > -0.0 ought to be false, which would yield a return value of -0.0 for max(+0.0, -(+0.0))
Why do you say that? It extracts the bits using doubleToRawLongBits which does not do NaN canonicalization by design. And it gets back to a double with longBitsToDouble which "returns the double floating-point value with the same bit pattern". So it doesn't change anything either.
You made a good point. But one might argue that -nan should become nan after an abs operation. I don't know enough about this topic to determine whether such an argument has merit though.
It's not -NaN vs NaN. There are a bunch of NaN types, which are not related like that. It could (according to the spec) change "NaN: Attempt to arcsin value greater than 1" to "NaN: Zero divided by Zero operation"
The sign of the NaN might change (as you'd expect from an absolute value), but the NaN payload (which IEEE 754 envisaged might carry debug information) won't change, right?
The NaN payload's debug information is allowed to use the sign bit as part of the debug code (according to my knowledge of IEEE 754). Personally, I cannot imagine needing that many debug codes in most common usage, and I don't know if many implementations do, but it would definitely be something to check in your specific implementation details.
If you're packing values into NaNs it's easy to end up using all the bits. For example, if you start storing 48 bit pointers you only have 4 bits plus the sign bit left over for the type of the value.
This is true. I doubt those are the result of an operation however (what system creates NaNs like that?). It sounds like that is interesting data manipulation.
> Imagine if a NaN said exactly where in your code it was created.
I'm not sure how that would be useful. A debugger should have better ways of identifying NaN creation and I don't see anything else using a pointer to code in production. It seems cool though.
It would also lead to problems like:
x = 5.0 * y
y is NaN with some address as the NaN type.
NaN * 5.0 => NaN. Which address (the original or the * 5.0 line) is used?
> I don't see anything else using a pointer to code in production.
If you log it, you could trace it back to the line of code at your leisure.
> It would also lead to problems like: x = 5.0 * y y is NaN with some address as the NaN type. NaN * 5.0 => NaN. Which address (the original or the * 5.0 line) is used?
I don't mean to be rude, but isn't the answer to that question extremely obvious?
The behavior that's much more useful and enshrined in the IEEE spec is that you keep the original NaN.
If you log it, why wouldn't you just log it at creation time. Are you going to periodically scan all floating-point numbers used everywhere in your code to see where they are created?
> The behavior that's much more useful and enshrined in the IEEE spec is that you keep the original NaN.
I didn't realize it was in the spec. I just checked, and you're right! That does however seem strange - X was never set by operation NNNN, Y was.
To me, trying to handle exceptions every time a NaN shows up sounds like it's riskier to performance than a few checks at important places. You wouldn't scan. Probably depends on code base.
> That does however seem strange - X was never set by operation NNNN, Y was.
If you're using NaN payloads, it would almost never be helpful for 95% of your NaNs to turn into a generic "used a NaN in arithmetic" version.
It’s interesting how fairly complex bit twiddling hacks are well-known for integers, but even very simple but useful bit twiddles for floats (like this one) are obscure enough to warrant their own article.
Of course, the most famous bit manipulation of floats is the fast inverse square root trick [0]. The bits of a float as an integer literal are proportional to its log2 value (plus a constant), which is handy for computing a reciprocal square root in logspace, which when converted back to a float gets exponentiated, yielding the answer (after a couple more refinement steps). If you actually take the time to manually hammer out the bitwise manipulations, the fast inverse square root is quite simple, but it is nonetheless regarded as black magic, likely due to the totally non-explanatory comments in the Quake source code.
Funnily enough, directly following the fast inverse square root in the Quake source code is a function to compute absolute values by zeroing the float sign bit [1], just as described in the article.
Given the amount of discussion this article has sparked on HN, I’d argue that as obvious as it might be, the technique is nonetheless obscure. The amount of highly experienced and accomplished programmers I’ve encountered who have no idea how floating point numbers work is astoundingly high.
If a similar article were written about how shifting an integer to the right by one bit is equivalent to dividing it by two, I highly suspect it would attract no attention on this site, aside from a couple dismissive comments grousing about why such a basic fact needs its own article.
You can of course have some idea how floating point numbers work, but still not know how to bit twiddle abs().
I don't know what to take from the fact that Java stdlib didn't use it until Java 18? It does seem like it wasn't obvious to at least someone who knew something.
> If a similar article were written about how shifting an integer to the right by one bit is equivalent to dividing it by two, I highly suspect it would attract no attention on this site,
Nah, it would be full of comments of people saying "well of course I knew that! doesn't everyone who's anyone? I can't believe how many newbies don't though!" :)
Well, with bit twiddling for floats, you have some very important questions. Like, do you maintain the distinct NaN values or are you okay with all NaNs producing a valid (but possibly different) NaN value.
Bit twiddling integers doesn't require unsafe type punning like this. It's also largely safe to assume a nominal two's complement machine is all your code will ever run on. If not, you'll know to be careful. You can't make such blanket assumptions with floating point representations.
Great points. Out of curiosity, what are some examples of modern architectures that don’t use IEEE 754 floats? Realistically, how often would one ever write code that needs to be portable between an architecture that uses IEEE 754 and one that doesn’t? I could only imagine this ever being an issue for code that needs to be deployed to a wide variety of exotic embedded systems, which is an extremely niche case IMO.
>what are some examples of modern architectures that don’t use IEEE 754 floats
Mainstream? I can't think of one. Googling the question yields a stackoverflow question from 2010 that lists the 5 or 6 major architectures (x86, PowerPc, SPARC, Arm,.. ) and about a dozen more as supporting x86. So it's a fairly safe assumption to make.
One area where vanilla IEEE754 isn't used that immediately came to my mind is high performance ML at companies like facebook and Google, the generic range-precision tradeoff that FP offers is inefficient and they ended up designing several new formats with smaller sizes or larger ranges. Googling "Alternatives to FP in ML" or "Custom FP format <company-name>" yields a lot.
Another "niche" area is finance, insurance, taxes and anything nearby, fixed point in those area is, I hear, a must. Because all the usual invariants of high school algebra that FP breaks actually have significance. (imagine the well-known FP flaw "x+epsilon == x, with epsilon non-zero" playing out in these contexts.)
In any case, number portability in a language like C++ is a breeze. You design a myNumber class and overload all relevant operators and that's it. Compiler will make sure to inline the calls if they're a one-line simple floating point operations, but otherwise you can freely change the implementation of the class and the calculations never look different.
> Another "niche" area is finance, insurance, taxes and anything nearby, fixed point in those area is, I hear, a must. Because all the usual invariants of high school algebra that FP breaks actually have significance.
In particular, double-entry bookkeeping (standard practice for centuries) uses the accounting equation [1] as a checksum. If the two sides of the equation are ever unequal, then there's been a mistake somewhere that needs to be corrected.
If you add the same value to a large asset account and a small liability account, for example, you can run into problems: The internal rounding performed by floating point arithmetic can mean that the increase to each side of the equation is slightly different, throwing the system out of balance.
> what are some examples of modern architectures that don’t use IEEE 754 floats?
Lots of medical devices support floating point format from IEEE-11073 instead. I'd guess there are a significant number of them in existence and they need to be done well.
In addition to sibling comments, two more oddballs are MSP430 floats and DSPs.
MSP430-style floats are common in industrial automation and such, since many sensors either use or used to use MSP430s to collect data. And, of course, in that field, legacy carries more inertia than a 50,000-ton press....
Newer DSPs tend to use IEEE-754 floats, but this was not always the case. Older DSPs used every trick in the book to go faster, and "compatibility with standard floating-point formats" was never in their product requirements. So you can guess at what manner of horrors came out of that! Fortunately, their customers are mostly the sort of people who (1) didn't mind changing to standard floats, since it's not like things were stable before; and (2) are very, very comfortable writing this stuff directly in assembly. So they definitely know whether their architecture has an instruction to take the absolute value, needs a short branching program, or just needs a single bit-operation.
>Bit twiddling integers doesn't require unsafe type punning like this.
Not only that but, at least for Fast Inverse Square Root, it's not advantageous to carry out the bit twiddling since the rsqrtss instruction is faster and more precise, and vectorized instructions are faster still for large strings of floats.
Just convert to string then replace the "-" char with empty, then you can convert to the type you want. Sure performance suffers, but it gets you home by quitting time. Don't actually do this.
> After such interviews, there will be rumors about your company.
It makes no sense to bring this up in interview questions. Just about every programmer knows that IEEE 754 contains some quirks. If you want to know the boring details, just look them up in the manual.
If the company only needs cowboy programming, corner cases don't matter.
For some industries, ensuring your code doesn't fail on corner cases is very important, and questions about quirks show whether a candidate cares about corner cases and their comprehension of them.
Given that the limitations of IEEE754 are the same in in virtually all programming languages, I suspect it is actually really good to use floating point quirks to probe a candidate on how skillful they are at programming.
I roll my eyes at some stack overflow answers for their complete failure to consider even common corner cases like NaN and Infinity.
I have certainly been caught out by my own mistakes in my own code with with integer and floating point errors, and I think I am more careful than most.
Yes. For working, say, with physical simulations an experienced worker should know IEEE754. It does not take more than half a day of learning the standard to know how to approach the problem of the post with confidence.
The real quirks lie elsewhere, in the hardware implementations and rounding rules (e.g. does FMA have the same precision of multiply-then-add? why does the order of the elements of a sum make a difference? etc.). And once you know these you can treat floating points with familiarity and expect your operations to be predictable to their last bit.
If I were hiring for a developer of FDTD simulations I would expect them to be familiar with things like -0.0, subnormal numbers and NaN types and to be able to learn if necessary how to go around the murky stuff like rounding. Floats shouldn't be mysterious, they are at the end 16/32/64/80/128 bit variables with deterministic operations defined on them.
Computing the absolute value of a signed (fixed-size, two's complement) integer is not trivial either, because there is no positive i such that `abs(INT_MIN) == i`. Some languages like Java just give you `abs(INT_MIN) == INT_MIN` because:
-100…000 = ~100…000 + 1 = 011…111 + 1 = 100…000
where ~ is the bitwise complement. Even worse, in C and C++ specifically, because overflow of signed integers is undefined behavior, so is `abs(INT_MIN)`! To be able to represent the absolute value, you need to either return the corresponding unsigned type (if your language has them), promote to a wider type, or return a tuple `(carry_bit, result)`.
This is why Rust has `abs`, `checked_abs`, `wrapping_abs`, `overflowing_abs`, `saturating_abs` and `unsigned_abs`.
Checked returns a Result enum that is an error on overflow, wrapping silently wraps (as in your answer), overflowing wraps but has a boolean indicator whether a wrap has occurred, saturating bounds at the types limits rather than wrapping and unsigned abs always gives the correct result albeit as a different type.
The unqualified abs panics on overflow on debug but silently wraps on release mode, like all Rust arithmetic code does by default.
In programming, the result of a comparison of floating point numbers in general and a comparison with zero in particular is problem (and scale) dependent. Some values that are close to zero are actually zero, within a certain margin of error (the “epsilon”). I wish computers directly operated on intervals instead of single points.
The motivation for the article is this change in OpenJDK, where they got a 10% performance improvement by switching from the single-branch implementation to the branchless one:
The article is about how the Java standard library implements its equivalent of fabs() and why it changed from a careful and clever comparison to bit twiddling
109 comments
[ 2.6 ms ] story [ 155 ms ] threadThis must be a Java-ism. In most languages I've used you get a division by zero error or NaN.
It exploits the division by zero sign logic of IEEE-754, so that the min/max operations used also handles the edge cases where the ray is parallel to an axis.
[1]: https://tavianator.com/2011/ray_box.html
IEEE754 section 7.3:
The divideByZero exception shall be signaled if and only if an exact infinite result is defined for an operation on finite operands. The default result of divideByZero shall be an ∞ correctly signed according to the operation: ― For division, when the divisor is zero and the dividend is a finite non-zero number, the sign of the infinity is the exclusive OR of the operands’ signs (see 6.3). ― For logB(0) when logBFormat is a floating-point format, the sign of the infinity is minus (−∞).
You'd get NaN from 0.0/0.0 or division by zero if you did 1/0 as integer division.
Change my mind.
https://people.freebsd.org/~das/kahan86branch.pdf from the father of ieee-754 is illuminating.
And given that we have to implement some sort of approximation to the usual math on a closed subset of the reals (rationals, even), why not extend it with some "numbers" (negative zero, plus/minus infinity, NaNs) that, yes, enable programming convenience?
https://hackernoon.com/negative-zero-bbd5fd790af3
But, bottom line, Kahan (not only the father of IEEE 754, but also, with Golub, father of the SVD!) has thought about it a lot, and though he called the signed zero "a pain in the ass", he also said "There really wasn’t a way around that and you were stuck with it."
http://history.siam.org/pdfs2/Kahan_final.pdf
1/ (+0) = +∞ but 1/ (–0) = –∞
but not that there be two, say, 3s so that
1/ (3⁺ – 3) = +∞ but 1/ (3⁻ – 3) = –∞
It's not that it's important there be two, just every number already had 2 except for 0.
Curiously, Go gives me a compiler error if I divide constant literals, but assigning 0.0 to a variable and dividing by it does work. On top of that, assigning -0.0 to a float variable leads to it having the value +0.0, which I would call a borderline compiler bug. Both are probably a consequence of Go constant literals not being IEEE-754 floats but some arbitrary precision number types, but it's very unexpected behavior.
Python and C#.
I hope others will take the entire thread into consideration before downvoting.
Or it raises an exception, which once disabled a navy warship for hours:
> Now decommissioned, the USS Yorktown was among the first warhips extensively computerized to reduce crew (by 10% to 374) and costs (by $2.8 million per year).
> On 21 Sept. 1997, the Yorktown was maneuvering off the coast of Cape Charles, VA, when a crewman accidentally ENTERed a blank field into a data base. The blank was treated as a zero and caused a Divide-by-Zero Exception which the data-base program could not handle. It aborted to the operating system, Microsoft Windows NT 4.0, which crashed, bringing down all the ship’s LAN consoles and miniature remote terminals.
> The Yorktown was paralyzed for 2 3/4 hours, unable to control steering, engines or weapons, until the operating system had been re-booted. Fortunately the Yorktown was not in combat nor in crowded shipping lanes.
> If IEEE 754’s default had been in force, the division by zero would have insinuated into the data-base an ∞ and/or NaN, which would have been detected afterwards without a crash.
http://people.eecs.berkeley.edu/~wkahan/Boulder.pdf
https://www.wired.com/1998/07/sunk-by-windows-nt/
EDIT: It was 2 3/4 hours, not 23 hours. I copied/pasted carelessly from the PDF.
1. Interface accepted blank value for a form when a numeric value is required.
2. Application and database permitted a null or coerced a zero when the value is required and cannot be zero. You've got to sanitize your input.
3. Application did not check for or gracefully handle something as common as a division by zero exception. Instead it proceeded to a buffer overrun and crashed.
4. Application crashing apparently caused an OS crash? Yes, okay, it's NT 4.0 and a buffer overrun, but that still should not happen. Protected mode is a thing.
5. There is no redundancy in a system that manages the steering, weapons, and engines of a warship.
6. A warship was unable to restore the single CNC system to working order for 23 hours.
Furthermore... there's no evidence that the software would have behaved more sensibly or more controllably if it had a value of infinity instead of an error. It shouldn't have had a zero in the first place.
Of course the answer to "have you covered all weird corner cases" is always 'no', but the article addresses the issue of different NaNs:
> […] However, it turns out that doubleToLongBits is also not entirely trivial, because it canonicalizes NaNs. There are many ways to encode a not-a-number as a double, but only one of them is canonical. These different NaNs are even more similar twins, they can be distinguished neither via Double.compare, nor via any operation. Even string representation is the same. But they look different in computer memory. To avoid surprises, doubleToLongBits converts any NaN to the canonical form, which is encoded in long as 0x7ff8000000000000L. Of course, this procedure adds more conditions, which we do not need here.
> What can we do? It appears that there's another method doubleToRawLongBits. It doesn't do any smart conversion of NaN and just returns exactly the same bit representation: […]
The sign bit is totally independent of the value being encoded, whether that’s a number, q/sNaN, or infinity.
Nitpick: we are not flipping the sign bit, but rather setting it to 0.
chs() ("flip the sign bit") is the other operation that's defined as effectively just an operation on the representation rather than a "do a real floating point computation".
https://github.com/pedrocr/imagepipe/blob/12269f04ac2fa0d165...
Try it and you'll see.
Looking at the actual implementation[0], doubleToRawLongBits is used again here to take care of this edge case. So it would work, but a lot slower than the version given in the article.
0: https://github.com/openjdk/jdk/blob/36e2ddad4d2ef3ce27475af6...
(changing greater equal to just greater). Would the problem then still persist?
The problem is that 0.0 == -0.0. As long as you don't take care of signed zero, you cannot guarantee that abs(x) has no negative sign.
I don't know if anything does but it could certainly be useful. Imagine if a NaN said exactly where in your code it was created.
I'm not sure how that would be useful. A debugger should have better ways of identifying NaN creation and I don't see anything else using a pointer to code in production. It seems cool though.
It would also lead to problems like: x = 5.0 * y y is NaN with some address as the NaN type. NaN * 5.0 => NaN. Which address (the original or the * 5.0 line) is used?
If you log it, you could trace it back to the line of code at your leisure.
> It would also lead to problems like: x = 5.0 * y y is NaN with some address as the NaN type. NaN * 5.0 => NaN. Which address (the original or the * 5.0 line) is used?
I don't mean to be rude, but isn't the answer to that question extremely obvious?
The behavior that's much more useful and enshrined in the IEEE spec is that you keep the original NaN.
> The behavior that's much more useful and enshrined in the IEEE spec is that you keep the original NaN.
I didn't realize it was in the spec. I just checked, and you're right! That does however seem strange - X was never set by operation NNNN, Y was.
> That does however seem strange - X was never set by operation NNNN, Y was.
If you're using NaN payloads, it would almost never be helpful for 95% of your NaNs to turn into a generic "used a NaN in arithmetic" version.
Of course, the most famous bit manipulation of floats is the fast inverse square root trick [0]. The bits of a float as an integer literal are proportional to its log2 value (plus a constant), which is handy for computing a reciprocal square root in logspace, which when converted back to a float gets exponentiated, yielding the answer (after a couple more refinement steps). If you actually take the time to manually hammer out the bitwise manipulations, the fast inverse square root is quite simple, but it is nonetheless regarded as black magic, likely due to the totally non-explanatory comments in the Quake source code.
Funnily enough, directly following the fast inverse square root in the Quake source code is a function to compute absolute values by zeroing the float sign bit [1], just as described in the article.
[0] https://en.wikipedia.org/wiki/Fast_inverse_square_root#Alias...
[1] https://github.com/id-Software/Quake-III-Arena/blob/dbe4ddb1...
If a similar article were written about how shifting an integer to the right by one bit is equivalent to dividing it by two, I highly suspect it would attract no attention on this site, aside from a couple dismissive comments grousing about why such a basic fact needs its own article.
I don't know what to take from the fact that Java stdlib didn't use it until Java 18? It does seem like it wasn't obvious to at least someone who knew something.
> If a similar article were written about how shifting an integer to the right by one bit is equivalent to dividing it by two, I highly suspect it would attract no attention on this site,
Nah, it would be full of comments of people saying "well of course I knew that! doesn't everyone who's anyone? I can't believe how many newbies don't though!" :)
Mainstream? I can't think of one. Googling the question yields a stackoverflow question from 2010 that lists the 5 or 6 major architectures (x86, PowerPc, SPARC, Arm,.. ) and about a dozen more as supporting x86. So it's a fairly safe assumption to make.
One area where vanilla IEEE754 isn't used that immediately came to my mind is high performance ML at companies like facebook and Google, the generic range-precision tradeoff that FP offers is inefficient and they ended up designing several new formats with smaller sizes or larger ranges. Googling "Alternatives to FP in ML" or "Custom FP format <company-name>" yields a lot.
Another "niche" area is finance, insurance, taxes and anything nearby, fixed point in those area is, I hear, a must. Because all the usual invariants of high school algebra that FP breaks actually have significance. (imagine the well-known FP flaw "x+epsilon == x, with epsilon non-zero" playing out in these contexts.)
In any case, number portability in a language like C++ is a breeze. You design a myNumber class and overload all relevant operators and that's it. Compiler will make sure to inline the calls if they're a one-line simple floating point operations, but otherwise you can freely change the implementation of the class and the calculations never look different.
In particular, double-entry bookkeeping (standard practice for centuries) uses the accounting equation [1] as a checksum. If the two sides of the equation are ever unequal, then there's been a mistake somewhere that needs to be corrected.
If you add the same value to a large asset account and a small liability account, for example, you can run into problems: The internal rounding performed by floating point arithmetic can mean that the increase to each side of the equation is slightly different, throwing the system out of balance.
[1] https://en.wikipedia.org/wiki/Accounting_equation
Lots of medical devices support floating point format from IEEE-11073 instead. I'd guess there are a significant number of them in existence and they need to be done well.
MSP430-style floats are common in industrial automation and such, since many sensors either use or used to use MSP430s to collect data. And, of course, in that field, legacy carries more inertia than a 50,000-ton press....
Newer DSPs tend to use IEEE-754 floats, but this was not always the case. Older DSPs used every trick in the book to go faster, and "compatibility with standard floating-point formats" was never in their product requirements. So you can guess at what manner of horrors came out of that! Fortunately, their customers are mostly the sort of people who (1) didn't mind changing to standard floats, since it's not like things were stable before; and (2) are very, very comfortable writing this stuff directly in assembly. So they definitely know whether their architecture has an instruction to take the absolute value, needs a short branching program, or just needs a single bit-operation.
Not only that but, at least for Fast Inverse Square Root, it's not advantageous to carry out the bit twiddling since the rsqrtss instruction is faster and more precise, and vectorized instructions are faster still for large strings of floats.
I watched a video[0] a while back about that. Once that coincidence(?) was explained, Q_rsqrt finally made sense.
[0]: https://youtu.be/p8u_k2LIZyo (~20 minutes)
Regarding signed zero, read this paper from Kahan: https://homes.cs.washington.edu/~ztatlock/599z-17sp/papers/b...
0/(-x) would return -0, for all x > 0
Likewise, underflows on negative numbers would also return -0.
I find this one a little weird, but it seems to be true.
The more normal way to get -0 would be to divide a positive number by negative infinity, or a negative number by positive infinity.
It makes no sense to bring this up in interview questions. Just about every programmer knows that IEEE 754 contains some quirks. If you want to know the boring details, just look them up in the manual.
For some industries, ensuring your code doesn't fail on corner cases is very important, and questions about quirks show whether a candidate cares about corner cases and their comprehension of them.
Given that the limitations of IEEE754 are the same in in virtually all programming languages, I suspect it is actually really good to use floating point quirks to probe a candidate on how skillful they are at programming.
I roll my eyes at some stack overflow answers for their complete failure to consider even common corner cases like NaN and Infinity.
I have certainly been caught out by my own mistakes in my own code with with integer and floating point errors, and I think I am more careful than most.
The real quirks lie elsewhere, in the hardware implementations and rounding rules (e.g. does FMA have the same precision of multiply-then-add? why does the order of the elements of a sum make a difference? etc.). And once you know these you can treat floating points with familiarity and expect your operations to be predictable to their last bit.
If I were hiring for a developer of FDTD simulations I would expect them to be familiar with things like -0.0, subnormal numbers and NaN types and to be able to learn if necessary how to go around the murky stuff like rounding. Floats shouldn't be mysterious, they are at the end 16/32/64/80/128 bit variables with deterministic operations defined on them.
But as they are corner cases, you are very unlikely to just know them all from memory, so you would need to look them up anyways.
Checked returns a Result enum that is an error on overflow, wrapping silently wraps (as in your answer), overflowing wraps but has a boolean indicator whether a wrap has occurred, saturating bounds at the types limits rather than wrapping and unsigned abs always gives the correct result albeit as a different type.
The unqualified abs panics on overflow on debug but silently wraps on release mode, like all Rust arithmetic code does by default.
if (value < 0) return -value; else if (value > 0) return value; else return +0.0
The motivation for the article is this change in OpenJDK, where they got a 10% performance improvement by switching from the single-branch implementation to the branchless one:
https://github.com/openjdk/jdk/pull/4711
When I need absolute value, I usually do it like that in C++: