9,999,999,999,999,978 and 1 – 2 incorrect on iOS Calculator

2 points by windows2020 4y ago ↗ HN

A few observations (landscape mode only): * The above results in 9,999,999,999,999,978. * Add 1 a bunch of times and subtract 2 once to reach the same number. * Add 1 a bunch of times and subtract 1 a bunch of times and the answer is still displayed in scientific notation, until 2 is subtracted, at which point the above number is displayed.

5 comments

[ 2.9 ms ] story [ 23.7 ms ] thread

pwg 4y ago ↗

You should read this:

What Every Computer Scientist Should Know About Floating-Point Arithmetic

https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.h...

yorwba 4y ago ↗

Then learn how to do better: https://cacm.acm.org/magazines/2017/8/219594-small-data-comp... (The Android calculator app uses exact arithmetic on arbitrary-precision real numbers.)

fsflyer 4y ago ↗

Microsoft open sourced the Windows calculator in 2019. Ratpak[0] is their infinite precision math library.

[0] https://github.com/microsoft/calculator/tree/master/src/Calc...

[deleted] 4y ago ↗

(comment deleted)

ksaj 4y ago ↗

This reminds me of the Intel FDIV bug.

https://en.wikipedia.org/wiki/Pentium_FDIV_bug

As well as the Windows 3.1 Calc.exe bug that would give bizarre results if you subtracted close decimals like 1.111 - 1.011.