Archit Sharma

Posted on Jul 25, 2022

Why do computers suck at math?

#javascript #webdev #beginners #programming

I discovered something strange when experimenting with JavaScript a few days ago, so I did some research and found my answers.

I'm writing this article in the hopes that some of you will find it useful or may learn something new.

I used to believe that computers were better at math than humans, until I tried adding 0.1 + 0.2 and got the result 0.30000000000000004 in my browser console.

Then I performed this comparison 0.1+0.2===0.3 and got the output false.

I initially believed it was a bug in JavaScript, so I attempted the same thing in Java and Python and had the same result in both.

After doing a lot of research, I concluded this is not an bug.
I found out this is math: floating-point arithmetic.
Let's go a little deeper to see what's going on behind the scenes.

Computers have a limited amount of memory and so they need to make a trade-of between range and precision.

Numbers in JavaScript should be stored within 64 bits, which means we can have integers accurate up to 15 digits and a maximum of 17 numbers after the decimal point. It is called a floating point because there is no fixed number of digits before or after the decimal point, allowing it to represent a wide range of numbers both big and small.

The problem is that computer use a Base-2 system i.e. binary while humans use a Base-10 system that leads to rounding errors when all the memory has been used up.

This is the reason behind getting 0.1 + 0.2 = 0.30000000000000004.

Thank you for reading this article; I have not detailed the entire math behind it, but only enough for you to comprehend what's going on.

Top comments (9)

Ryan Brown • Jul 25 '22

Back when my org still had Programming Summer Students/Interns, this usually came up every other year.

Student uses float for money values
Total of extended price: sum(qty*price) != Expected Value
Talk about precision with regard to basic types and why many languages have a Decimal type

Archit Sharma • Jul 26 '22

wow that's cool

FJones • Jul 26 '22

This is actually missing a huge lot of context. Base2 vs Base10 has little to do with floating-point precision (in fact, Base2 has very much the same concept of fractions as Base10, i.e. 0.1₂ is ½.)

The problem doesn't come from binary, it stems from the fact that binary needs a lot more positions to represent values than decimal, which makes fractions a particularly nasty scaling issue. Floating-point values are encoded in a very different format, which optimizes for space efficiency at the cost of accuracy, hence the funny edge cases. (Though per the specification of that encoding, there's no real precise values anyway, barring 0)

Peter Vivo • Jul 26 '22

I use this solution for give limit decimal place.

export const epsilonN = N => num => Math.round( num * N + Number.EPSILON ) / N;
export const epsilon = epsilonN(1e7);
export const epsilon2 = epsilonN(1e2);
export const epsilon3 = epsilonN(1e3);

epsilon(.1 + .2);
// 0.3

Alex Lohr • Jul 26 '22

It's actually the opposite: floating point maths sucks at being expressed precisely with limited size binary numbers.

Jon Randy 🎖️ • Jul 26 '22

Being a sophomore in Computer Science (according to your bio) - I am shocked that you did not know this. What has happened to teaching of CS???

Archit Sharma • Jul 26 '22

I am student of Computer Science Engineering!!!
The core studies begin in the second year of college; the first year was devoted to common engineering disciplines (Physics, Chemistry, Math etc.)
Everything I've learnt thus far is on my own.
and I didn't know this was so common...🤷‍♂️

Jon Randy 🎖️ • Jul 26 '22

TBH I don't have a degree, but I do know plenty of people who did CS degrees and they all know this. I learned this about age 11 (35 years ago! Seems like forever)

Archit Sharma • Jul 26 '22

Well, I'm sure none of the folks you know got their CS degree from India.

DEV Community

Why do computers suck at math?

I discovered something strange when experimenting with JavaScript a few days ago, so I did some research and found my answers.

I'm writing this article in the hopes that some of you will find it useful or may learn something new.

Thank you for reading this article; I have not detailed the entire math behind it, but only enough for you to comprehend what's going on.

Top comments (9)

Read next

M-Pesa Express (STK Push) API Guide

TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters

New Research Breaks Through AI Language Model Safeguards, Exposing Security Risks

xLSTM: Fast and Efficient Large Recurrent Action Model for Robotics