• gravitas_deficiency@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    1
    ·
    edit-2
    27 days ago

    Look at what they need to mimic a fraction inaccurately.

    The fundamental mathematical nature of how binary floating point values are stored means that extremely straightforward and rational (in the mathematical sense of the term) base-10 arithmetic can surprisingly often yield results that are irrational (again, mathematically) in binary - hence why you’ll sometimes see a result of 3.000000000101325 or something like that in places where you’d expect the result to be simply 3.0