Info Catalog gawk: Inexactness of computations gawk: FP Math Caution gawk: Try To Round

gawk: Getting Accuracy

 
 15.4.2 Getting the Accuracy You Need
 ------------------------------------
 
 Can arbitrary-precision arithmetic give exact results?  There are no
 easy answers.  The standard rules of algebra often do not apply when
 using floating-point arithmetic.  Among other things, the distributive
 and associative laws do not hold completely, and order of operation may
 be important for your computation.  Rounding error, cumulative precision
 loss, and underflow are often troublesome.
 
    When 'gawk' tests the expressions '0.1 + 12.2' and '12.3' for
 equality using the machine double-precision arithmetic, it decides that
 they are not equal!  (Comparing FP Values.)  You can get the
 result you want by increasing the precision; 56 bits in this case does
 the job:
 
      $ gawk -M -v PREC=56 'BEGIN { print (0.1 + 12.2 == 12.3) }'
      -| 1
 
    If adding more bits is good, perhaps adding even more bits of
 precision is better?  Here is what happens if we use an even larger
 value of 'PREC':
 
      $ gawk -M -v PREC=201 'BEGIN { print (0.1 + 12.2 == 12.3) }'
      -| 0
 
    This is not a bug in 'gawk' or in the MPFR library.  It is easy to
 forget that the finite number of bits used to store the value is often
 just an approximation after proper rounding.  The test for equality
 succeeds if and only if _all_ bits in the two operands are exactly the
 same.  Because this is not necessarily true after floating-point
 computations with a particular precision and effective rounding mode, a
 straight test for equality may not work.  Instead, compare the two
 numbers to see if they are within the desirable delta of each other.
 
    In applications where 15 or fewer decimal places suffice, hardware
 double-precision arithmetic can be adequate, and is usually much faster.
 But you need to keep in mind that every floating-point operation can
 suffer a new rounding error with catastrophic consequences, as
 illustrated by our earlier attempt to compute the value of pi.  Extra
 precision can greatly enhance the stability and the accuracy of your
 computation in such cases.
 
    Additionally, you should understand that repeated addition is not
 necessarily equivalent to multiplication in floating-point arithmetic.
 In the example in Errors accumulate:
 
      $ gawk 'BEGIN {
      >   for (d = 1.1; d <= 1.5; d += 0.1)    # loop five times (?)
      >       i++
      >   print i
      > }'
      -| 4
 
 you may or may not succeed in getting the correct result by choosing an
 arbitrarily large value for 'PREC'.  Reformulation of the problem at
 hand is often the correct approach in such situations.
 

Info Catalog gawk: Inexactness of computations gawk: FP Math Caution gawk: Try To Round