* Info Catalog Up gawk: Inexactness of computations Next gawk: Comparing FP Values

gawk: Inexact representation

 
 15.4.1.1 Many Numbers Cannot Be Represented Exactly
 ...................................................
 
 So, before you start to write any code, you should think about what you
 really want and what's really happening.  Consider the two numbers in
 the following example:
 
      x = 0.875             # 1/2 + 1/4 + 1/8
      y = 0.425
 
    Unlike the number in 'y', the number stored in 'x' is exactly
 representable in binary because it can be written as a finite sum of one
 or more fractions whose denominators are all powers of two.  When 'gawk'
 reads a floating-point number from program source, it automatically
 rounds that number to whatever precision your machine supports.  If you
 try to print the numeric content of a variable using an output format
 string of '"%.17g"', it may not produce the same number as you assigned
 to it:
 
      $ gawk 'BEGIN { x = 0.875; y = 0.425
      >               printf("%0.17g, %0.17g\n", x, y) }'
      -| 0.875, 0.42499999999999999
 
    Often the error is so small you do not even notice it, and if you do,
 you can always specify how much precision you would like in your output.
 Usually this is a format string like '"%.15g"', which, when used in the
 previous example, produces an output identical to the input.
* Info Catalog Up gawk: Inexactness of computations Next gawk: Comparing FP Values