gawk: Setting precision
15.4.4 Setting the Precision
----------------------------
'gawk' uses a global working precision; it does not keep track of the
precision or accuracy of individual numbers. Performing an arithmetic
operation or calling a built-in function rounds the result to the
current working precision. The default working precision is 53 bits,
which you can modify using the predefined variable 'PREC'. You can also
set the value to one of the predefined case-insensitive strings shown in
Table 15.4 table-predefined-precision-strings, to emulate an IEEE
754 binary format.
'PREC' IEEE 754 binary format
---------------------------------------------------
'"half"' 16-bit half-precision
'"single"' Basic 32-bit single precision
'"double"' Basic 64-bit double precision
'"quad"' Basic 128-bit quadruple precision
'"oct"' 256-bit octuple precision
Table 15.4: Predefined precision strings for 'PREC'
The following example illustrates the effects of changing precision
on arithmetic operations:
$ gawk -M -v PREC=100 'BEGIN { x = 1.0e-400; print x + 0
> PREC = "double"; print x + 0 }'
-| 1e-400
-| 0
CAUTION: Be wary of floating-point constants! When reading a
floating-point constant from program source code, 'gawk' uses the
default precision (that of a C 'double'), unless overridden by an
assignment to the special variable 'PREC' on the command line, to
store it internally as an MPFR number. Changing the precision
using 'PREC' in the program text does _not_ change the precision of
a constant.
If you need to represent a floating-point constant at a higher
precision than the default and cannot use a command-line assignment
to 'PREC', you should either specify the constant as a string, or
as a rational number, whenever possible. The following example
illustrates the differences among various ways to print a
floating-point constant:
$ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", 0.1) }'
-| 0.1000000000000000055511151
$ gawk -M -v PREC=113 'BEGIN { printf("%0.25f\n", 0.1) }'
-| 0.1000000000000000000000000
$ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", "0.1") }'
-| 0.1000000000000000000000000
$ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", 1/10) }'
-| 0.1000000000000000000000000
Info Catalog
gawk: Try To Round
gawk: FP Math Caution
gawk: Setting the rounding mode