Info Catalog octave: Integer Data Types

octave: Integer Arithmetic

 
 4.4.1 Integer Arithmetic
 ------------------------
 
 While many numerical computations can’t be carried out in integers,
 Octave does support basic operations like addition and multiplication on
 integers.  The operators ‘+’, ‘-’, ‘.*’, and ‘./’ work on integers of
 the same type.  So, it is possible to add two 32 bit integers, but not
 to add a 32 bit integer and a 16 bit integer.
 
    When doing integer arithmetic one should consider the possibility of
 underflow and overflow.  This happens when the result of the computation
 can’t be represented using the chosen integer type.  As an example it is
 not possible to represent the result of 10 - 20 when using unsigned
 integers.  Octave makes sure that the result of integer computations is
 the integer that is closest to the true result.  So, the result of 10 -
 20 when using unsigned integers is zero.
 
    When doing integer division Octave will round the result to the
 nearest integer.  This is different from most programming languages,
 where the result is often floored to the nearest integer.  So, the
 result of ‘int32 (5) ./ int32 (8)’ is ‘1’.
 
  -- : idivide (X, Y, OP)
      Integer division with different rounding rules.
 
      The standard behavior of integer division such as ‘A ./ B’ is to
      round the result to the nearest integer.  This is not always the
      desired behavior and ‘idivide’ permits integer element-by-element
      division to be performed with different treatment for the
      fractional part of the division as determined by the OP flag.  OP
      is a string with one of the values:
 
      "fix"
           Calculate ‘A ./ B’ with the fractional part rounded towards
           zero.
 
      "round"
           Calculate ‘A ./ B’ with the fractional part rounded towards
           the nearest integer.
 
      "floor"
           Calculate ‘A ./ B’ with the fractional part rounded towards
           negative infinity.
 
      "ceil"
           Calculate ‘A ./ B’ with the fractional part rounded towards
           positive infinity.
 
      If OP is not given it defaults to "fix".  An example demonstrating
      these rounding rules is
 
           idivide (int8 ([-3, 3]), int8 (4), "fix")
             ⇒ int8 ([0, 0])
           idivide (int8 ([-3, 3]), int8 (4), "round")
             ⇒ int8 ([-1, 1])
           idivide (int8 ([-3, 3]), int8 (4), "floor")
             ⇒ int8 ([-1, 0])
           idivide (int8 ([-3, 3]), int8 (4), "ceil")
             ⇒ int8 ([0, 1])
 
      See also: ldivide XREFldivide, rdivide XREFrdivide.
 

Info Catalog octave: Integer Data Types