Representation of Numbers

This page has been reproduced from original A+ documentation provided by Morgan Stanley at www.aplusdev.org. Copyright remains with the authors.

Comparison Tolerance

Arithmetic of Inf

NaN

The machines on which A+ is run represent numbers in binary (base 2), for efficient storage and computation. A+ (via C) represents integers in 32 bits (binary digits) and floating-point numbers in 64. The largest integer representable in this way is (2*31)-1 and the smallest is ¢2*31 (-2147483648); the greatest magnitude representable as a floating-point number is approximately 1.7977e+308.
In display and print, of course, A+ represents numbers in decimal. People tend to think in decimal and their input is usually decimal. Numbers like .1 and .01 seem exactly representable intrinsically (as they are in decimal), but, in fact, these numbers cannot be exactly represented in binary, and decimal to binary and binary to decimal conversions are one source of imprecision.
Another source of imprecision is the necessarily limited number of digits that can be used to represent a number. One effect of this limitation is inexactitude, which can be seen in simple examples:
123456789123456789 - 123456789123456788 0 12345678912345679 - 12345678912345678 2 .123456789123456789 - .123456789123456788 0
and yet:
.000000000000000001 1e-18
Another effect of this limitation is, of course, that there are numbers that are simply too large to be numerically represented in A+ at all, such as 10*1234.
Results can only be meaningfully displayed out to a maximum of 16 digits. Any digits shown beyond the 16th digit are invalid. For example, the actual value of pi begins with 3.141592653589793238; but if we set $pp to 20 (a value that is beyond the meaningful range), o1 will return 3.141592653589793116. Only the first 16 digits are correct.
To allow for slight imprecisions in the representation of numbers by considering numbers to be equal when they are very close to equal, the concept of a comparison tolerance is used. To allow computations to continue when some of the numbers involved have become too large to be represented, a special representation, Inf, has been introduced, together with its negative, ¢Inf.

Comparison Tolerance

Comparison tolerance cannot be set; it is fixed at 1e-13.
It is used in both conversion of floating-point numbers to integers and explicit comparisons, as in Equal to, Match, and Find.
The fundamental notion is to put a band around one of the comparands and see whether the other comparand falls within that band. Consider x=y, for instance. Roughly speaking, the equality holds if x is within (without tolerance, of course) the interval from y«1-1e-13 to y«1+1e-13. More precisely, the difference between the comparands must be less (without tolerance) than the tolerance times the smaller of them in absolute value: |x-y must be less than 1e-13«(|x)Ä|y. Because multiplication is involved, only zero is equal to zero within the comparison tolerance.
Comparison tolerance applies only to floating-point comparisons. The largest magnitude that can be represented in A+ in integer type is 2*31, which is |¢2*31. That times the tolerance, i.e., 1e-13«2*31, is only about 0.0002. Thus the tolerance could not have any effect on comparisons between two integers.
When examining the effect, or apparent effect, of comparison tolerance, you must take into account the printing precision (which may make two unequal numbers look equal) and the fact that the Equal to function itself uses comparison tolerance. As an intolerant check for equality of a and b, you can use 0=a-b.
Consider numbers shown in an example above, where their difference was approximated as 2:
     12345678912345679 = 12345678912345678
 1
Clearly, Equal to uses comparison tolerance to arrive at the result: A+ disregards the insignificant (and in fact inexact) difference it finds between them. But doesn't comparison tolerance apply only to floating-point numbers? Yes, but there are several ways to enter numbers for representation in floating point: e.g., with a decimal point (99. or 3.14, for instance), in exponential notation (6.023e-23), and with too many digits for it to be represented internally as an integer (12345678912345679). Each number entered is converted to its internal representation before anything else is done with it.
Comparison tolerance is used in:

Ceiling, Floor, and Residue;
Equal to, Greater than, Greater than or Equal to, Less than, Less than or Equal to, and Not equal to;
Find, Match, and Member;
and no other functions.
When two floating-point numbers are equal within the comparison tolerance, they are called tolerably equal. This term is also used in a more general way, to mean equal within the tolerance for floating-point numbers and strictly equal where no floating-point number is involved.
Note that the term restricted whole number, as discussed above and in "Restricted Whole Numbers", involves even more toleration, since it includes numbers whose absolute value is less than 1e-13, whereas zero is not tolerably equal to any nonzero number.

Arithmetic of Inf

A+ uses Inf to denote positive numbers too large for it to represent, and ¢Inf for negative numbers whose magnitude is too great to be represented. Inf is tantamount to infinity. You can enter these notations directly; also, if you enter, say, 1e309, it will be shown and stored as Inf.
Instead of a domain error, division by zero of a positive number yields Inf, and of a negative number, ¢Inf. The Min reduction of an empty vector yields Inf, and the Max reduction yields ¢Inf. And so on.
When either of these values is an argument to a function, most of what you might expect happens: -¢Inf is Inf, and «Inf is 1, and 4ÄInf is 4.
Any operation which heightens Inf or ¢Inf produces it as its result: 10«Inf is Inf, and ¢Inf-100 is ¢Inf, and Inf+Inf is Inf, and so forth. Operations involving representable numbers that tend to lessen Inf or ¢Inf are also accepted, however, so that, for example, Inf-1e99 is Inf. (Indeed, although they are unlikely to occur, 1e309ß1e308 is Inf and 1e308ß1e309 is 0, because 1e309 is converted to Inf upon input, before the division, and 1e308 is representable.)
Domain errors result from indeterminate expressions like Inf-Inf and 0«Inf, as well as 0ß0.

NaN

A+ does not handle NaN (Not a Number) but it provides tools that let you detect NaN's. See the _nanfind system function and the $dbg command, especially the table "$dbg Subcommands (Arguments)".

Back to Home page

doc@aplusdev.org