exact numbers

[Jos Schaars]

To my horror I discovered that:
number with fractional part - int(number) doesn't return the fractional part
the same with for example:
mod(number,1)

eg 6.2 - int(6.2) = 0.2
returns false!!!

The only way to work around this is to use round()?

[Tom Cone Jr]

RE: exact numbers

Jos,

Describe the context in which you're doing this.

When I try the exact same sequence in the interactive editor it works fine. Here's what I see:

?6.2 - INT(6.2)
= 0.200000

-- tom c

[Jos Schaars]

RE: exact numbers

Indeed it looks ok, but now try this:
i = 6.2 - int(6.2)
? i=0.2

or

?(6.2 - int(6.2)) = 0.2

surprise, surprise

[Al Buchholz]

RE: exact numbers

Jos

The definition of int() is to remove the fractional part of a number and return on the the integer portion of the number. Your examples shows that it is working properly.

What are you trying to do and what are you expecting from the int() function?

[Jos Schaars]

RE: exact numbers

int() works ok, just used it to show that when the integer part is substracted from a number not the exact fractional part is left, try:
?(6.2-6)=0.2

it returns FALSE !!

[Stan Mathews]

RE: exact numbers

Must be something in the air where you are....


?(6.2-6)=0.2
= .T.

? (6.2 - int(6.2)) = 0.2
= .T.

i = 6.2 - int(6.2)

? i=0.2
= .T.

All work for me.

[Jos Schaars]

RE: exact numbers

Thanks for your reply.
No wonder the confusion, copied directly from your reply
?(6.2-6)=0.2
to the interactive window and got:
= .F.

Also something like:
?(mod(6.2,1))=0.2
results in:
= .F.

And now:...
i = (6.2-6)-0.2
?i
= 0.000000
?i=0
= .F.
i=1000*i
i=1000*i
i=1000*i
i=1000*i
?i
= 0.000167

Using Alpha Five version 4.5 build 266
Running Windows XP Professional, version 2002 with service pack 1 installed.
Located in the Netherlands, so the Dutch version.
The air is fine over here, anyone an idea?

[JohnZaleski]

RE: exact numbers

When dealing with variables, you sometimes have to be careful when comparing calculations. Certain fractional arithmatic leaves very small residuals that are basically too small to be seen but pop up in comparisons - this has been cited before.By the way, the same type behavior has been observed in Excel.

When comparing results of calculations kept in memory, I use the round function.

This should evaluate as true:
round(6.2 - int(6.2),2) = 0.2

[Jos Schaars]

RE: exact numbers

I already assumed that I would have to use round() extensively with any calculations dealing with possible fractional parts. Not only with int(), mod() ..., but also
with plain additions and substractions.

And how come that Stan Mathews gets true with
?(6.2-6)=0.2
while I get false?

This behavior only occurs when the result is between -2.2 and 2.2, sounds a bit buggy to me.

[JohnZaleski]

RE: exact numbers

First, I have no idea why it works for Stan. He didn't give any details about the platform.

In the interactive window of Alpha try the following:
?3.33 + 0.55 + 0.33 + 8 + 0.21 -12.42 = 0
= .F.
Put this into a cell on an Excel spreadsheet:
=3.33 + 0.55 + 0.33 + 8 + 0.21 -12.42 = 0
It should evaluate to FALSE

There seems to be some difficulty going from decimal to binary and back to decimal.Bottom line is be careful when comparing calculation results not just in Alpha, but in other apps as well.

[Stan Mathews]

RE: exact numbers

Sorry guys, I was on my V5 only machine and didn't consider the implications. (xp and v5)

Can confirm the problematic results with 4.5 on a Win 98 machine as well.

I'll try to stay more awake next time I drop in on the v4.5 board.

[Jos Schaars]

RE: exact numbers

OK, Excel also has problems with this.
But the conversion from binary to decimal and vice verse can be done correctly. Just use more precission internally with calculations and automaticly round the result to a lesser precission.

Glad to hear the problem is fixed in version 5, because I probably opt for version 6.

[Tom Cone Jr]

RE: exact numbers

Jos,

>>Glad to hear the problem is fixed...

John's advice is solid. I don't think you should ever rely on exact comparisons automatically rounding exactly to the displayed number of decimals. Some decimal numbers simply cannot be expressed "exactly" in binary form. The best any computer can do is to approximate the decimal number in binary form.

A commonplace example of this is found in the payroll module of QuickBooks. Often the computed payroll tax liabilities for an entire quarter differ slightly from the sum of the monthly payroll tax deposit calculations. Intuit and the IRS describe it as a 'rounding error', and manual adjustments are permitted.

It's a fact of life with computers which work in binary, while "we" work in decimal.

In short, if you're using exact comparisons in your application the developer must shoulder the responsibility for manually rounding the computed results before doing the comparisons.

-- tom

[Jos Schaars]

RE: exact numbers

Hi Tom,

Can't agree with:
The best any computer can do is to approximate the decimal number in binary form.

Any computer should probably read any program.
But besides that, one option is to store numbers in BCD format, another to do calculations as mentioned in a higher precission and round the results to a lesser one.
Suppose version 5 does this, because it hasn't this problem. And there other programs that do it right.