Strange val() behaviour in V12

[ssmoore1]

Hi All,

In V12 if I run this simple script in the interactive window, the resulting number in balance comes out like this 1887.7999999999993

debug(1)

dim test as c
dim test2 as c

test = "12322.34"
test2 = "10434.54"

dim balance as c

balance = val(test) - val(test2)

The same script in V10.5 returns a proper value. Any ideas why Val() is returning such a funky number in V12?

[Garry Flanigan]

Re: Strange val() behaviour in V12

Not sure why you have
dim balance as c
rather than
dim balance as n
I am sure that :
balance = convert_type(val(test) - val(test2),"c")
is good practice and will always work and survive changes.

If you really want balance as c (not in front of a V12 pc with V12 at present).
probably this also works (I saw Selwyn use it in a video):
balance = "" + (val(test) - val(test2))

We are sometimes spoilt by Alpha allowing us to not be 100% correct with data types but long term code will always be more robust if we are.
Hope this helps.

[Peter.Greulich]

Re: Strange val() behaviour in V12

Garry,

Not sure I understand your point:

test = "12322.34"

test2 = "10434.54"


? val(test) - val(test2)
= 1887.7999999999993

? convert_type(val(test) - val(test2),"c")
= "1887.7999999999993"

[Ray in Capetown]

Re: Strange val() behaviour in V12

It has nothing to do with val() behavior or any other functions.
We know from waa....aaay back that the 8086 maths processsor works to 18 decimal places over multiple iterations, and when pushed to more decimal places than you need you will see this

test = 12322.34
test2 = 10434.54
?str(test-test2,30,20)
= "1887.799999999999300"

[Garry Flanigan]

Re: Strange val() behaviour in V12

Peter, as I mentioned, I was not in front of V12 to test and now I see from others that I was way off track, so I did not have a valid point

[RichCPT]

Re: Strange val() behaviour in V12

Unless I am mistaken, the point behind using the character variables, "dim test as c", in the post is because in xBasic your numeric data from window controls on your components usually comes back to you in xBasic as the CHARACTER data type.

[Ray in Capetown]

Re: Strange val() behaviour in V12

Absolutely not.
This is a bad example, don't be misled.

[GGandhi]

Re: Strange val() behaviour in V12

window controls on your components usually comes back to you in xBasic as the CHARACTER data type.

hello

are you referring to web, on the web all are characters, as far as i know.

[RichCPT]

Re: Strange val() behaviour in V12

Perhaps, adopting Scott's code to work with two MONEY fields in a web app with a SQL DB will demonstrate the significance if this issue.

Make a grid with updateable detail view that includes two controls associated to SQL "money" columns. Put something like the following in the server-side AfterUpdateRecord event:

dim str as C
str = "xBasic Variable Type for SQL MONEY column = " + typeof(e.DataSubmitted.ProductPrice) + crlf() + \
"Value1= " + e.DataSubmitted.ProductPrice + " Value2= " + e.DataSubmitted.Price2 + crlf() + \
"Difference = " + ( val(e.DataSubmitted.ProductPrice) - val(e.DataSubmitted.Price2)) + crlf() + \
"Using 'Convert_Type' Difference = " + (convert_type(e.DataSubmitted.ProductPrice,"N") - convert_type(e.DataSubmitted.Price2,"N")) + crlf()

aa = e.rtc.A_AjaxResponses
aa[].text = "alert('" + js_escape(str) + "');"
aa[..].id = 100

Using scotts example values I get the following value in STR:

xBasic Variable Type for SQL MONEY column = C
Value1= 12322.34 Value2= 10434.54
Difference = 1887.7999999999993
Using 'Convert_Type' Difference = 1887.7999999999993�

[RichCPT]

Re: Strange val() behaviour in V12

I am submitting this as a bug to Alpha.

[Ray in Capetown]

Re: Strange val() behaviour in V12

You really don't get it!
There is no issue
there is no bug
This goes way back to the maths co-processor 8086 functions - look it up.
It has nothing to do with alpha.
Where in reality do you expect to calculate numbers in a string and then to 14 decimals?.
At 2 or 3 or even 10 decimals this is still numerically 1887.8

[RichCPT]

Re: Strange val() behaviour in V12

"Really!" Ray?

You are going to content that the following output from my Xbasic interactive window is acceptable output for a business class development environment? This might be acceptable output if you were writing an interpretive compiler at the highschool class project level.

?version("V") + "." + Version("B") + " " + version("E")
= "12.0.1620 Developers edition"

dim var1 as N = 12322.34
dim var2 as N = 10434.54
dim diff as N
diff = var1 - var2
?diff
= 1887.7999999999993

[Ray in Capetown]

Re: Strange val() behaviour in V12

No I'm not asserting anything, it is not my contention. As I said, research it there are hundreds of references to addition and subtraction by the processor/ This is the first hit on google

http://flint.cs.yale.edu/cs422/doc/a...m/pdf/CH14.PDF

[ssmoore1]

Re: Strange val() behaviour in V12

Hey Everyone,

Thanks for all the input! For my own personal needs I was simply subtracting one dollar amount from another. So obviously I don't want the return value to be .343434434 etc. To fix the current issue I simply added the round command to two decimal places after the return value.

dim total_balance as c
dim total_credit as c

total_balance = rs.data(1)
total_credit = rs.data(2)

delete balance
dim balance as c

balance = val(total_balance) - val(total_credit)

balance = round(val(balance),2)

However, in reading through the thread it seems there are two questions.....

1) When working with dollar amounts, what is the best way to subtract two figures in Xbasic to get a result with only two decimal places..

2) A more technical discussion which I don't claim to fully understand at this point on how Alpha Five works with certain computations.

Curious to see what Alpha thinks!

Cheers,

Scott

[RichCPT]

Re: Strange val() behaviour in V12

Ray, I did not notice any mention of packed decimal formats in the article you referenced. Packed decimal number format is expected, and usually required, for software developed for financial institutions:

dec Number library
LibP BCD/
Microsoft CLR DECIMAL
intel-decimal-floating-point-math-library