Alpha Software Mobile Development Tools:   Alpha Anywhere    |   Alpha TransForm subscribe to our YouTube Channel  Follow Us on LinkedIn  Follow Us on Twitter  Follow Us on Facebook



The Alpha Software Forum Participation Guidelines

The Alpha Software Forum is a free forum created for Alpha Software Developer Community to ask for help, exchange ideas, and share solutions. Alpha Software strives to create an environment where all members of the community can feel safe to participate. In order to ensure the Alpha Software Forum is a place where all feel welcome, forum participants are expected to behave as follows:
  • Be professional in your conduct
  • Be kind to others
  • Be constructive when giving feedback
  • Be open to new ideas and suggestions
  • Stay on topic

Be sure all comments and threads you post are respectful. Posts that contain any of the following content will be considered a violation of your agreement as a member of the Alpha Software Forum Community and will be moderated:
  • Spam.
  • Vulgar language.
  • Quotes from private conversations without permission, including pricing and other sales related discussions.
  • Personal attacks, insults, or subtle put-downs.
  • Harassment, bullying, threatening, mocking, shaming, or deriding anyone.
  • Sexist, racist, homophobic, transphobic, ableist, or otherwise discriminatory jokes and language.
  • Sexually explicit or violent material, links, or language.
  • Pirated, hacked, or copyright-infringing material.
  • Encouraging of others to engage in the above behaviors.

If a thread or post is found to contain any of the content outlined above, a moderator may choose to take one of the following actions:
  • Remove the Post or Thread - the content is removed from the forum.
  • Place the User in Moderation - all posts and new threads must be approved by a moderator before they are posted.
  • Temporarily Ban the User - user is banned from forum for a period of time.
  • Permanently Ban the User - user is permanently banned from the forum.

Moderators may also rename posts and threads if they are too generic or do not property reflect the content.

Moderators may move threads if they have been posted in the incorrect forum.

Threads/Posts questioning specific moderator decisions or actions (such as "why was a user banned?") are not allowed and will be removed.

The owners of Alpha Software Corporation (Forum Owner) reserve the right to remove, edit, move, or close any thread for any reason; or ban any forum member without notice, reason, or explanation.

Community members are encouraged to click the "Report Post" icon in the lower left of a given post if they feel the post is in violation of the rules. This will alert the Moderators to take a look.

Alpha Software Corporation may amend the guidelines from time to time and may also vary the procedures it sets out where appropriate in a particular case. Your agreement to comply with the guidelines will be deemed agreement to any changes to it.

Bonus TIPS for Successful Posting

Try a Search First
It is highly recommended that a Search be done on your topic before posting, as many questions have been answered in prior posts. As with any search engine, the shorter the search term, the more "hits" will be returned, but the more specific the search term is, the greater the relevance of those "hits". Searching for "table" might well return every message on the board while "tablesum" would greatly restrict the number of messages returned.

When you do post
First, make sure you are posting your question in the correct forum. For example, if you post an issue regarding Desktop applications on the Mobile & Browser Applications board , not only will your question not be seen by the appropriate audience, it may also be removed or relocated.

The more detail you provide about your problem or question, the more likely someone is to understand your request and be able to help. A sample database with a minimum of records (and its support files, zipped together) will make it much easier to diagnose issues with your application. Screen shots of error messages are especially helpful.

When explaining how to reproduce your problem, please be as detailed as possible. Describe every step, click-by-click and keypress-by-keypress. Otherwise when others try to duplicate your problem, they may do something slightly different and end up with different results.

A note about attachments
You may only attach one file to each message. Attachment file size is limited to 2MB. If you need to include several files, you may do so by zipping them into a single archive.

If you forgot to attach your files to your post, please do NOT create a new thread. Instead, reply to your original message and attach the file there.

When attaching screen shots, it is best to attach an image file (.BMP, .JPG, .GIF, .PNG, etc.) or a zip file of several images, as opposed to a Word document containing the screen shots. Because Word documents are prone to viruses, many message board users will not open your Word file, therefore limiting their ability to help you.

Similarly, if you are uploading a zipped archive, you should simply create a .ZIP file and not a self-extracting .EXE as many users will not run your EXE file.
See more
See less

wierd math side effect

  • Filter
  • Time
  • Show
Clear All
new posts

  • wierd math side effect

    In my program, a payment may be applied to several charges, and I have a function that checks the integrity of the payment by getting the amount of the payment and placing it in a variable, (var->gpn), and then subtracting the amount applied to all relevant charges one at a time. Then, if the variable gpn 0 there is a problem - e.g., one if its charges was deleted or returned - and I flag the payment as being erroneous.

    If the amount of the payment has .55 in it, like 8.55 or 32.55, (but not 8.50 or 32.50) it doesn't return 0 - it returns -0. The following is how I got around it, but it will only occur if .55 is part of the amount.

    if round((1 - var->gpn),0) 1 '(this works)

    I tried saying if var-> gpn gpn > 0, but it would not function correctly. I think that it is a rounding issue, but I couldn't figure out why.
    Cole Custom Programming - Terrell, Texas
    972 524 8714
    [email protected]

    "A young man who is not liberal has no heart, but an old man who is not conservative has no mind." GB Shaw

  • #2
    RE: wierd math side effect

    I believe it is a rounding issue that crops in the conversion between your decimal input and math in binary the machine performs and the conversion back again to decimal. Some fractional numbers simply don't convert cleanly.

    You can find similar rounding errors in certain versions of certain database products.


    • #3
      RE: wierd math side effect

      Martin, I've been around the block many times and gone 15 rounds with this issue. Are you charging sales tax on these items? Are you able to make the quantity being sold a fraction? If the answer is is true to either of the above your problem does lie in rounding. What happens is on the individual sale on some items rounds up and some down. Thats OK for that particular sale but when you start combining the items in several invoices the total may not add up to same amount as would be calculate by adding only the totals of each invoice. If you try to fix it by always rounding one way or the other you will run into a stuation once in a while where an invoice itself will show a penney or two error.
      Does this sound like your problem, if so:
      When does sales tax get figured? On each item and then totaled or on the sub total of all items?


      • #4
        RE: wierd math side effect

        thanks for the replies. No, I don't currently have to deal with taxes (Texas). An no, all units are thankfully whole numbers. But I will have to get taxes added before April, when I plan to demo the app at a convention.

        It is just that with certail "cents" amount, I get the error. Like if I start with 8.55 and deduct .55 and then 8.00 I end up with -0 instead of 0 which has it fail my test if variable 0.
        Cole Custom Programming - Terrell, Texas
        972 524 8714
        [email protected]

        "A young man who is not liberal has no heart, but an old man who is not conservative has no mind." GB Shaw


        • #5
          RE: wierd math side effect


          I can't get Alpha to accept a -0.00 value so it's pretty certain that the low level value in the system is something like -0.0000016 with up to 6 zeroes before the number. You can check this in the debugger. Since you are dealing with 2 decimal places the correct value of your calculation is, effectively, 0. So why not change your zero checks?

          For nVar=0 instead use
          if nVar>-0.001 .and. nVar0
          if nVar0.001

          You can see this in the debugger
          Equalto=(nVar>-0.001 .and. nVar0.001)
          = .F.

          I do this in several places and it works fine.



          • #6
            RE: wierd math side effect


            I've been unable to replicate the error you are seeing, while working in the interactive editor.

            If I store 8.55 in a numeric var, and then subtract 0.55, and then subtract 8.00, I get 0.00000

            Are you certain this is all your script is doing?

            Are you converting from char to numeric along the way?

            -- tom


            • #7
              RE: wierd math side effect

              I'm not sure how you're doing everything, but are you using any form.values to calculate any apply-to totals?? If so, is it possible that any of these values are zero and you have not actually put a zero in the field ( as per Bill Warner's tip about setting numeric default values ). This could give you erroneous results or worse.


              • #8
                RE: wierd math side effect

                following is the part of the function that checks to see that a payment has distributions that equal the amount of the payment

      "Payment history")
      "payments") 'payment distributions
                query.filter="cust_number=cnum22" 'payments for one customer
                if idx4.records_get() > 0
                --dim tempn2 as n
                --while .not. pay.fetch_eof()
                ----gpn=pay.amount 'get the amount of the payment
                ----query.filter="paymentlink=tempn2" 'get all relevant distributions
                ----if idx5.records_get() > 0
                ------while .not. paid.fetch_eof()
                --------gpn=gpn - paid.clientapply 'if applied to client
                --------gpn=gpn - paid.carrierapply 'if applied to carrier
                ------end while
                ----end if
                ----if .not. between(var->gpn,-0.001,0.001)
                ------check="Y" 'flag used at end of script to mark customer
      "Y" 'general purpose marker field
                The part where I was having trouble was
                -----if .not. between(var->gpn,-0.001,0.001)
                I was using
                if gpn 0
                this always works unless, e.g, a payment was 8.55, and had a distribution of 8.00 and .55

                I have used Finnian's as I like it better than the one I was using most recently
                if round((1 - var->gpn),0) 1

                I foresee, like Jeff, further necessary tweakings when and if I have to add tax to the system, multiplying by .0855, etc.

                Cole Custom Programming - Terrell, Texas
                972 524 8714
                [email protected]

                "A young man who is not liberal has no heart, but an old man who is not conservative has no mind." GB Shaw


                • #9
                  RE: wierd math side effect

                  I just discovered basically the same problem in another form – when entering a new payment.

                  z=payamount.value ‘the amunt of the new payment
                  tbl=table.current(2) ‘charges in embedded browse where amounts to apply to are filled in
                  while .not. tbl.fetch_eof()
                  ---x=x + tbl.clientdue ‘get total due for all client charges
                  ---y=y + tbl.carrierdue ‘get total due for all carrier charges
                  end while
                  if ans="Client"
                  ---if x=0
                  -----ui_msg_box("Error","Client has no amounts due on any items")
                  ---end if
                  if z > x ‘*********** if the amount of the payment is greater than the total due (could possibly apply to)
                  here is one place among many where the comparison will return false for a true value if I am working with cents.
                  charges (tbl 2) are 70.55, 8.00, .55, and 35.33 for the client
                  so x=114.43 (total client due)

                  if I enter a payment of 114.43 (z) it returns true (if z > x) when it should return false, since both sums are 114.43.

                  if I say ---if str(z,10,2) > str(x,10,2) --- it returns correctly

                  this had never come up as an issue before, because I was always working with either whole amounts (1200.00) or even cents amounts (.30, 50, etc.) until an operator had a postage amount of .55 instead of .50 or .75 or 1.00.

                  But as soon as I throw the extra cent in there it bombs. This could require mucho time to go through a thousand scripts in my app to convert dollars to strings for comparisons!!!

                  I tried looking at the two variables str(x,16,12) and str(z,16,12) – but they both show exactly the same amount.

                  Cole Custom Programming - Terrell, Texas
                  972 524 8714
                  [email protected]

                  "A young man who is not liberal has no heart, but an old man who is not conservative has no mind." GB Shaw


                  • #10
                    RE: wierd math side effect

                    here is an example in an interactive window that you can try for yourself

                    dim tempx as n
                    dim tempy as n
                    tempy=3.33 + .55 + .33 + 8.00 + .21
                    ui_msg_box("","" + if(tempx = tempy,"true","false" + str(tempx,16,12) + " " + str(tempy,16,12)))

                    this will show both values the same to 12 decimal places, but the comparison will show false (at least on my computer!)
                    Cole Custom Programming - Terrell, Texas
                    972 524 8714
                    [email protected]

                    "A young man who is not liberal has no heart, but an old man who is not conservative has no mind." GB Shaw


                    • #11
                      RE: wierd math side effect


                      Rounding the variable values to the 3rd decimal will also return the correct true or false that you need. I don't know that it's any easier or better than the string comparisons.



                      • #12
                        RE: wierd math side effect

                        thanks Finnian - yes I noticed that round(x,3) works. Logically you would think it better to keep working with numbers than to convert to strings for comparisons - I think I'll try the round approach. !!What a pain!!
                        Cole Custom Programming - Terrell, Texas
                        972 524 8714
                        [email protected]

                        "A young man who is not liberal has no heart, but an old man who is not conservative has no mind." GB Shaw


                        • #13
                          RE: wierd math side effect

                          Martin - Finian
                          Here is where it happens. Granted it is an irrelevant difference in business apps, but it produces uncomfortable results in comparisons. A bug clear and simple. I'm using a less than sign in the comparison in case it doesn't show up.
                          dim h1 as n
                          dim h2 as n
                          h1 = 12.42
                          h2 = 3.33 + .55 + .33 + 8 + .21


                          • #14
                            RE: wierd math side effect

                            I also had problem with "INT", quite a while ago

                            Look at browse1 to see the different results

                            Column 1: Value
                            Column 2: Result using INT(Value*100)/100
                            Column 3: Value * 100
                            Column 4: INT(column 3 result)
                            Column 5: (Column 4 result)/100

                            Look at the rows in bold ,INT(Value*100)/100 does not return the Value

                            Bizarre ?


                            I hope file is there


                            • #15
                              RE: wierd math side effect

                              Yes, I see that. As you say it doesn't effect a typical business app unless you are aware that direct comparisons of stored numbers may produce unpredictable results unless "treated" in some way.