How do I handle so many calculations?

[Howard G. Cornett]

I have numerous calculated fields that I am trying to place on my form. These are nested many layers deep (a calc field based on a calc field which is based on a form field, or deeper). When I get to about 10 fields, it takes about 15 or 20 seconds to do anything in the calc field builder, but the form itself does not seem to take a performance hit. How do I run these calculations so that it doesn't bog down the builder, my form, or the database in general? How do I make this work most efficiently? As always, thanks in advance for any help.

Howard

[csda1]

Howard,

Form calculations (at least in their current and past incarnations) are real inefficient. As the calculations can be based upon other calculations, they are actually executed many times as you fetch a record and move from field to field in a form. Most times this is not an issue, as the speed degradation is not noticable to the user. On the other hand, you can manually (well sort of - I really mean XBasic or Action script) have variables set to your calculation values as you fetch or enter a record, move in and out of paticular fields, or when you save the record. These have the advantage of only being recalculated when you decide a particular field that the calculation is based upon has changed.

Not much fun manually coding, but it can enhance the snappiness of the display and values.

On the other hand, if you can improve the efficiency of your form calculations, it might not be needed. The expressions you used can be often recoded with faster functions, and the nesting can be unrolled (basically using longer expressions to include the other calculation's expression). This unrolling can be, but may not always be, more efficient.

Finally, sometimes calculation expressions can be done more efficiently using XBasic code in functions that perform part or all of the calculations in a more efficient way than a single expression line can do.

Regards,

Ira

[Steve Andrews]

Howard,

I agree with Ira that functions may be the way to go even though efficiency will still be questionable. While converting a system from V1 to V5, I encountered a dozen or so reports each of which contained the same set of calculated fields - about 25 of them nested and very complicated. I wrote one UDF which replaced all of these calculated fields.

Aside from efficiency, there are two main advantages of using a UDF. First, debugging is much easier. When we had a question about some figures on the older reports, we had to go through by hand and reproduce every calculated field to analyze the problem. With the UDF, we can use debug() to see exactly what is going on with the calculations.

Secondly, in cases like mine with duplicated calculations between layouts the UDF will assure consistency. In my transformation I discovered a few inconsistencies due to mistyping. Also, if there are any future changes in the criteria which governs the calculations, we now only make the change at one place (my UDF) rather than going to each report to make the same change.

So in general when expressions get to lengthy or calculated fields of a layout get too complicated, I prefer using xbasic in a UDF. My 2 cents.

Steve

[johnkoh]

Good Morning,

I have about 20 calculation UDF and 3 of the tablesum() function on a form and used it on the calculation value expression.

The form is running too slow so I added the Ui_msg_box() on one of the UDF. Whenever open or closed this form, the same message is shown several times until the end of related UDFs.

Is the way faster and avoid the repeating the scripts?
One Sample of my UDF.

FUNCTION Calc_ChngOdr_Fica AS N (vChangeOdrId as c)
	
'Get field value from Current ChangeOrder form
dim tChangeHd as p
dim qChangeHd as p
dim nChangeHd as n

dim vProjectId as c
dim vAdjRate as c
tChangeHd = table.open("change_hdr",FILE_RO_SHARED)
query.order="hdr_id"
query.filter ="hdr_id="+quote(vChangeOdrId)
qChangeHd = tChangeHd.query_create()
nChangeHd = qChangeHd.records_get()
if nChangeHd>0 then	
	vProjectId = tChangeHd.project_id
	vAdjRate = tChangeHd.Adjust_fica
	
tChangeHd.close()

'Get the Default value from Contracts table within same project
dim vBdgtRate as n
  vBdgtRate = Bdgt_FicaRate(vProjectId) 

dim vConvRate as c
  vConvRate = convert_type(vAdjRate,"n")  
dim vItemRate as n
 vItemRate = if(isblank("var->vAdjRate"),vBdgtRate,vConvRate)
  

'How to get the Current Items value?   
dim vTempSum as n  'Toal value of the Labor
vTempSum = Calc_ChngItm_SumLbr(vChangeOdrId) 'Get the Labor Value for current change order
  'ui_msg_box("Materials","Total material Amount is  "+vMtrSum)
  
'Final Calulation  
Calc_ChngOdr_Fica = Round(vTempSum*vItemRate/100,2)
  'ui_msg_box("Final SalexTax","Calculated Salex Tax is  "+"co"+vChangeOdrId +" rate  " +vItemRate+"  and Result Value is  "+Calc_ChngOdr_Fica)

end if
END FUNCTION

[Mike Wilson]

Not quite certain what your question is. I do see significant errors in your code. You have variables dimmed as character mixed in with variables dimmed as numeric. Unless vMtrSum is global, it is not defined in this UDF. And you have numberic variables in you message string, that should return an error.

FUNCTION Calc_ChngOdr_Fica AS N (vChangeOdrId as c)
	
'Get field value from Current ChangeOrder form
dim tChangeHd as p
dim qChangeHd as p
dim nChangeHd as n

dim vProjectId as c
dim vAdjRate as c
tChangeHd = table.open("change_hdr",FILE_RO_SHARED)
query.order="hdr_id"
query.filter ="hdr_id="+quote(vChangeOdrId)
qChangeHd = tChangeHd.query_create()
nChangeHd = qChangeHd.records_get()
if nChangeHd>0 then	
	vProjectId = tChangeHd.project_id
	vAdjRate = tChangeHd.Adjust_fica
	
tChangeHd.close()

'Get the Default value from Contracts table within same project
dim vBdgtRate as n  
vBdgtRate = Bdgt_FicaRate(vProjectId) 

dim vConvRate as c
  vConvRate = convert_type(vAdjRate,"n")  
dim vItemRate as n vItemRate = if(isblank("var->vAdjRate"),vBdgtRate,vConvRate)
  

'How to get the Current Items value?   
dim vTempSum as n  'Toal value of the Labor
vTempSum = Calc_ChngItm_SumLbr(vChangeOdrId) 'Get the Labor Value for current change order
  'ui_msg_box("Materials","Total material Amount is  "+vMtrSum)
  
'Final Calulation  
Calc_ChngOdr_Fica = Round(vTempSum*vItemRate/100,2)
  'ui_msg_box("Final SalexTax","Calculated Salex Tax is  "+"co"+vChangeOdrId +" rate  " +vItemRate+"  and Result Value is  "+Calc_ChngOdr_Fica)

end if
END FUNCTION

[johnkoh]

Mike,

My question is why my UDF scripts are running too slow and repeating.

I selected the Character data type for the "Adj_Rate" field(s) in the table because I want to calculate if the field value contain "0.00" which is not Null value, "Qty * 0.00 = 0.00" and If the field value is Null then use vBdgtRate, "Qty * vBdgtRate = Result".
I noticed that the Number data type is not able to find the Null value.

[Stan Mathews]

My question is why my UDF scripts are running too slow and repeating.

We can't answer either question without looking at your application. Viewing the code of one udf doesn't help diagnose that.

Is the table change_hdr the parent table of the set upon which the form is built?

tChangeHd = table.open("change_hdr",FILE_RO_SHARED)

If so you can probably save some time by getting a pointer to the current instance.

tChangeHd = table.current(1)

If you don't change that line then you should at least close the table pointer instance somewhere in your function.

tChangeHd.close()

[Mike Wilson]

Yes BUT:
The variable (vAdjRate) you have collecting the Adjust_fica field value is dimmed Character. Is the Adjust_fica a character field? If so further in your code....

dim vConvRate as c
  vConvRate = convert_type(vAdjRate,"n")

You are trying to convert the character value in the vAdjRate variable to numeric and have that numeric value in the variable vConvRate which you have dimmed character. Then further down...

vItemRate = if(isblank("var->vAdjRate"),vBdgtRate,vConvRate)

your if statement is trying to say "have the numeric variable vItemRate be the numeric value vBdgtRate if this, but if not then be the character value of vConvRate". This can't and will not work.

[johnkoh]

Mike,

You are right. I changed it to "N" data type.

dim vConvRate as N
  vConvRate = convert_type(vAdjRate,"n")  
dim vItemRate as n
 vItemRate = if(isblank("var->vAdjRate"),vBdgtRate,vConvRate)

Thank you.

[johnkoh]

Stan,

We can't answer either question without looking at your application. Viewing the code of one udf doesn't help diagnose that.

I attached the zip file and open the Change_Orders form.

Thank you for help.

[Stan Mathews]

Many of your calculated fields draw info from the contracts table which is not included in your attachment. Can't do much without it as the lack creates multiple errors.

[johnkoh]

Stan,

I opened the Zip file which are fine here.
I attached the 2nd zip files.

[Stan Mathews]

I misspoke. It is the proposal table that is not included. You have your database spread across two directories. Teamwork and changeorders.

Your zip does not include the tables/sets in the teamwork directly, only the changeorders directory. It works on your machine because the teamwork directory exists on your machine.

The field rules for the spec_section field in the contacts table has a lookup to the proposal table which fails here (something like 150 times).

[johnkoh]

Good Morning Stan,

Before I will send the new zip files, I'd like to ask about

Your zip does not include the tables/sets in the teamwork directly

Sorry, I do not understand the " the teamwork directly ".

I checked with my zip file, I can see the Bestteam. ALB .ALM. ALX & .Muf. See a screen shot.

Thank you

[Stan Mathews]

I checked with my zip file, I can see the Bestteam. ALB .ALM. ALX & .Muf. See a screen shot.

Thank you

Your zip does not include the tables in your c:\teamwork directory.

I was able to create a dummy proposals table to eliminate the spec_section field rule error. I was able to create a dummy user_profile table to eliminate the autoexec error. Both these errors were due to the tables they use being in your c:\teamwork directory and not in the c:\changeorders directory. Any tables not in c:\changeorders are not in your zip.

I do see why you are having sluggish reactiion on the form.


You changed (for example).

'Customer_Name = lookup("customer","customer_id="+quote(vCustID),"Customer_name")

to

FUNCTION Customer_Name AS C (vCustId AS C )
	
    dim tCustom as p
    dim qCustom as p
    dim nCustom as n
    dim vCustName as c
    dim vCustAdd as c
    dim vCustCity as c
    dim vCustState as c
    dim vCustZip as c
    tCustom = table.open("customer",FILE_RO_SHARED)
    query.order ="customer_id"
    query.filter = "customer_id="+quote(vCustId)
    qCustom = tCustom.query_create()
    nCustom = qCustom.records_get()
    if nCustom>0 then
        qCustom.drop()
        vCustName = tCustom.Customer_name
     end if
	Customer_Name = alltrim(vCustName)
	tCustom.close()
END FUNCTION

Your function is slower than the original single line. You could use Ira's utility to find out how much slower but I would guess it is significantly slower.

I'm not sure you even need either. A different set structure might allow you to link the customer table and no function/lookup would be needed.

[johnkoh]

Stan,

Here, new zip files and I changed some of Field rules and added the tables as much I can.

If you can open my Change_orders form, Please see following my issue
(1) Move the record - it took about 10 second
(2) Change field record - it took about 10 second for just ready to edit.
(3) When I type - it took some time
(4) After Changed the Rate fields - it took some time for calculation.

I think, each of my current UDF where I used on the Calculation Value are running until the end itself.

If each of my UDF does not contain any error, I think, It may be caused by execution structure(flow style), I mean on the Calculation value.

If slow action was caused by flow style, Can I use some of UDF on the FieldRule or Do you have any better solution?