speed in xbasic 2nd post

[Robert Adler]

Thanks to all who have responded to my request for assistance in speeding up the edit process. to simplify the process for some i have recreated the zip file it is attached to this. the zip file should be opened in a dircectory on your "C" drive called "Vry". you must create this directory before opening the zip file and extracting the data. All pointers are now aimed at this directory. it should eliminate the previous problems.

i am retesting in small increments. but the initial results are not positive. it takes 73 seconds to read the editpurchase.dbf in xbasic if nothing is done. I have commented out all routines in my tests except for the fetch_next.

but i will check to ensure that the file is opened only once in rw_exclusive mode.

again thanks for your support.

bob adler

[Tom Cone Jr]

RE: speed in xbasic 2nd post

Bob, first time through the new version of the app my intervals were:
time Purch 389
time All 533


Here's where I'm going to look first. You use more UDF's than many, and they get called inside your main loop. I'm wondering whether time is lost interpreting the functions each time they get called. Will see if things speed up by placing the some of the UDF code inside the main script...

-- tom

[Tom Cone Jr]

RE: speed in xbasic 2nd post

Bob, the editPurchases script takes about six minutes here, all by itself. This seems to be the longest interval I've been able to identify among the several scripts which run...

In this script you load memory variables from values in the table and then validate them. Often you call UDF's to do the validation. In other places, these memory variables are used to post or update other values in other tables. At the end of the validation sequences you write the record back to the same table.

Since Alpha Five's field values are already buffered, writing them out to memory variables before you begin validating them is probably not necessary. You might try manipulating them directly inside a tbl.change_begin() and tbl.change_end() pair.

in pseudo code:
while .not. eof
put table in change mode
validate field 1
validate field 2
...
validate last field
end the change, thereby writing changes to the table
update the related tables
fetch the next record
end while

Again, I think possibly you may also get a slight speed increase if you don't rely so heavily on the UDF's... where possible, I'd embed the logic of the UDF inside the editPurchases script itself.

Let us know how things turn out.

-- tom

[Jim Rowlen]

RE: speed in xbasic 2nd post

I go in to various errors when trying to run you app. I'll look again to make sure I've got everything set up correctly (has been extracted to c:Vry - errors inidicate not being able to find a table on D:).

A thought in passing. We've noticed DRAMATIC speed increases when using a system with a lot of open drive space (apparently stuff done in backround runs faster when it doesn't have to look for space). Does the system you're running this on have a "lot" of open space? Defrag, etc, done to optimize performance? Even 20 or 30 M of unused space after deferag speeds things up.

Since I never received responses to my previous posts re improving efficiency in A5 re command structures which are more efficient (take less time, fewer pointers, etc), I'm not sure, but would tend to mirror Tom's considerations re the number of UDFs. If scripts are fully compiled before run time, then it shouldn't take any appreciable extra time for them. If, however, scripts run in "interpretive" mode, they could slow you down (for evaluation at run). One concern I had was calling a number of external global scripts. Could calling a number of global functions cause a slow down in operation (increased number of pointers, A5 has to switch to another script, etc)? I've run in to this in the past with other languages / development tools.

Will try to get time to look at this in more depth and get pack to you.

[Peter.Wayne]

RE: speed in xbasic 2nd post

As others have pointed out, you use a lot of user-defined functions from within you while-end while loop in the editpurch script. If you just read the records one at a time, without calling your validation functions and without writing any records, the processing time on my 200-Mhz notebook PC is 13 seconds. If I add back your different functions, the processing time slows down.
It is not clear to me why you have external functions that are only caled from the editPurch script. The slowest function is the wrtPurchRec() funtion (I may have the names of the functions wrong - I don't have it open now). In my testing the wrtPurchRec() function accounts for 4 of 7 minutes of the running of the editPurch script.
The structure of your functions is a little unusual for an Xbasic programmer. Instead of relying on your functions to return a value, you use the functions as subroutine calls, and you seem to rely on the side effects of the functions. Since Xbasic is not compiled, each function call from within a while-end while loop exacts a heavy toll.
You don't need as many global variables - you can minimize the number of variables if you bring some of the functions in-line. Also, some of your functions could be optimized - one seems to check repeatedly for "Q" or "P" in a series of if..end if statements.
If you bring most of your functions in-line, you will be able to dispense with some oher operations, such as the series of alltrim() operations that set up global variables. You probably don't need nearly as many global variables - you can do a lot of your verification and testing right from the table's field values themselves.
I also notice that if you have invalid data in a record, you keep processing the rest of the record, and don't abort the processing until you are ready to write the record. Depending on how many records have invalid data, taking action sooner will save a lot of processing time.

I don't think you will ever get the speed from Xbasic that you achieved in Clipper, simply because Xbasic is interpreted, not compiled. It has little to dow ith the Windows environment. But I think you can cut the processing down from 7 minutes to probably 2 to 3 minutes, and maybe even do a little better. You won't do any better in Access because that's also interpreted. If you want to get Clipper-like speed you would probably have to use Delphi, but the learning curve for Delphi makes Alpha Five look like an afternoon stroll.

Good luck. If you only import records once a day, then it shouldn't mater ery much whether it takes 15 seconds, 3 minutes, or 7 minutes. If you keep importing records all day long, then obviously you need to optimize your code or select a programming tool that will compile a fast executable.

[Robert Adler]

RE: speed in xbasic 2nd post

Peter:

thanks. I have begun moving the functions in line and eliminating the memeory fields. the editing is done per requirements. all fields must be edited and saved with error indicators on.
I was not aware that the udfs would slow down the processing until tom cone mentioned it. but your knowledge of a5's processing provides a clear picture. I must still unlearn clipper and c programming habits scince they are compiled and linked.
initial test of bringing the chkvendorid logic inline reduced the processing by 50 to 60 seconds. I hope this will be reflected in the rest.

again my thanks to you and everyone who has responded to this query. I hope in the future i can be of assistance to others.

i wish everyone a joyous and save Independce Day!!

bob adler

[Tom Cone Jr]

RE: speed in xbasic 2nd post

Bob, let us know how much improvement you see. You were using more UDF's than most of us attempt but your experience, with and without them, will be instructive for the rest of us.

I hope you succeed in getting the time down to a more manageable level.

-- tom

p.s. One other suggestion... your script seems to pass through the edPurch table more than once. If you can find a way to get the work done on a single pass, you might save the file i/o delays resulting from finding and saving each record multiple times. In a faster environment, using compiled code, this may not have been an issue.

-- tom

[Robert Adler]

RE: speed in xbasic 2nd post

Tom:
the script should loop once for each record. with each field being edited and flagged.

I have started moving the udfs into the mainline. the throughput has improved remakedly. I now process the first for edits(udfs) in 73 seconds. time to process should remain low as much of the coding is unique to this script.

this method of coding was required on mainframe systems for real time performance. I was unaware the the udfs were being recompiled each time they were called. I thought once the udfs were called they stayed linked and active.

I will now use the udfs for unique requirements only.

it is always good to learn something new.

thanks again for your assistance.

have a happy holiday.

bob adler