Speed in xbasic

[Robert Adler]

I have posted this problem before. This time I have attached a zip file that contains a working version of the application in question. The zip file contains a Readme.Txt that explains how to run the application and the application components.

I find the operations in A5 run quickly, but the processing files in xbasic extremely slow by comparison.

Furthermore, adding more horsepower to the hardware is an unacceptable solution. The current version of this applicaton is run in clipper on the same hardware and software environment. Time to process 10,000 records from import to export is 20 seconds versus 20 minutes in A5.

the bulk of the time is generated in the xbasic processing.

any help to improve performance is appreciated.

thank you

bob

[Willard]

RE: Speed in xbasic

Bob just started looking at your files and find that you have your table pointers directed exactly to your machine, ie. error one is cannot find D:download|.... I beleive it would take me too long to change your tables/scripts for the default directory so that anyone can test your codes Jack

[Stan Mathews]

RE: Speed in xbasic

Robert,

First let me say that I admire your coding discipline. While I don't understand all your conventions, I can tell that it is painstakingly constructed and exemplifies the the best of everything I have ever heard taught.

My old Toshiba 325CDS (233mhz) has a 4 gig drive partitioned into c and d drives and I was able to recreate yur directory structure and successfully run your application. This is a machine that by today's standards would be better suited to being a boat anchor if it was a little heavier. After entering the data as indicated, I received the "No customer records" "Record count is 0" in something under thirty seconds. Several reports preview with no records shown or the "there are no pages in the layout" messages.

From start to finish the whole process takes less than a minute on my machine.

You don't state in this message what your present hardware configuration is. If you are moving from a Clipper (DOS) based application to A5 (Windows) based application then your speed problem is not Alpha but Windows.

If what I have described gives you any clues to something else I need to do to make it run more as you designed, let me know.

[Stan Mathews]

RE: Speed in xbasic

Anyone who decides to give this a shot-----

There is a table "tblImportDirectory" that holds a path which needs to be edited to contain values which are valid on your machine.

Again......

nice code here

[Finian Lennon]

RE: Speed in xbasic

Robert:

When confronted with such tightly and elegantly written code, one is apt to hesitate before offering any suggestions. So the following is tendered gingerly!

If I am correct that the greatest amount of time is consumed validating the fields in the EditPurchase script, I wonder if the validation functions are somehow causing the batch_begin() and batch_end() to fail so you are not realising any benefit there. I really don't know, just speculating.

I have found that imports and appends in A5 are fast, even when you are converting/validating data in the process. Would it be possible to shift at least some of the validations out into the appends? For example, define an expression for each field in the append - all your UDFs will be available in the expression box, you may be able to use them as they are - and then check to see if there is any speed improvement.

All conjecture, of course.

Finian

[Tom Cone Jr]

RE: Speed in xbasic

Bob, I could not run your scripts because of pathing errors. Is it possible the tables to be imported were also omitted by mistake?

The script seems to crash in the import routine used at end of the beginPurchcust script. The error message refers to folders on Drive D:, and of course as things would have it, I don't have a drive D:, or the folders mentioned therein.

Will be glad to investigate this further, but need a bit more help from you to be able to replicate the issue.

Can you set this up so it will run on Drive C:, and can you verify that all necessary tables are there? (They probably are, but I can't get past the errors far enough to tell...)

-- tom

[Tom Cone Jr]

RE: Speed in xbasic

Bob, even though I couldn't run the scripts, I was able to read them over. The script in question validates and massages the data in the table, using a series of validation or check functions you have defined. It's really quite an elegant approach, as others have noted.

You might try stepping through the script once again, especially the functions, to verify that the tables which are being used to lookup various values are not being opened and closed for each record being validated. This would slow things down considerably. They should be opened once, and then closed at the end (unless I'm way off the mark here!)

-- tom

[Tom Cone Jr]

RE: Speed in xbasic

One other thought... since the validations often require you to fetch_find() a related record in a lookup table, you should carefully check the properties of the value being searched for... Is it possible for example, that the keyvalue doesn't quite match the indexkey specified for each validation routine? Are the indices on the lookup tables present and working? Would it help to do a fetch_first() after each fetch_find() to reposition the record pointer in the lookups?

Most of your code works in memory, and should be very fast... I'd focus my inquiry on the routines which fetch data from the disk and then compare it to values in the table being validated...

-- tom

[Tom Cone Jr]

RE: Speed in xbasic

Bob, after correcting folder referrences in the file_control table and the tblimportdirectory table, I got your form to run.

The import and validtion routine produces a msg box which says no customers were present, but then preview a report for SPR purchases.

Total lapsed time on my 500mhz Celeron pc is about 20 seconds.

I'm not running on a network, and don't know if you are working stand alone, also, or not.

-- tom

[Robert Adler]

RE: Speed in xbasic

Stan:
Clipper is dos based and it is fast.
if the problem is windows then i will stop converting to Access. scince that too will be windows based. The time to process has been long. a straight read of the imported editpurchase file requires 73 seconds. compared to 10 seconds of importing and appending the same 7600 records using the operations commmands built in to A5.

This would indicate to me that the file handling is different in the operations mode.

The code is a mix of the old names from existing code in clipper. I was to lazy to go back and change. Newer code reflects a more stingent attempt to identify items and thier purpose without extra documentation.


The zero count means the customer records failed the edit. I must check the down load of the file. I will put it out with a drive "C" designation and a single drive. All file access should be controlled through the FILE_CONTROL.dbf fields and the tblImpDirectory dbf.

thanks for your response I will attempt to download the system single directory access based on the "C" drive only.

again thanks for your insight.

bob adler

[Robert Adler]

RE: Speed in xbasic

Finian:
I have run debug on smaller files and did not observe any problem with the batch_begin batch_end process.

all files are opened once at the start of the editing process and closed at the end of the process.

I have started to include some editing in the append process, but this program is intended to handle input data from 60+ vendors. They work from standard file layouts, but do not follow them. They send data that does not conform to our main process. This small system is intended to replace 300 programs.

So editing done in the append process may be risky. But i will explore any option that would reduce the time to process.

again thanks for your input.

bob adler

[Robert Adler]

RE: Speed in xbasic

Tom:

my programs are run stand alone right now. My pc is a 300mhz intel processor with 128mg memory. the time to run this set of files is 12 minutes on this pc. if the system is run on the NT network it takes 20 minutes(the machine is running a 10mhz card). a faster pc will do better and at least match the stand alone time.

but i have never seen times like yours.

i am in the process of reducing file I/O by testing the search fields for duplication. If the prior input field equals the new input field the file is not read and the previos match is used. This has reduced the time by 40 seconds. still keeping the overall time at 12 minutes. 5 minutes on the editpurchase.dbf.

i will check further to see if their are settings on my pc that may permit the process to go faster.

thanks for your help and guidance.

bob adler

[Finian Lennon]

RE: Speed in xbasic

Bob:

That will be a big help since I haven't had the time so far to do more than read the scripts.

Finian

[Tom Cone Jr]

RE: Speed in xbasic

Bob, can you measure time lapse for chunks (a technical term, to be sure) of your scripts, displaying the results in a series of msg boxes as the script works its way to conclusion? Maybe that will shed light on where the delays are occurring.

Here, the import seems to finish in less than 5 seconds.

Using the data you supplied the only report which prints is the first one, rptNoPurchDays. Both rptPurchNoCustsummary and rptPurchNoTotalRecord report no matching records, so nothing prints. Is your timelag spread equally among these reports?

-- tom

[Robert Adler]

RE: Speed in xbasic

tom:

the time tests in a5 are done with the following code borrowed from a5:
dim t1 as n
dim t2 as n
t1 =toseconds(time())

script or operation executes

t2=toseconds(time())
trace.writeln("time Purch"+str(t2-t1))

you just have to view the trace table in view at the end of the run.

I am reposting the application.

thanks again

bob adler