Sloooooow Report

[Ohlen Cartmell]

I've finally got around to converting my main app to V4. Other than a few troubles with V4 not allowing my sloppy variable definition, I got it ok. However, a report I run several times a day executes in V3 in about 30 sec and in V4, about 2min or longer. The report uses about 12 tableum expressions to get daily and monthly totals by type of sale. Can anyone give insight to why the report runs so slow in V4 and what I might do to speed it up?

[csda1]

RE: Sloooooow Report

Ohlen,

30 sec versus 2 minutes is not bad. It would be worse if it was 1 hour vs 4 hours and you needed the report hourly. However...

Make sure you use run_total() instead of total() and similar wherever possible. Using the non-"Run_" functions is like doing preprocessing of a report in A4. Only use these when absolutely necessary (e.g. to compute percentages).

If you must use these non-"Run_" functions, what you can do is compute them prior to running the report, and then pass the totals to the report as variables.

Regards,

Ira J. Perlow
Computer Systems Design & Associates
[email protected]

[French]

RE: Sloooooow Report

Limit the use of line objects on your reports and forms, particularly lines that are dashed. They seem to take forever to print.

[Stephen Williams]

RE: Sloooooow Report

You live and learn...I use total() a lot, never thought to try run_total().

I finally ran join, update and summary operations as needed to create a new table of just what I wanted, and made a report with very little extra processing from this new table. I found this dramatically improved speed, plus I found I could present the information in much better ways in the final report.

I also got MUCH better performance with 64MB ram over 32MB...I bet 128MB would go even faster...

Steve Williams

[CALocklin]

RE: Sloooooow Report

Steve,

It was interesting to see your comment:

I finally ran join, update and summary operations as needed to create a new table of just what I wanted, and made a report with very little extra processing from this new table. I found this dramatically improved speed, plus I found I could present the information in much better ways in the final report.

I've used this method a few times (although I usually create the intermediate tables with XBasic - in one case 4 intermediate tables for a complex report) and I agree completely on both points - faster and better presentation.

What surprises me is that you are the only other person I've seen make note of this on the message board.

[CALocklin]

RE: Sloooooow Report

In case anyone else is interested, the idea is to create intermediate tables which select/sort the necessary data and then append the final selection into a special table designed just for the report. Before appending the new data, the previous data must first be zapped.

The final data selection must be appended to the special report table in order to be able to save the report format. Only the data needs to be updated for each report. I have seen as much as a 10 to 1 speed improvement using this method on a complex report. If I remember correctly, I even did one report which could not have been created any other way.

Of course, this is only suitable for complex reports which take a long time to generate. I find it most useful in certain circumstances where it's necessary to query a large set to get only a few records. If anyone needs more details, feel free to contact me.

[Ohlen Cartmell]

RE: Sloooooow Report

Thanks to all who replied. I created an intermediate table containing just the information needed and designed the report around that. I've got most of what I want; but, the original report was on a set. The intermediate table, therefore, contains fields from the parent and the children. When I tablesum on a field originally in the parent, I get a sum too large since the parent fields repeat for each child. I've thought of creating another intermediate table to contain just the parent fields and then base the tablesum on it. However, does anyone know how to account for the duplicate instances of the parent fields. Will blank_duplicate work here??

[Stan Mathews]

RE: Sloooooow Report

Just thinking out loud here....

When you create your intermediate table, could you not sum (or whatever)the values for each child record you need from the child table to the new table for each parent record? I mean instead of just pulling a parent record and a child record all the way through, pull a parent record and perform your calculation on the child fields.

This presumes you don't need some values "intact" from the child table for each parent record.

[Elliott Lester]

RE: Sloooooow Report

Cal,

Jim has been doing this for years. It is actually what
all SQL programmers have begun to do--and it is an
essential part of OLAP. You can do it in advance by
determing what data you will need for a particular report and posting it into a table set up for that report. Then
use that table to produce the report. The blue bar of death searchs only those records that have been preassembled
for the report. It take planning but is well worth it.

Good luck

[Ohlen Cartmell]

RE: Sloooooow Report

Stan,
I created a summary field in the copied from table but the calculated fields can't be copied. I solved it with 2 intermediate tables. One for the detail and one for the parent information. Works fine. Thanks to all.

[Rich Drabik]

RE: Sloooooow Report

I've used a similar approach when importing data into a table that contained fields requiring lookups. If there are hundreds of thousands of records to be imported, lookups or a separate update process can take forever. But, if the data is imported to a temporary table, then joined with the the lookup table to go into the final, "real" table, it seems to take no time at all to update the records with the lookup results.

[csda1]

RE: Sloooooow Report

Richard,

Lookups, and just about any singular type reference to another table's values is extremely inefficient on a network because it gets access (locking) and then unlocks for each record.

Depending upon the process, .batch_begin() and .batch_end() around the process will speed this up (but will slow others while this happens)

But in general, anytime you can use a high-level A5 operation (such a a join, crosstab etc) or create a special set that links the data that you want to reference, the speed will increase by orders of magnitude. For short reports, this is typically overkill for the savings in time (and potentially complicates code tremendously!), however, for long reports it is often the best way.

Any temporary files should ALWAYS be created and used in the A5 private directory, not the A5 shared or database directory. If you need to have a report that's associated with a database, create a database that serves as a template in the database directory, and duplicate it to the private directory as needed.

I have not studied this precisely, but I suspect that non-running values (like total() as opposed to run_total()) are calculated once in a report's process (for the scope of the value, e.g. group 1). However, if that is not true (and someday I might get around to testing this), then if you can precalculate a value before running the report (which really only helps for Grand Totals and similar) and store the value in a variable, then it might make the report faster as well.

Regards,

Ira J. Perlow
Computer Systems Design & Associates
[email protected]

[Rich Drabik]

RE: Sloooooow Report

Thanks, Ira. I had learned about.batch_begin() and .batch_end()from your earlier postings, but tend to forget about it unless I see a reminder. This whole issue of speeding up certain processes certainly looks ripe for examination.

[Stephen Williams]

RE: Sloooooow Report

My report IS prepared by xbasic. But it was done by first creating and saving the operations needed, and the final report, the stitching it all together with action scripting. Not at all hard with respect to coding in Alpha Five. It was a little mind numbing tracking the field names etc through the operations, and as I built the report I rebuilt the operations till I got it hashed out by putting a posting rule on one table so I kept an up to date summary in another linked table (this was the biggest time saver), then ran a join and an update to get another set of summarized data not available any other way. This report summarizes employees paid hrs, billed hours and productivity over a two week period, on a daily basis, with a summary of worked, holiday, vacation, regular hours and overtime hours. Plus it gives me the company totals. The final print fits 10 employees on a single page so I can spot missed time sheets, possible billing errors, plus take the pulse of who is working how hard, and quantify it. The report is used to fill in payroll numbers for a payroll service. It is a key report that previews a report on 12 employees in less than 10 seconds from raw data.

Steve Williams