WAS / AA Performance tuning

[pcaspa]

Hi,

I have a live app with a main page that has a large and complex UX with 10 embedded components. Most of these embedded components are also relatively complex. It works great and I have a very happy client however I can only run about 6 concurrent users on 1 WAS/CPU before the CPU hits the wall. I have resolved this with an AA based load balancing system which works fine.

However as I am expecting significant growth in usage I'm looking for ways to improve the performance of the WAS and/or my app so I am not limited to 6 concurrent users per WAS/CPU . I've tried all the obvious stuff on the app with indexing and components only loading on demand etc.

Any suggestions?

[reedallenbrown]

Re: WAS / AA Performance tuning

Had the same problems and once I moved up to a higher end server "hosted by Rackspace" everything was much better.

[DaveF]

Re: WAS / AA Performance tuning

Peter,

What are the specs for your server (e.g., CPU type, RAM, etc.)? What kind of load balancer are you using? How is the load balancer helping exactly if you only have a single web server?

Thanks

[reedallenbrown]

Re: WAS / AA Performance tuning

Dave,

We have three web servers serviced by a Rackspace Load Balancer.

Intel(R) Xeon(R) CPU E5-2670 @ 2.60GHZ 8 processors 8 GB of RAM 64 bit Windows server 2012

[DaveF]

Re: WAS / AA Performance tuning

Reed,

Those are certainly very fast Intel CPU's. Does each web server have a single or dual Xeon E5-2670? A dual would have a total of 16 cores.

Are you running a single WAS on each physical web server?

[reedallenbrown]

Re: WAS / AA Performance tuning

Dave,

Single core. Yes running a single WAS on each server.

[Pat Bremkamp]

Re: WAS / AA Performance tuning

Here are eight ways to get more users, and all involve increasing speed or reducing load:

More app server instances. You can put multiple instances of the app server on the server by assigning them to different ports. You don't specifically say you are doing this, but you imply it by talking about load balancing. In my case, I use the port 80 instance for the general public and the other port instances for employees.

Faster hardware. If the page can be processed faster, then you can serve up more pages in the same time and hence more users. ZebraHost can set you up with a server that uses SSD drives. They are noticeably faster.

Faster back end. You have probably done this already, but a SQL back end will be faster than a dbf, especially as file sizes grow. Look at increasing cache sizes, proper indexes and putting often used indexes in memory.

Use SQL stored procedures. Look for places where Alpha is making a lot of individual calls to SQL, and move those into an SQL stored procedure. In one case in my app, I reduced the processing time from almost 2 seconds to .05 seconds.

If your app involves lots of photos, reduce the photo size to reduce the time it takes to send the photos over the internet.

Reduce the load per page. You described the component as "large and complex". Does every user use the entire component at the same time? If you can split that up into smaller pieces so that each user experiences a small delay between functions, you could reduce the load. You will be trading off a small delay for each user for more users.

Use the report server. If your app generates reports, the report server will speed up delivery of the reports so each user will get off the computer faster.

If your application is used in-house by your client, don't let the users talk you into setting the session expiration to 4 hours so they can go to lunch and leave the application running on their computer. AA does a good job of cleaning out session folders (except for photos and pdf) so signing out and signing back in will reduce the load.

Congratulations on having a happy client!

[Clipper87]

Re: WAS / AA Performance tuning

I believe it has to do with Alpha's WAS being single threaded. That said take a look at this thread that talks about Process Lasso, down the end someone replies it also seems to do a great job to enhance the WAS performance. http://msgboard.alphasoftware.com/al...l=1#post569881

I wonder if the IIS WAS plugin will improve things; I would expect so

[RichCPT]

Re: WAS / AA Performance tuning

Look for places where Alpha is making a lot of individual calls to SQL, and move those into an SQL stored procedure.

Pat, do you mean places in xbasic code that we code like manually looping through a record set?

[bea2701]

Re: WAS / AA Performance tuning

I believe it has to do with Alpha's WAS being single threaded. That said take a look at this thread that talks about Process Lasso, down the end someone replies it also seems to do a great job to enhance the WAS performance. http://msgboard.alphasoftware.com/al...l=1#post569881

I wonder if the IIS WAS plugin will improve things; I would expect so

32 bits plugin on 32 bits IIS or 32 bits plugin on 64 bits IIS ---> 64 bits plugin on 64 bits IIS that will make a difference , we wait an see.....

[pcaspa]

Re: WAS / AA Performance tuning

Pat, Thanks for a great checklist of things to consider however I can tick them all except maybe the Stored Procedures. Using Firebug its not one SQL call that causing a problem, I just have a lot of them. Some room for optimising I expect

Re load balancing I have allocated WAS's to CPUs using always up, I track activity on each WAS and the login link redirects to the least busy WAS (This excludes Process Lasso). Simple but works well and allows me to scale. This is a solution but with 6 users per WAS I'm going to need a server farm.

I'm talking to my Hosting provider, Web24, here in Australia re CPU options.

Thanks for the advice

[Pat Bremkamp]

Re: WAS / AA Performance tuning

Rich and Peter,

Slowest part of the process is sending the data over the internet and opening and closing the SQL table. Idea is to minimize that.

Here are a few examples that worked for me:

I have a client who collects daily data which involves a lot of calculations and getting daily balances which requires looking at the previous day's balances and checking for changes earlier in the month. I started by recalcing the data one day (row) at a time. I switched to reading in the whole month's data into Alpha arrays from one SQL call, doing the calculations in alpha, then writing back the data in one SQL statement with up to 31 update values. 5 times faster that one at a time. (Alpha is faster at math than MySQL).

I have another where the user enters data in any units they have ( feet or meters for example ) and I needed to convert to standard units ( say inch and mm ) and format for use in a report. I started with the conversions in an alpha UDF and would read in one row to Alpha, do the calculations and write back the answer (about 300 rows). Switched to a stored program using SQL functions for the conversions. One call and MySQL does the rest. That's the one that went from 2 seconds to .05. They produce about 600 reports a day, so that is a big time saver.

It also works within SQL. MySQL has a "prepare" statement that is like the alpha prepare template (which is another good thing to learn). I have a procedure that reads about 100 rows of data from 5 different tables, formats it and writes it out to a single table to use in a report. I used prepare so the write was created into a text variable, then written as one statement. About twice as fast as one row at a time. Only downside is if there is an error in any of the lines, then none get written.

I have about 70 SQL functions and stored procedures in my latest app, and they are a big time saver.

[JerryBrightbill]

Re: WAS / AA Performance tuning

Pat, do you mean places in xbasic code that we code like manually looping through a record set?

Any process that loops through records will be slow, sometimes very slow. There is a term for that - RBAR (Row By Agonizing Row). Most SQL operations can be done in some bulk operation, and loop methods should be avoided.

However, if row by row operation can't be avoided, this should be done in a stored procedure in the SQL database if possible. If it must be done in xbasic, you should not step through a result set while leaving a connection open unless the result set is very small. A better solution is to use a method such as Resultset.toPropertyArray() or ResultSet.toString() to dump the data to an array or string. Then close the connection and step through the array or string with normal xbasic methods.

We recently added significant changes to those methods to greatly increase their speed by orders of magnitude. Result sets now can be dumped to a string or array in less that 10% of the previous time. In one test, the time to build a very large array from a result set was only 2% of the previous time before the updates. These changes are in the latest pre-release builds.

Also, if you are using a loop process to buildup a string by adding to it on each loop, such as creating HTML output row by row, syntax such as

dim txt as c
dim looptxt as c
for i = 1 to 100
	looptxt = "Some text"+convert_type(i,"c")+crlf()
	txt = txt + looptxt
next

can be slow if the number of loops is large and the string created is very large. Instead, use the *concat() function to add new text to the existing string

for i = 1 to 100
	looptxt = "Some text"+convert_type(i,"c")+crlf()
	*concat(txt,looptxt)
next

Because of the way *concat() allocates memory, it will be significant faster when building a very large string using many multiple iterations. In small loops and strings the difference is negligible, but as the number of loops increase and strings gets larger, the different can be significant.

[pcaspa]

Re: WAS / AA Performance tuning

Hi,

Thanks for the great advice re RBAR which will come in very useful however the components I'm trying to optimise do not use this technique (fortunately). I have done some further testing and it seems the vast majority of the CPU load comes from loading the component, not calls to SQL. One component I'm trying to optimise is 612KB and the most of that size comes from the controls. I can't see any way to optimise this component short of reducing its functionality.

Any Suggestions?

Thanks

[Steve Wood]

Re: WAS / AA Performance tuning

Try this, convert from a Table to SQL Select Statement (if not already), then under Edit SQL Query Builder go to Properties and select Show First and pick some number between 100 and 1000. If you have a Search part, you can check the first Search Property to omit the Grid unless a Search has been performed. You should structure your Search field so that a search will yield fewer records less than your "Show First" value. Depending onw conditions, it can greatly improve grid performance.