Memory Leak with .NET

[dparker]

Using Build 2549_3903
I keep getting memory leaks. I require myself to set the DotNet services variables in XBasic to null_value() once they are done. Not doing this seems to result in a very slow memory leak. This is fine for short processes.
However, i'm screwed with using long processes via alpha. I can only assume this will be fine once its across to IIS.

I currently have one process that is triggered on_click on dialog's button.
This performs an ajax call back. And i have placed the process into a single thread to monitor it in the .NET assembly, This enables me to continue navigating the site without and hinderance to performance as it will run on another core... that is until it starts hogging memory and eventually crashes the WAS at about 1.5GB!!!
This is after about 936 of the 1440 pdf files are done (which took about 12-15 minutes all up)... soo close.

The assembly itself is fine. I've created a console application to run the process in and it never goes over 150MB of memory usage when running the same process,
it averages at about 100MB or mem usage the entire time as it bounces between 50MB of mem usage and 100MB of mem usage
i can literally watch it pick up a PDF in the directory, process 4 Jpegs of it - resulting in a spike of about 100MB mem usage - then release all that memory and move onto the next file in the list.

While i watch mem usage for the WAS... i can see the same 100MB spike, yet it seems to not release about 1-2 meg per file processed, resulting in leak i can watch happening before my eyes in slow motion.

the XBasic function

function processTask as c (e as p)
dim sv as DotNet::Services
dim Alpha5Net4 as DotNet::AssemblyReference
Alpha5Net4.FileName = "C:\Users\Administrator\Documents\Visual Studio 2010\Projects\IconLib2\IconLib2\bin\Release\IconLib2.dll"

sv.RegisterClass("IconLib2","ProcessWebsiteXMLImages","IconLib2.ProcessWebsiteXMLImages",Alpha5Net4)

dim ProcessWebsiteXMLImages as IconLib2::ProcessWebsiteXMLImages

if e.dataSubmitted.radTasks = "1" then
	ProcessWebsiteXMLImages.processNow(e.dataSubmitted.txtIssueNo)
	ProcessWebsiteXMLImages = null_value()
else
	?"Task not available at this time"
end if

Alpha5Net4 = null_value()
sv = null_value()
dasa = "Finished"

end function

As demonstrated by the console application, the garbage collection is running efficiently (i've been specific when to release memory - i've literally hand coded each object's disposal just to be sure, but uneccessarily so)
But why is this not the case when I run the same assembly from Alpha?
Is there anything i can do?

[Tom Cone Jr]

Re: Memory Leak with .NET

This is way over my head, but you might want to submit a formal bug report, using the "Help" button in Alpha Five. Include what's necessary for them to replicate your experience in context at Alpha's headquarters.

[Finian Lennon]

Re: Memory Leak with .NET

Check out the dispose() method http://msdn.microsoft.com/en-us/library/bb738845.aspx.

[dparker]

Re: Memory Leak with .NET

I've freed all objects using that method... i am very vigilant with this...
even as someone who has been using C# for a while, i still don't trust .NET garbage collection to automatically free up resources at the correct time, so i definately implement disposal when and where i can... I've been over the assembly many times looking for anything i've missed freeing up, and yep... its all complete.

However, like I said, not a single memory leak was present when run this assembly is run from a console application, and now tested with Forms and WPF, working fine.

I can literally watch as all resources (and i literally mean ALL in the scope of the processing!!) free up after each iteration within that function. Naturally forms and WPF use more base memory than the console application... but no leaks.

(1440 iterations in total, ImageGlue .NET works very fast for PDF processing - lightning fast compared to others i've tried, or even batch processing from within Acrobat Professional (notorious leaker), and some of these PDF files are up to 30 - 40MB in size, so i'm happy with that)

I mean, there is a lot going on in this process. And under Alpha Five WAS it was processing just as fast - so no direct code execution complaints... but... i can just watch the mem usage grow and grow and grow...

The WAS is holding on to something.. i just can't pinpoint it.
Leak caused crash of WAS.
.dmp file empty.
logs show nothing from the crash

Bug report submitted.

Hopefully its just a solution and i'm doing something wrong from within XBasic or the Callback.

[dparker]

Re: Memory Leak with .NET

Check out the dispose() method http://msdn.microsoft.com/en-us/library/bb738845.aspx.

Hmm still stuck on this.
Are you saying i should inherit my own classes from IDisposable and call the dispose() method from within XBasic?

Also I've also just tested this out on a very small class in .NET that executes a SQL lookup, and returns the string value, it has its uses for the site (though not entirely neccesary as can be done from within xbasic, but this is just a test).

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data.SqlClient;
using System.Data.Sql;
using System.Configuration;

namespace IconLib2
{
    public class HeaderFunctions
    {
         // Main: Auto executes on class creation.
        void Main(String[] Args) 
        {
                        
        }
        
        // Gets company UUID
        public string getCompanyGUID(string username)
        {
            string SQLSelect =  "SELECT Salesperson.FK_Company FROM Salesperson " +
                                "INNER JOIN Logon ON Logon.FK_Salesperson = Salesperson.PK_Salesperson " +
                                "WHERE Logon.UserName=@Filter";
            return filterLookup(username, SQLSelect);              
        }

        // Returns Company Upload Directory
        public string getCompanyDirectory(string CompanyGuid)
        {
            string SQLSelect = "SELECT CompanyUploadFolder FROM Company WHERE PK_Company=@Filter";

            return filterLookup(CompanyGuid, SQLSelect);
        }

        // Lookup function.
        public string filterLookup(string Filter, string SQLSelect)
        {
            try
            {
                SqlConnection vpa1Conn;
                vpa1Conn = new SqlConnection("Data Source=ISQLSERVER;Initial Catalog=VPA_Test;Integrated Security=True");

                string returnString = "";
                SqlCommand getLookupCommand = new SqlCommand();
                getLookupCommand.Connection = vpa1Conn;
                getLookupCommand.CommandType = System.Data.CommandType.Text;
                getLookupCommand.CommandText = SQLSelect;
                getLookupCommand.Parameters.Add(new SqlParameter("Filter",Filter));

                vpa1Conn.Open();

                returnString = getLookupCommand.ExecuteScalar().ToString();

                getLookupCommand.Dispose();
                vpa1Conn.Close();
                return returnString;
            }
            catch (Exception ex)
            {
                return ex.ToString() + " " + ex.StackTrace.ToString() + " " + ex.Data.ToString();
            }
        }
    }
}

and the XBasic calling it on a page

<%a5
dim sv as DotNet::Services
dim Alpha5Net4 as DotNet::AssemblyReference
Alpha5Net4.FileName = "C:\Users\Administrator\Documents\Visual Studio 2010\Projects\IconLib2\IconLib2\bin\Release\IconLib2.dll"

sv.RegisterClass("IconLib2","HeaderFunctions","IconLib2.HeaderFunctions",Alpha5Net4)
dim HeaderFunctions as IconLib2::HeaderFunctions

dim companyGuid as c = HeaderFunctions.getCompanyGUID(a5ws_getcurrentuser())
HeaderFunctions.getCompanyDirectory(companyGuid)
session.uploadFolder = HeaderFunctions.getCompanyDirectory(companyGuid)
? session.uploadFolder

HeaderFunctions = null_value()
Alpha5Net4 = null_value()
sv = null_value()
%>

This works and is a very simple piece of code. Displays the company folder string fine... however, when I run this script, Mem usage climbs by about 200K and never frees up - This does not occur when i run from a .NET executable. Even if this was the only .NET my site ran, having users logging on and off for a couple of days will eventually result in running out of memory. Not a leak per se - just not releasing of resources.
*scratching head*

[Finian Lennon]

Re: Memory Leak with .NET

My .Net experience is extremely limited and desktop-based, not web. That said, the following may be of interest. It comes from the help files of a third party dll that I am using and applies to their construction. Whether the approach is applicable to your situation only testing would show.

quote:
void Dispose ()

While DotNet's garbage collection automatically cleans up memory associated with instances that no longer have references, CRsiNetwork spawns threads that continue to point to instances of these classes even when your references no longer do so; these pointers prevent the memory allocated to these instances from being automatically cleaned up: the result would be significant memory leaks if you didn't dispose of instances of these classes when you were done with them. You must dispose of CRsiNetwork explicitly for the memory to be freed. (CRsiComHandle and its subclasses CRsiComWinsock, CRsiComSerial, and CRsiComModem must also be explicitly disposed of.)

Disassociating the Network and ComHandle Before Disposing
As noted above, both the network and ComHandle need to be disposed of. If the network is correctly associated with the ComHandle, disposing of the network also disposes of the ComHandle. However, if an error prevented the network from being associated with the ComHandle, disposing of the network would not dispose of the ComHandle. This means that to guarantee that the ComHandle is properly disposed of, you must dispose of it directly. However, trying to dispose of the ComHandle if the network has already disposed of it causes an error. The solution is to use SetCom(null) to disassociate the network from the ComHandle first. Once you do this, disposing of the network does not dispose of the ComHandle; this leaves you free to dispose of the ComHandle with no possibility of trying to dispose of it twice.
unquote.

[dparker]

Re: Memory Leak with .NET

Registering the classes i need in an Autoexec script on the server stopped the leak. Which works fine for what i need them for and hopefully wont run into any instance where i'd need to register an assembly or class from within a page or component.

Heres what i did. (for anyone around with a similar problem)
Registering classes for the Alpha server:
1. In the control panel, create an xbasic script.
2. Register your assemblys/classes
i.e:

dim sv as DotNet::Services
dim Alpha5Net4 as DotNet::AssemblyReference
Alpha5Net4.FileName = "C:\Users\Administrator\Documents\Visual Studio 2010\Projects\IconLib2\IconLib2\bin\Release\IconLib2.dll"

sv.RegisterClass("IconLib2","HeaderFunctions","IconLib2.HeaderFunctions",Alpha5Net4)

sv.RegisterClass("IconLib2","GenericFunctions","IconLib2.GenericFunctions",Alpha5Net4)

sv.RegisterClass("IconLib2","ProcessWebsiteXMLImages","IconLib2.ProcessWebsiteXMLImages",Alpha5Net4)

3. Save code, name it "Autoexec" - This should ensure its execution on startup.
4. Use the classes (on a a5w page, Xbasic function in a component etc...)
i.e:

<%a5
dim ProcessWebsiteXMLImages as IconLib2::ProcessWebsiteXMLImages
dim result1 as C = ProcessWebsiteXMLImages.ToString()

dim GenericFunctions as IconLib2::GenericFunctions
dim result2 as C = GenericFunctions.ToString()
%>

Registering the classes on the server this way allowed me to execute the process to convert 360 pdfs into to 1440 jpegs in time just as fast as it were Native .NET. and no longer with any memory leakages.
Where as before it would climb to 1,500,000K of mem usage and over... now whilst processing the largest PDF file it got up to 330,000K.. and once it was done with that file it dropped right back down to 220,000K of memory usage before moving onto the next file and finally setting on 180,000K once the process was fully complete, idled over night, got to work this morning no change. Fantastic... we were beginning to worry about using Alpha and .NET together.

Our correspondence with an alpha employee suggest that the leak has already been fixed and will be available in the next release.