sorting limitation of list on large data

[Peter.Greulich]

Re: sorting limitation of list on large data

Amit,

Maybe you should try a disconnected approach? Load your data into the device's SqlLite db. Not sure about a million records, but we have done that with 500,000 records - and queried against it. But we are typically returning only one record in the query. Don't know if that is applicable in your case, but it might be worth a look.

[Selwyn Rabins]

Re: sorting limitation of list on large data

if you have run a test using KendoUI then you are doing server side pagination and are likely bringing down 1 page of records (the page length is defined by you, but would probably be in the range of 10 to maybe 1,000 records) at a time. using server side pagination in a Alpha List control would also be extremely performant. but this would be a connected app -- not a disconnected app.

you are definitely not bringing down 1 million records from the server to the client - if you were, kendoUI would be god awful slow (as would any other client side list control). and of course, this has nothing to do with KendoUI -- it is just physics -- 1 million rows of data is a very large payload and bringing a payload of that size from the server to the client is going to be slow. not to mention the time taken to load the data into the the KendoUI list, once you have gotten the data onto the client.

if you want to have a disconnected app that has access to 1 million rows of data on the client side. the ONLY way to do this (in my opinion) is to use something like SQLite on the device. You can easily use Alpha do to this. There are built in actions in action Javascript to make an ajax callback to the server -- create a SQLite database on the server with all of the data you want to have on the client - and then download the SQLite file to the device. I suspect that there would be absolutely no problem doing this in Alpha using the SQLite Phonegap plugin and the actions exposed in Action Javascript.

using Xamarin, in an of itself, will NOT provide a solution. the data still needs to be stored in an on device database (like SQLite) regardless of whether you build the App in Alpha or Xamarin.

[amitloh]

Re: sorting limitation of list on large data

Hi Selwyn,

WE are not bringing 1 million records at one go. Currently the kendo grid has paging enabled meaning we only bring set of 50 out of million records real fast. When user clicks on next/previous/page numbers the grid automatically finds the next set of 50 and bring it fast(but even if it brings set of 50 when I sort desc it will not sort just 50 it will sort server side based on actual set). I see that alpha list does the same way when I enable paging which is great but the sorting does not work on whole set of records with paging enabled(it does it client side ), I tried server side sort which is great now I am trying to see if I can use custom layout to do this. We do not want to use xamarin and fully coded solution so trying to see if alpha phonegap app can handle 1/2 million records set with filter, sort, export, search(need to have performance of it in less than 10 seconds) as current live application does that with coded approach with backend as sql.

[amitloh]

Re: sorting limitation of list on large data

We bought alpha for one app use case and Now I am trying to see if alpha can be applied to build another huge app based on huge data(1 to 5 million rows), I tested phonegap list based app with 500k and it worked fine except few limitations like client side sort and few other things. Will give it a try over the weekend.

[Steve Workings]

Re: sorting limitation of list on large data

I think, Amit, that here's your problem - emphasis is mine, the statement is yours from this thread:

no+time

I am still looking for some video or document that explains how to use session and state variables globally in phonegap alpha app.(no time to R&D)

You have no time for R&D but have seen how far and fast you can get with Alpha Anywhere. Maybe it's time for some R&D, and I'd especially heed Selwyn's words.

Users on this board have been generous to you despite your "no time to R&D" so maybe it's time for some study and design work.

[amitloh]

Re: sorting limitation of list on large data

You are right Steve partially. We are building one app really fast with alpha, now I am trying to see if another advance app can be built real fast(I will spend some time on design)

I would like to put that "no+time" is another post and nothing to do with this post. Sometimes if the issue is small I may not be in position to do R&D as this is RAD tool.
I am doing good R&D when time permits and these are very valid use cases for large complex app(high usability) and I am just asking on this forum to get opinions.
I am specifically looking for like below one which is live in one of our web based apps-(click on 3 dots in kendo grid)
http://demos.telerik.com/aspnet-ajax...defaultcs.aspx

http://demos.telerik.com/aspnet-ajax...defaultcs.aspx

http://docs.telerik.com/kendo-ui/con...-you-type.html


Our scenarios in some of apps are very advance(all above 3 features are great for 1 million plus rows) and I am just trying to see if above features can be met with Alpha or not.
I agree with you regarding alpha forum- compared to other tools alpha forum is really great.

[Selwyn Rabins]

Re: sorting limitation of list on large data

Hi Selwyn,

WE are not bringing 1 million records at one go. Currently the kendo grid has paging enabled meaning we only bring set of 50 out of million records real fast. When user clicks on next/previous/page numbers the grid automatically finds the next set of 50 and bring it fast(but even if it brings set of 50 when I sort desc it will not sort just 50 it will sort server side based on actual set). I see that alpha list does the same way when I enable paging which is great but the sorting does not work on whole set of records with paging enabled(it does it client side ), I tried server side sort which is great now I am trying to see if I can use custom layout to do this. We do not want to use xamarin and fully coded solution so trying to see if alpha phonegap app can handle 1/2 million records set with filter, sort, export, search(need to have performance of it in less than 10 seconds) as current live application does that with coded approach with backend as sql.

sorting in a list (that is paginated) happens client side (on just the current page of records in the list) if you have configured the list to operate this way.
if you configure the list to do server side sorting, then it will sort server side. the property that governs where storing takes place (client side or server side) is set on each column in the List layout.
if you configure the list to sort server side, and you have 1,000,000 rows in the list, then you better be sure to have appropriate indexes on your database so that the sql database can perform the sort in a reasonable time.

[amitloh]

Re: sorting limitation of list on large data

perfect! Thanks Selwyn. I tried server side sort on 500k and it works good with acceptable performance. Amazing thing is even without indexing alpha android app opens up fast and does server side sort, paging in real good time. I will be adding indexing for sure.

[kkfin]

Re: sorting limitation of list on large data

I want to apologize my earlier message (now deleted) and I want to specially apologize to David.

But million records is lot and so is half of it.

So Amit why not show the example in live. If you handle millions of records then a live page should not be an any problem. I really want to know your solution!

I have moved to SQLite about a year a ago but the biggest problem I have found is how to secure the data way that also the customer understand it.

Ken

[amitloh]

Re: sorting limitation of list on large data

Hi ,

I never said I want to show 1 million records at one go(no one does this).
I have paging enabled and the sorting needs to be on 1 million records.

you can check any standard telerik grids on telerik site with paging enabled
http://demos.telerik.com/aspnet-ajax...defaultcs.aspx
http://demos.telerik.com/aspnet-ajax...defaultcs.aspx

I think I got good answer from David and Selwyn. Again I do not show 1 millions rows on one single page.(its paged)

[kkfin]

Re: sorting limitation of list on large data

Amit,

I have used Kendo UI about 4 years. You can check this from my posts to this forum. If you need to sort anything(read your first post) then you need to handle all records. And that is all Server side. Kendo UI is totally client side. You use it just to show records. Wake up man.

Ken