Dynamic pagination of rich:dataTable/rich:datascroller with large datasets

Question

I currently have a Richfaces dataTable bound to a backing bean that contains all the data. This table is bound to a Richfaces data scroller.

Users can then enter scroll between the data.

There is also a text box that dynamically updates the data table results displayed to them, based on text they enter into it. It reRenders the dataTable and datascroller on the keyUp event.

The backing bean it’s bound to first pulls all the data from a number of database tables. This data is pulled when the user submits a normal post request to the server, and it’s the results of this request that are used for all subsequent Ajax related queries (Results saved as list of objects, no more database calls made).

I have a problem in that the dataset can be huge at times, 100,000s of records. This causes the initial request to the server to take a very long time.

A possible solution I'm looking at is pulling back only a small amount of the data in one thread for the initial user port request. This data can then be displayed in the data table while the main thread works in the background pulling the bulk of data back.

Is this feasible? Would it be possible to update my datatable/datascoller as the main thread pulls back new data? Would it be difficult?

Database and query optimization has been looked at, so no more improvements of any significance can be made there.

Thanks in advance (I know this probably is not an easy question to answer)

I have a problem in that the dataset can be huge at times, 100,000s of records. This causes the initial request to the server to take a very long time. I believe if you have used pagination it will fetch first m rows only initially — jmj
Seam Application Framework has an in-built pagination that you can use for free. But then you need to add seam to your project. (Which is a great idea if you are using JSF and Richfaces). docs.jboss.org/seam/2.2.1.CR1/reference/en-US/html/… — Shervin Asgari
Jigar Joshi - I think you have misunderstood my question (Or I haven't phrased it correctly). Initially the user submits a request to the server (Standard Http Post, Not Ajax), it pulls the data from database and stores all the data in a list of objects. The page is then displayed to the user with the list of objects bound to dataTable and dataScroller. From this new page they can perform the ajax requests on the data. — Thomas Buckley
I don't think Jigar misunderstood. What you need to do is implement pagination, and only retrieve first X rows, and then with pagination you retrieve next etc. Meaning, when you initially load the page, you only retrieve say 100 rows — Shervin Asgari
@Shervin: Yes, I do need to implement pagination and it is already implemented. The problem is I have a process pulling back a large list of objects initially, this process may take 30 seconds. So I want to initially show a small subset of records, say 30 records or 10 per paginated page (So user is'nt looking at a blank screen). So once all the objects have been retrieved I can then re-populate the dataTable and update the pagination details to reflect the complete data set. Am I making sense? — Thomas Buckley

Shervin Asgari Shervin Asgari · Accepted Answer · 2011-02-02T12:43:07

Seems like you need exactly what Seam Application Framework's Query object is providing.

If you don't want to use Seam, you can either view the source, and copy how they are doing it and just steal the idea.

Basically what you need to do is fetch a given result set for each time the user press next, previous, first, last etc

Dynamic pagination of rich:dataTable/rich:datascroller with large datasets

2 Answers