Pagination in Marklogic while using Search API

Question

I have around 53,00,000 documents in MarkLogic server and I am building a simple search application. User enters a search term and MarkLogic server searches that term in all the nodes in all the documents and returns the matching documents as the result. I have implemented a custom paging to show results per page. I am showing 10 results per page.

I am using search api for this as:-

import module namespace search="http://marklogic.com/appservices/search" at "/Marklogic/appservices/search/search.xqy";

declare variable $options:=

<options xmlns="http://marklogic.com/appservices/search">

    <transform-results apply="raw"/>

</options>;

search:search($p, $options, $noRecFrom, 10)/search:result

where $p is the input from the user $noRecFrom is the number which indicates from where we have to show records. For example for page 1 $noRecFrom will be 1, for page 2 $noRecFrom will be 11, for page 3 $noRecFrom will be 21 and so on. For paging there are hyperlinks to go to First, Next, Prev and Last pages.

To calculate the total number of records returned I am using:-

for $x in search:search($p, $options)

return $x//@total;

While First, Next and Prev hyperlink works perfectly but if someone clicks Last the application stops responding and the query does not show any output. Is it due to the large number of documents in the database or I am implementing it wrongly.

Is there any efficient way for pagination in MarkLogic (for search:search) so that the user can go the Last page without delay in query result for such a large database ?

I'm suspecting flaws in your calculations to just to First, Next, Prev and Last. But you will have to share them with us to make sure. You could be accidentily requesting a lot of search results to return the last page, which would explain why it is responding slow. — grtjn

cwhit cwhit · Accepted Answer · 2012-04-11T14:45:09

The way you've implemented it, you're running the search repeatedly in your for loop. And that would indeed be slow.

Instead, you should be calculating a $start parameter based on the @total and number of documents per page, and passing that in as an argument (I think it's the third one) to search:search.

I would also recommend making sure you can run in unfiltered mode. There is good information about optimizing for fast pagination (indexes, etc) on the developer site; the idea is to resolve queries out of indexes to give very good, accurate unfiltered performance.

If you do it that

Pagination in Marklogic while using Search API

3 Answers