TYPO3 Extbase - Paginate through a large table (100000 records)

Question

I have a fairly large table with about 100000 records. If I don't set the limit in the repository

Repository:

public function paginateRequest() {

$query = $this->createQuery();
$result = $query->setLimit(1000)->execute();
//$result = $query->execute();
return $result;

}

/**
 * action list
 *
 * @return void
 */
public function listAction() {

$this->view->assign('records', $this->leiRepository->paginateRequest());

//$this->view->assign('records', $this->leiRepository->findAll());

}

... the query and the page breaks although I'm using f:widget.paginate . As per the docs https://fluidtypo3.org/viewhelpers/fluid/master/Widget/PaginateViewHelper.html I was hoping that I can render only the itemsPerPage and 'parginate' through the records ...

List.hmtl

<f:if condition="{records}">
<f:widget.paginate objects="{records}" as="paginatedRecords" configuration="{itemsPerPage: 100, insertAbove: 0, insertBelow: 1, maximumNumberOfLinks: 10}">                     
    <f:for each="{paginatedRecords}" as="record">
        <tr>
            <td><f:link.action action="show" pageUid="43" arguments="{record:record}"> {record.name}</f:link.action></td>
            <td><f:link.action action="show" pageUid="43" arguments="{record:record}"> {record.lei}</f:link.action></td>
        </tr>
    </f:for>
</f:widget.paginate>

Model:

class Lei extends \TYPO3\CMS\Extbase\DomainObject\AbstractEntity {

...
/**
 * abc
 *
 * @lazy
 * @var string
 */
protected $abc = '';
...

César Dueñas César Dueñas · Accepted Answer · 2020-02-17T21:28:50

I use in TYPO3 9.5. The next function in repository:

public function paginated($page = 1, $size = 9){
    $query = $this->createQuery();
    $begin = ($page-1) * $size;
    $query->setOffset($begin);
    $query->setLimit($size);
    return $query->execute();
}

And in the controller I am using arguments as parameter to send the page to load in a Rest action.

public function listRestAction()
{

    $arguments = $this->request->getArguments();
    $totalElements = $this->repository->total();
    $pages = ceil($totalElements/9);
    $next_page = '';
    $prev_page = '';
    #GET Page to load
    if($arguments['page'] AND $arguments['page'] != ''){
        $page_to_load = $arguments['page'];
    } else {
        $page_to_load = 1; 
    }
    #Configuration of pagination
    if($page_to_load == $pages){
        $prev = $page_to_load - 1;
        $prev_page = "http://example.com/rest/news/page/$prev";
    } elseif($page_to_load == 1){
        $next = $page_to_load + 1;
        $next_page = "http://example.com/rest/news/page/$next";
    } else {
        $prev = $page_to_load - 1;
        $prev_page = "http://example.com/rest/news/page/$prev";
        $next = $page_to_load + 1;
        $next_page = "http://example.com/rest/news/page/$next";
    }

    $jsonPreparedElements = array();
    $jsonPreparedElements['info']['count'] = $totalElements;        
    $jsonPreparedElements['info']['pages'] = $pages;
    $jsonPreparedElements['info']['next'] = $next_page;
    $jsonPreparedElements['info']['prev'] = $prev_page;
    $result = $this->repository->paginated($page_to_load);
    $collection_parsed_results = array();
    foreach ($result as $news) {
        array_push($collection_parsed_results, $news->parsedArray());
    }
    $jsonPreparedElements['results'] = $collection_parsed_results;
    $this->view->assign('value', $jsonPreparedElements);
}

The result of this, is a JSON like this:

{
"info": {
    "count": 25,
    "pages": 3,
    "next": "",
    "prev": "http://example.com/rest/news/page/2"
},
"results": [
    { ....}
] }

TYPO3 Extbase - Paginate through a large table (100000 records)

3 Answers