Magento catalog URL rewrite indexing taking too long

0
votes

I've been dealing with this problem with around 10k+ products in two store views in magento 1.7.

The URL indexing process took around 30 hours to change its state to ready. Also i found multiple entries of the same product being made in the core_url_rewrite table and the number of rows now reached upto 6500k.

This is causing deadlocks. I tried clearing the locks but that didn't help. Is there a workaround solution for this problem as this is magento core functionality?

1
magentoindexingmagento-1.7

1 Answers

2
votes

There's some good general advice on the Magento Stack Exchange site covering common indexing problems.

It's also common for larger store to create a rewrite/code-pool-override for the following method

#File: app/code/core/Mage/Catalog/Model/Resource/Url.php
protected function _getProducts($productIds, $storeId, $entityId, &$lastEntityId)
{
    //...
}

This method queries for the products that need a URL reindex. By default, this includes all simple and configurable products. However, if you're not displaying simple products individually, you can tweak this query to not include those products. That can greatly reduce the number of URLs Magento needs to generate.