How to fix memory leaks in Magento related to load() method?

Question

I'm importing data(products, customers, and orders) from a Magento store to another Magento store.

For example, when I try to import the products, I need to load the product to check if it already exists and use some attributes of the loaded product. I'm using the method Mage::getModel('catalog/product')->load($sku) and I run the script using command line.

Something like: $ php -f shell/mymodule.php

The problem is that Magento doesn't stop to increase the memory usage when I use the load() method in a loop.

foreach ($result['items'] as $item) {
     echo $index . ' - Memory: ' . memory_get_usage() . "\n";

     /** @var Mage_Catalog_Model_Product $product */
     $product = Mage::getModel('catalog/product');
     $product->load($product->getIdBySku($item['sku']));

     $product->getOptionInstance()->unsetOptions()->clearInstance();
     unset($product);
     gc_collect_cycles();

     $index++;
}

1 - Memory: 28147376
2 - Memory: 34600320
3 - Memory: 34661976
4 - Memory: 34721128
5 - Memory: 34776600
...
10 - Memory: 35051216
...
100 - Memory: 40148904

In some cases I need to import thousands of products, which causes a memory overflow. As you can see in the script above, I also tried to run some optimization functions within the foreach but that isn't enough in performance to prevent memory overflow.

$product->getOptionInstance()->unsetOptions()->clearInstance();
unset($product);
gc_collect_cycles();

I found a kind of solution that changes the Magento core, but it was made for Magento 1.4 and doesn't work for Magento 1.9 that I am using.

https://ringsdorff.net/2009/07/23/guest-post-fix-for-memory-leaks-in-magento/

Is there any effective solution to prevent the increase of memory usage in Magento 1.9?

Try to check how magento walk() function works. Its an iterator through collection. You may find some clue how you can avoid this. — Kingshuk Deb
Load within a loop causes the memory used to increment by the size of the collection for every iteration until completion or memory exhaustion, whichever comes first. magento.stackexchange.com/questions/69488/… — Fiasco Labs

arushi arushi · Accepted Answer · 2016-05-24T17:42:16

You can use Mage::getSingleton() method, it will reduce the memory issue by 50% and also try to execute the data in chunks.

How to fix memory leaks in Magento related to load() method?

2 Answers