250000 Grouped products in Magento

3
votes

I've started working on a new Magento webshop which has roughly 250000 different products. Each product can have different conditions (new, used, damaged, etc., each with their own price.). Magento doesn't seem to have a method to implement this at the moment. Of those 250000 products, there are about 150000 different conditions in stock and another 150000 conditions which aren't in stock but do have a price (which can be put on the wishlist).

Some numbers: 1500 categories, drop down attributes (country) with > 300 options, integer attributes (year). Starting with two websites 6 languages each.

I've thought out two solutions to solve this problem:

Grouped / Simple product structure

We create a grouped product which is the container product, each condition will be a different simple product. We'll relate these products to the grouped product.

The nice thing about this solution is that is pretty easy to implement, we'll have to import the data in the correct way and all the information is exactly presented in the way we want.

New Product type

We create a new product type which can have multiple conditions, each with their own inventory. On checkout the inventory is substracted. The problem with this is that implementing the CatalogInventory model in this way probably is difficult and building a whole product type is time consuming and bugprone in general.

The advantage of this solution is this that there will be about 2-2.5x times less products in the system.

Others

Are there any other options to solve this? Maybe there is a module that does all this?

In conclusion: Of course I prefer the first solution but can Magento handle that? Does anyone have experience with this much Grouped Products? The system will have about 550000 products (grouped + simple) products in the system, what will be the performance implications of this? What happens when the site grows and we'll have twice the amount of products?

1
magentomagento-1.7
You will hit performance bottlenecks with that many products and that many relations. The specifics will be impossible to predict. Don't proceed without Magento talent on board to help you scale this. You're entering into territory where it might be worth it to write custom software from scratch to do this. (also, it sounds like you're using grouped products as configurable products — why not just use configurable products?) – Alan Storm
I've created a little test to see if Magento can even handle the 250000 grouped products and that seemed to run just fine (on my local machine: 3 seconds uncached, 0,5 seconds when hitting a query cache). It seems the performance bottleneck in Magento is getting the total count of the collection for each page. I haven't tested under load, but with proper caching catalog pages wont take a huge hit on the database. – Paul Hachmang
Don't know about the impact of the relations though.I'm not that good of a developer to write such an eCommerce solution form scratch. The business is growing pretty substantially so the demand for a scaleable platform getting bigger by the month. – Paul Hachmang
— we're veering into business from programming, but you're going to see lots of little failures under scale with that many products, especially if you pull in the relations. There's no market for out of the box software that solves your specific program. It's custom software, whether your develop that internally yourself or with a partner. – Alan Storm
You can use newrelic.com in order to see where you have problems loading or so and scale it accordingly. – Mike

1 Answers

1
votes

Without knowing more details I'd lean towards using a new product type or just adding the feature independently of product types if you have a use for configurable products (I definitely wouldn't try to duplicate the configurable product type). I'd disable inventory management and use some additional tables which hold individual item inventory with the per-item conditions and maintain a separate inventory that way. Use events and overrides to control the CatalogInventory stock status as needed. Creating new products constantly which are largely duplicates seems like a hassle worth avoiding if this is a long-term endeavor that needs to scale.

However, the Grouped/Simple method might be a viable short-term solution and appropriate if the project is in it's early stages and can't afford a huge initial expenditure. If well-planned, a script should be able to convert all of the old grouped/simple products into your new product type when ready to launch.