Optimize joined order by query

1
votes

I Have the following query:

SELECT `p_products`.`id`, `p_products`.`name`, `p_products`.`date`, 
       `p_products`.`img`, `p_products`.`safe_name`, `p_products`.`sku`, 
       `p_products`.`productstatusid`, `op`.`quantity`
FROM `p_products` 
INNER JOIN `p_product_p_category` 
        ON `p_products`.`id` = `p_product_p_category`.`p_product_id`
LEFT JOIN (SELECT `p_product_id`,`order_date`,SUM(`product_quantity`) as quantity 
           FROM `p_orderedproducts` 
           WHERE `order_date`>='2013-03-01 16:51:17' 
           GROUP BY `p_product_id`) AS op
       ON `p_products`.`id` = `op`.`p_product_id`
WHERE `p_product_p_category`.`p_category_id` IN ('15','23','32') 
  AND `p_products`.`active` = '1'
GROUP BY `p_products`.`id`
ORDER BY `date` DESC

Explain says:

id  select_type table   type    possible_keys   key key_len ref rows    Extra
1   PRIMARY p_product_p_category    ref p_product_id,p_category_id,p_product_id_2   p_category_id   4   const   8239    Using temporary; Using filesort
1   PRIMARY p_products  eq_ref  PRIMARY PRIMARY 4   pdev.p_product_p_category.p_product_id  1   Using where
1   PRIMARY   ALL NULL    NULL    NULL    NULL    78  
2   DERIVED p_orderedproducts   index   order_date  p_product_id    4   NULL    201 Using where

And I have indexes on a number of columns including p_products.date.

Problem is the speed when there are more then 5000 products in a number of categories. 60000 products take >1 second. Is there any way to speed things up?

This also holds true if I remove the left join in which case the result is:

id  select_type table   type    possible_keys   key key_len ref rows    Extra
1   SIMPLE  p_product_p_category    index   p_product_id,p_category_id,p_product_id_2   p_product_id_2  8   NULL    91167   Using where; Using index; Using temporary; Using filesort
1   SIMPLE  p_products  eq_ref  PRIMARY PRIMARY 4   pdev.p_product_p_category.p_product_id  1   Using where

The intermediatate table p_product_p_category has indexes on both p_product_id and p_category_id aswell as a combined index with both.

Tries Ochi's suggestion and ended up with:

id  select_type table   type    possible_keys   key key_len ref rows    Extra
1   PRIMARY   ALL NULL    NULL    NULL    NULL    62087   Using temporary; Using filesort
1   PRIMARY nr1media_products   eq_ref  PRIMARY PRIMARY 4   cats.nr1media_product_id    1 Using where
2   DERIVED nr1media_product_nr1media_category  range   nr1media_category_id    nr1media_category_id    4   NULL    62066   Using where

I think I can simplify the question to how can I join my products on the category intermediate table to fetch all unique products for the selected categories, sorted by date.

EDIT:

This gives me all unique products in the categories without using a temp table for ordering or grouping:

SELECT
    `p_products`.`id`,
    `p_products`.`name`,
    `p_products`.`img`,
    `p_products`.`safe_name`,
    `p_products`.`sku`,
    `p_products`.`productstatusid`
FROM
    p_products
WHERE
    EXISTS (
        SELECT
            1
        FROM
            p_product_p_category
        WHERE
            p_product_p_category.p_product_id = p_products.id
        AND p_category_id IN ('15', '23', '32')
    )
AND p_products.active = 1
ORDER BY
    `date` DESC

Above query is very fast, much faster then the join using group by order by (0.04 VS 0.7 sec), although I don't understand why it can do this query without temp tables.

I think I need to find another solution for the orderedproducts join, it still slows the query down to >1 sec. Might make a cron to update the ranking of the products sold once every night and save that info to the p_products table.

Unless someone has a definitive solution...

2
mysqlperformancegroup-bysql-order-byleft-join

2 Answers

2
votes

You are joining every type of category to products - only then it gets filtered by category id

try to limit your query as soon as possible for e.g. instead of

INNER JOIN `p_product_p_category`

do

INNER JOIN ( SELECT * FROM `p_product_p_category` WHERE `p_category_id` IN ('15','23','32') )

so that you will be working on smaller subset of products right from begining

2
votes

One possible solution would be to remove the derived table and just do a single Group By:

Select P.id, P.name, P.date
    , P.img, P.safe_name, P.sku
    , P.productstatusid
    , Sum( OP.product_quantity ) As quantity
From p_products As P
    Join p_product_p_category As CAT
        On p_products.id = CAT.p_product_id

    Left Join p_orderedproducts As OP
        On OP.p_product_id = P.id
            And OP.order_date >= '2013-03-01 16:51:17' 

Where CAT.p_category_id In ('15','23','32') 
    And P.active = '1'
Group By P.id, P.name, P.date
    , P.img, P.safe_name, P.sku
    , P.productstatusid
Order By P.date Desc