POSTGIS query optimization in RAILS

Question

I am using postgis2.0 with postgres 9.1 db. My goal is to write as close as possible to an optimized query to get nearby locations within certain radius and out put them with their distance order. The Location model has attribute latlong which of spatial type postgis extension and a method distance_from to calculate distance from a given POINT(long lat). I wrote query as follow in rails code:

def self.nearby(lat, long, radius)
    nearby = Location.where("ST_DWithin(ST_GeomFromEWKB(latlong), ST_GeomFromText('POINT(#{long} #{lat})', 4326),?, false )", radius)
    .order("ST_Distance_Sphere(ST_GeomFromEWKB(latlong) , ST_GeomFromText('POINT(#{long} #{lat})', 4326) ) ")
    .map{|ar| 
      { "id" => ar.id,
        "distance" => ar.distance_from(lat, long)
      } 
    }
end

I can see that i double calculate the distance twice with order clauses and map clause but can't think of how should i store immediate value of distance from sql query. So in the map{} i recalculate it.

 `.order("ST_Distance_Sphere(ST_GeomFromEWKB(latlong) , ST_GeomFromText('POINT(#{long} #{lat})', 4326) ) ")`

"distance" => ar.distance_from(lat, long)

If I am not wrong, using ST_DWithin in my case could help me get answer quickly whether a location is within rather than calculate distance first. So if say one query would return only 10-100 locations, ST_DWithin will help speed up query than purely use STDistance.

How much more can I improve? My locations db size will be around 10000 records. Appreciate your time, thanks.

Benjamin M Benjamin M · Accepted Answer · 2013-03-04T11:03:02

At the moment I'm also working at an application using Rails & PostGIS. :-)

For complex queries I chose the way to write plain SQL instead of using ActiveRecords methods, makes things a bit more easy to maintain.

Yours is:

SELECT
  *
FROM location
WHERE
  ST_DWithin(ST_GeomFromEWKB(latlong),
    ST_GeomFromText('POINT(#{long} #{lat})', 4326), ?, false)
ORDER BY
  ST_Distance_Sphere(ST_GeomFromEWKB(latlong),
    ST_GeomFromText('POINT(#{long} #{lat})', 4326))

By the way, those coordinates are called latlon without the g ;-)

Give me a few minutes I'll try to figure out how Postgres will optimize your query and if it's needed to optimize it by hand.

This query can be faster (If there are a lot of matches), but can also be slower, because ST_DWithin is much faster than ST_Distance or ST_Distance_Sphere. So please test it with a huge amount of data:

SELECT
  *
FROM ( 
  SELECT
    l.*,
    (
      ST_DISTANCE_SPHERE(ST_GeomFromEWKB(latlong),
        ST_GeomFromText('POINT(#{long} #{lat})', 4326))
    ) AS d
  FROM location l
) x
WHERE d < ?
ORDER BY d

Explanation:

Your original query will first filter the results using the fast ST_DWithin and afterwards call ST_Distance_Sphere for all found objects.

My query will calculate ST_Distance_Sphere for ALL objects in database, and afterwards filter them using an integer comparison.

For use in Rails, you might simply call Location.find_by_sql(...)

EXPLAIN ANALYZE

(my table is called measurement and the column containing the Point is called groundtruth)

Your query:

Sort  (cost=341.05..341.06 rows=1 width=172) (actual time=3.676..3.731 rows=816 loops=1)
  Sort Key: (_st_distance(geography(groundtruth), '0101000020E6100000EE7C3F355EF24F4019390B7BDA011940'::geography, 0::double precision, false))
  Sort Method: quicksort  Memory: 139kB
  ->  Bitmap Heap Scan on measurement m  (cost=9.67..341.04 rows=1 width=172) (actual time=0.330..3.257 rows=816 loops=1)
        Recheck Cond: (groundtruth && '01030000000100000005000000EE7C3F355E724D4064E42CEC6907F43FEE7C3F355E724D408C9C853DED80264077BE9F1A2F3951408C9C853DED80264077BE9F1A2F39514064E42CEC6907F43FEE7C3F355E724D4064E42CEC6907F43F'::geometry)
        Filter: (('0101000000EE7C3F355EF24F4019390B7BDA011940'::geometry && st_expand(groundtruth, 5::double precision)) AND _st_dwithin(groundtruth, '0101000000EE7C3F355EF24F4019390B7BDA011940'::geometry, 5::double precision))
        ->  Bitmap Index Scan on groundtruth_idx  (cost=0.00..9.67 rows=189 width=0) (actual time=0.186..0.186 rows=855 loops=1)
              Index Cond: (groundtruth && '01030000000100000005000000EE7C3F355E724D4064E42CEC6907F43FEE7C3F355E724D408C9C853DED80264077BE9F1A2F3951408C9C853DED80264077BE9F1A2F39514064E42CEC6907F43FEE7C3F355E724D4064E42CEC6907F43F'::geometry)
Total runtime: 3.932 ms

My query:

Sort  (cost=9372.84..9391.92 rows=7634 width=172) (actual time=19.256..19.312 rows=816 loops=1)
  Sort Key: (st_distance(m.groundtruth, '0101000000EE7C3F355EF24F4019390B7BDA011940'::geometry))
  Sort Method: quicksort  Memory: 139kB
  ->  Seq Scan on measurement m  (cost=0.00..8226.01 rows=7634 width=172) (actual time=0.040..18.863 rows=816 loops=1)
        Filter: (st_distance(groundtruth, '0101000000EE7C3F355EF24F4019390B7BDA011940'::geometry) < 5::double precision)
Total runtime: 19.396 ms

As you can see: There were just 816 matching rows from 22901. And my query took much longer.

If I make the distance bigger, both queries become equal fast.

If all rows (= 22901 rows) are within the search radius, my query is a little bit faster: 180 vs. 210ms.

So you'd probably stay with your solution ;)

Another suggestion to maybe gain 1-2% performance: Don't use GeomFromText, you could just use rgeo to directly give your database a Point object as parameter, instead of 2 coordinates.

POSTGIS query optimization in RAILS

1 Answers

EXPLAIN ANALYZE