I've been having really slow Mongoose queries. I've narrowed it down to either being the way Mongoose creates objects or the actual latency in the db server requests. I have barely any data in the database (< 1000 objects in each of the 4 collections) and ran queries (which were similar to the queries in the product) manually and profiled them and they all ran < 2-4 ms.
Two big questions which I have not been able to figure out by searching online / looking at the docs:
- How do I get the raw queries that Mongoose is running on Mongo?
- How many trips to the db server does a call make - for example if I have a query with a couple populate calls, a sort call, and a nested populate call in the callback function how many trips does that make to the server.
Here's some of the latency times I'm seeing:
Login
localhost + localDB = 58ms
localhost + mongoHQ = 127ms
heroku + mongoHQ = 287ms
Get Data
localhost + localDB = 281ms
localhost + mongoHQ = 1657ms
heroku + mongoHQ = 2190ms
Update 1
I figured out how to log the queries and checked up on a bunch of them. I think its coming down to one query with a bunch of populate calls on it - I noticed that in generating the population part of the query, Mongoose outputs a Mongo query like:
users.find({ _id: { '$in': [ ObjectId("531ec0e17c0b16a82be4f506"),
ObjectId("531ec0e17c0b16a82be4f506"), ObjectId("531ec0e17c0b16a82be4f506") ...
in which the ids are repeated multiple times (in this case several hundred times) - I think this is the cause for the slowdown and testing the code I found that each populate call adds about .3s-.5s to the request time. The populate calls are also running on an array field - this also keeps the number of ids to be queried for in the 100s - low 1000 (of duplicate).
Is there a way to remove the duplicate entries when doing a populate call?
_id
s, without dups and see what happens). – WiredPrairie