dynamodb scan/query returning significantly less than 1MB in results

Question

I am performing a scan operation on one of my tables, and in the request I specify a "projectionExpression", which aims at reducing the amount of returned data. I am setting no limit to the scan (although I have also tried setting the limit to 50, 100 etc.) I am getting only about 20-30 results, weighing around 12KB-15KB total response data, I am using a javascript function to measure the size of the response.

I have also tried returning only the primary key in my porjectionExpression, to see if this affects the number of results I get, but I still get the same number of results.

I know from the documentation, that a scan operation will return only up to 1MB of data, but it surprises me that I get so few results, although my returning data is much less than 1MB, and that I did not specify a limit. I do get a LastEvaluatedKey and am able to continue scanning, but the nominal number of results seems very low.

Same happens with a query on an index.

So my question is: does the 1MB limit apply to the raw data or to the actual data returned in the response (the latter is the impression I got from the documentation).

Thank you, Ilan

Answer 1

The 1MB limit applies to the underlying data. A projection only reduces the amount of data sent over the wire.

You can use a GIS to project a small number of attributes if your items are quite large to make queries and scans less costly if you only need access to a subset of the fields.