BigQuery: Response too large to return when using GROUP EACH BY

Question

The table I am working with has 3 fields:

userId, timestamp, version

I am running the following query:

select  userid, MAX(version) as current_version FROM  my_table GROUP EACH BY userId;

The response I get is:

"errors": [
 {
  "reason": "responseTooLarge",
  "message": "Response too large to return."
 }

The size of the table is 644MB and it has 12,279,432 rows.

I thought GROUP EACH BY does not have the result size restrictions because it is distributed across multiple nodes. Anyway, What can I do about it?

The table stores each client version update by a user. The idea is to get the version of the client for a specific user at any point in time. For this query the goal is to just get the latest version of client for each user. Currently, the user base is over 17M .... — mbsheikh

Felipe Hoffa Felipe Hoffa · Accepted Answer · 2013-05-11T06:18:21

According to the comments, the user base is over 17 million rows? This means the query response will have at least 17 million rows, a result too large to handle.

The right query will depend on what your goal is. Do you really want to get a 17 million row answer? Or you only care about the max(version) for a particular set of users?

For example:

SELECT userid, MAX(version) AS current_version
FROM my_table
WHERE userId IN('user1', 'user2', ...)
GROUP BY userId;

BigQuery: Response too large to return when using GROUP EACH BY

1 Answers