BigQuery returning “query too large” on short query

Question

I've been trying to run this query:

SELECT
  created
FROM
  TABLE_DATE_RANGE(
      program1_insights.insights_,
      TIMESTAMP('2016-01-01'),
      TIMESTAMP('2016-02-09')
  )
LIMIT
  10

And BigQuery complains that the query is too large. I've experimented with writing the table names out manually:

SELECT
  created
FROM program1_insights.insights_20160101,
program1_insights.insights_20160102,
program1_insights.insights_20160103,
program1_insights.insights_20160104,
program1_insights.insights_20160105,
program1_insights.insights_20160106,
program1_insights.insights_20160107,
program1_insights.insights_20160108,
program1_insights.insights_20160109,
program1_insights.insights_20160110,
program1_insights.insights_20160111,
program1_insights.insights_20160112,
program1_insights.insights_20160113,
program1_insights.insights_20160114,
program1_insights.insights_20160115,
program1_insights.insights_20160116,
program1_insights.insights_20160117,
program1_insights.insights_20160118,
program1_insights.insights_20160119,
program1_insights.insights_20160120,
program1_insights.insights_20160121,
program1_insights.insights_20160122,
program1_insights.insights_20160123,
program1_insights.insights_20160124,
program1_insights.insights_20160125,
program1_insights.insights_20160126,
program1_insights.insights_20160127,
program1_insights.insights_20160128,
program1_insights.insights_20160129,
program1_insights.insights_20160130,
program1_insights.insights_20160131,
program1_insights.insights_20160201,
program1_insights.insights_20160202,
program1_insights.insights_20160203,
program1_insights.insights_20160204,
program1_insights.insights_20160205,
program1_insights.insights_20160206,
program1_insights.insights_20160207,
program1_insights.insights_20160208,
program1_insights.insights_20160209
LIMIT
  10

And not surprisingly, BigQuery returns the same error. This Q&A says that "query too large" means that BigQuery is generating an internal query that's too large to be processed. But in the past, I've run queries over way more than 40 tables with no problem.

My question is: what is it about this query in particular that's causing this error, when other, larger-seeming queries run fine? Is it that doing a single union over this number of tables is not supported?

By chance - are those can be all daily views that in turn reference real tables? — Mikhail Berlyant
Sure, this is a job id for a job that returned query too large: triggeredmail:bquijob_34a99916_152d90a4ad7 — Bryan
@FelipeHoffa, did the team uncover anything? Here is a fresh job_id: triggeredmail:bquijob_3465ac6a_152ebfc4ce5 Thank you for your help with this. — Bryan

Mikhail Berlyant Mikhail Berlyant · Accepted Answer · 2016-02-13T01:51:36

Answering question: what is it about this query in particular that's causing this error

The problem is not in query itself.
Query looks good.
I just run similar query against ~400 daily tables with total 5.8B (billion) rows of total size 5.7TB with:
Query complete (150.0s elapsed, 21.7 GB processed)

SELECT
  Timestamp
FROM
  TABLE_DATE_RANGE(
      MyEvents.Events_,
      TIMESTAMP('2015-01-01'),
      TIMESTAMP('2016-02-12')
  )
LIMIT
  10

You should look around - btw, are you sure you are not over-simplifying query in your question?

BigQuery returning “query too large” on short query

1 Answers