BigQuery bq command with asterisk (*) doesn't work in Compute Engine

3
votes

I have a directory with a file named file1.txt

And I run the command:

bq query "SELECT * FROM [publicdata:samples.shakespeare] LIMIT 5"

In my local machine it works fine but in Compute Engine I receive this error:

Waiting on bqjob_r2aaecf624e10b8c5_0000014d0537316e_1 ... (0s) Current status: DONE
BigQuery error in query operation: Error processing job 'my-project-id:bqjob_r2aaecf624e10b8c5_0000014d0537316e_1': Field 'file1.txt' not found.

If the directory is empty it works fine. I'm guessing the asterisk is expanding the file(s) into the query but I don't know why.

1
google-bigquerygoogle-compute-enginegoogle-cloud-platform
The shell is expanding the *, you need to escape it. See stackoverflow.com/questions/372807/… - Felipe Hoffa
Tried to escape it with backslash and set -f but it doesn't work.. - Eyal Levin
What shell are you using? It works fine with my GCE bash. (run 'ps -p $$') - Felipe Hoffa
I'm using bash on Ubuntu machine. uname- a output: Linux <machine-name> 3.16.0-36-generic #48~14.04.1-Ubuntu SMP Wed Apr 15 13:11:28 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux - Eyal Levin

1 Answers

3
votes

Apparently the bq command which is located at /usr/bin/bq has the following script: 

#!/bin/sh
exec /usr/lib/google-cloud-sdk/bin/bq ${@}

which expands the asterisk.

As a current workaround I'm calling /usr/lib/google-cloud-sdk/bin/bq directly.