Error importing Google Cloud Bigquery api module in python app

4
votes

I am trying to import bigquery into my python app from google.cloud import bigqueryand run it locally with dev_appserver.py, but I receive an error:

File "/Volumes/Budhi/Users/anjas/Projects/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/Volumes/Budhi/Users/anjas/Projects/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/Volumes/Budhi/Users/anjas/Projects/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/Volumes/Budhi/Users/anjas/Projects/wordworks/urlworker/main.py", line 9, in <module>
from google.cloud import bigquery
File "/Volumes/Budhi/Users/anjas/Projects/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/sandbox.py", line 999, in load_module
raise ImportError('No module named %s' % fullname)
ImportError: No module named google.cloud.bigquery

I have installed bigquery lib with pip:

pip install --upgrade google-cloud-bigquery

Also I've tried installing it as 3d party library into lib directory with no result.

Though it works when I try importing bigquery lib from python shell:

Python 2.7.10 (default, Jul 30 2016, 18:31:42) 
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type "help", "copyright", "credits" or "license" for more  information.
>>> from google.cloud import bigquery
>>>

Update:

It seems that "google" module installed to project/lib/ folder is clashing with "google" module in google-cloud-sdk/platform/google_appengine.

When trying to do

from google.cloud import bigquery

python looks for the module inside google-cloud-sdk/platform/google_appengine/google not inside project/lib/google/cloud

Any ideas?

4
pythongoogle-app-enginegoogle-bigquery
did you installed the library on your lib folder exists on project's root ? – Avinash Raj
sure, I've tried installing some other libs like Flask and there was no issue importing them – Andrei Ivasiuc
I am having this same issue. All my other installed vendor libs work fine. bigquery however I cannot import locally or remotely. I've followed every solution I can find with no success. – Halsafar

4 Answers

2
votes

I experienced exactly the same error and fixed this by deleting google folder in appengine sdk or you can just try to copy that to your lib folder . This will fix the problem on local development environment but you need to revert this when deploying your service on production.

I could not find a workaround to keep both google-cloud-sdk/platform/google_appengine/google and project/lib/google/cloud. You need to delete one of them

0
votes

Did you add the lib folder in the appengine_config.py?

from google.appengine.ext import vendor
vendor.add('lib')
0
votes

I had the same problem. If you're running this on your local environment, you may want to try adding the lib folder location into your $PYTHONPATH environment variable or adding below code lines before you import the bigquery lib which inserting the lib folder location into the a list of strings that specifies the search path for modules.

import sys    
sys.path.insert(0, 'lib')

I believe this works because now python files don't need to be inside of a package.

0
votes

I think you are experiencing something very close to what i am experiencing, though i am seeing it during tests.

I think it might be because of the path manipulation needed by google sdk: the libs shipped with the sdk are being seen before the ones you are putting in the libs folder.

Here's what happens to me, with some details:

Wrong dependency in Google Cloud SDK for google-auth?