Access Google BigQuery data from Jupyter local laptops

Question

I got some laptops and DataLab. For various reasons, I would like to access the same data from a local Jupyter laptop on my machine.

This question suggested several approaches that so far I can’t work.

In particular, the Gcloud library:

from gcloud import bigquery client = bigquery.Client() 

Give me a stack trace whose last line is:

 ContextualVersionConflict: (protobuf 2.6.1 (/usr/local/lib/python2.7/dist-packages), Requirement.parse('protobuf!=3.0.0.b2.post1,>=3.0.0b2'), set(['gcloud'])) 

The Pandas library seems promising:

 df=pd.io.gbq.read_gbq('SELECT CCS_Category_ICD9, Gender, Admit_Month FROM [xxxxxxxx-xxxxx:xxxx_100MB_newform.xxxxxx_100MB_newform]ORDER by CCS_Category_ICD9', project_id='xxxxxxxx-xxxxx') 

Also gives me a stack trace:

 IOError: [Errno 2] No such file or directory: '/usr/local/lib/python2.7/dist-packages/httplib2-0.9.1.dist-info/METADATA' 

Perhaps I have a problem with auth in the Pandas approach, although my browser is currently Auth'd for the project? or am i lacking addiction?

Any suggestions or recommendations are appreciated.

What is the best way to access a BigQuery data source from a local Jupyter laptop?

Answer 1

Based on the error from gbq.read() , it seems that httplib2 may not be installed correctly. On the pandas installation page, there are several additional dependencies required to support Google BigQuery (one of them is httplib2). To reinstall / fix the installation, try:

 pip install httplib2 --ignore-installed 

After installing additional dependencies to support Google BigQuery, the following code should work:

 from pandas.io import gbq df = gbq.read_gbq('SELECT * FROM MyDataset.MyTable', project_id='my-project-id') 

Answer 2

If you used special ways to access GCP for Datalab, then you can try instead https://github.com/googledatalab/datalab . This will give you Datalab compatible functionality in a Jupyter laptop.

Answer 3

I had the same problem, but I managed to solve it by installing the gdq version in conda, I already installed the anaconda distribution for python, so maybe some link is missing if you use pip

conda install pandas -gbq --channel conda-forge this team did business