Geek Answers Handbook

BigQuery Compressed Answer

Is it possible that BigQuery will return a compressed / gzip response? I noticed when running a query like:

select * from [dataSet.tableId]

from a table with a large number of columns ( diagram ), but only 1 MB in size, the response is 13 MB. It's clear that BigQuery FLATTENs is one of the entries, so the answer is more due to the JSON structure. However, the resulting table in BigQuery is 2.04 MB in size. I do not use any BigQuery client libraries, but instead do direct HTTP requests instead. Is there any work around to get a smaller answer with actual tabular data?

thanks

+4
source share
2 answers

HTTP-API, - User-Agent Accpet-Encoding. - , , , gzip.

Accept-Encoding: deflate, gzip
User-Agent: gzip

AppEngine : http://code.google.com/appengine/kb/general.html#compression. BigQuery AppEngine, AppEngine, , .

+2

tabledata . ( , jobs.getQueryResults(), .

, , "select *", tabledata.list() , "select *" , tabledata.list() .

:

PROJECT=my_project
DATASET=my_dataset
TABLE=my_table
BASE_URL=https://www.googleapis.com/bigquery/v2
TABLES_URL=${BASE_URL}/projects/${PROJECT}/datasets/${DATASET}/tables
TABLEDATA_URL=${TABLES_URL}/${TABLE}/data
curl  -H "Authorization: Bearer ${AUTH_TOKEN}" \
    -H "Content-Type: application/json" \
    -X GET \
    "${TABLEDATA_URL}?maxResults=10"

, 'pageToken'. :

PAGE_TOKEN=<page token from response>
curl  -H "Authorization: Bearer ${AUTH_TOKEN}" \
    -H "Content-Type: application/json" \
    -X GET \
    "${TABLEDATA_URL}?maxResults=10&pageToken=${PAGE_TOKEN}"
+2

More articles:


All Articles