Geek Answers Handbook

Analysis of the response from a large Google request

I get records from a large Google query using the gem "google-api-client" When I retrieve records from a table

client.execute(api_method: @compute_api.tabledata.list, parameters: {projectId: project, datasetId: dataset, tableId: table, maxResults: 10}).body

I get an answer like:

 { "kind": "bigquery#tableDataList", "etag": "\"iBDiwpngzDA0oFU52344ksWOrjA/-xEFKhLUueR63_XVaLG4z_mJt-8\"", "totalRows": "2000113", "pageToken": "BEIYURQ3J4AQAAAS23IIBAEAAUNAICAMCAGCBMFOCU======", "rows": [ { "f": [ { "v": "11873943041" }, { "v": " 639592585-0-1809110554@8.19.146.76 " }, { "v": "1.430438401E9" }, { "v": "1.430438402E9" }, { "v": "1.430438404E9" }, { "v": "1.430438862E9" }]}]}

Who has no column names in it, Does anyone have an idea on how to get column names along with data?

Currently, I need to make another API request to retrieve the schema and get the column names.

+5
source share
2 answers

I myself found the answer for this using the bigquery (bq) command line tool,

 bq --format=json query "select * from calls.details limit 10"

when using bq, if we do not provide the -quiet parameter, then it returns a response with additional text (status of a large request job), which causes a problem when parsing Json, as shown below.

 Waiting on bqjob_r36676afce1bcba8d_0000014f1ba0e36b_1 ... (0s) Current status: DONE [{"status":null,"userfield":null,"answer_stamp":"2015-05-01 00:00:04","term_roid":"a"}]

That's why I switched to using google api to retrieve data and again, which does not give you column names along with the data. But I found that we can remove this extra text using the -quiet option for the bq command, for example

 bq --quiet --format=json query "select * from calls.details limit 10"
+1
source

The API does not provide a way to get the schema and rows for an arbitrary table in a single API call. You need to call tables.get to get the schema, and then tabledata.list to get the rows.

However, if you execute the query, you can get the output schema and output strings in one API call using jobs.query or jobs.getQueryResults . You can even call jobs.getQueryResults in an already completed query job, even if that query job was done in other ways.

https://cloud.google.com/bigquery/docs/reference/v2/jobs/query https://cloud.google.com/bigquery/docs/reference/v2/jobs/getQueryResults

+1
source

More articles:


All Articles