Geek Answers Handbook

Register all BigQuery queries

Is it possible for all BigQuery queries to go into a file in Cloud Storage (or even better in a BigQuery table)? It seems that the -apilog option available in bq is mainly for debugging purposes, but what I would like to do is keep track of all requests, just like logging all requests to access a specific file in CloudStorage.

To be more specific, I do not just want to register my own queries, but (a) the queries of all users within the same project and optimally also (b) the queries of anyone who touches the table in the data set that I myself have.

+4
source share
2 answers

, GCP .

: BQ

+1

CLI bq ls -j -a . .

a bq show -j <job_id>, json:

bq show --format=prettyjson -j job_joQEqPwOiOoBlOhDBEgKxQAlKJQ

, , .....

{
  "configuration": {
    "dryRun": false, 
    "query": {
      "createDisposition": "CREATE_IF_NEEDED", 
      "destinationTable": {
        "datasetId": "", 
        "projectId": "", 
        "tableId": ""
      }, 
      "query": "", 
      "writeDisposition": "WRITE_TRUNCATE"
    }
  }, 
  "etag": "", 
  "id": "", 
  "jobReference": {
    "jobId": "", 
    "projectId": ""
  }, 
  "kind": "bigquery#job", 
  "selfLink": "", 
  "statistics": {
    "creationTime": "1435006022346", 
    "endTime": "1435006144730", 
    "query": {
      "cacheHit": false, 
      "totalBytesProcessed": "105922683030"
    }, 
    "startTime": "1435006023171", 
    "totalBytesProcessed": "105922683030"
  }, 
  "status": {
    "state": "DONE"
  }, 
  "user_email": ""
}

API, allUsers https://cloud.google.com/bigquery/docs/reference/v2/jobs/list#allUsers

+3

More articles:


All Articles