Geek Answers Handbook

Export Google Cloud Datastore and import into BigQuery programmatically

I'm looking for a way to export my cloud data warehouse and import it daily into BigQuery. The manual method is described on the google page . I did not find a clean way to automate it.

+4
source share
4 answers

There is no easy way to do this, but you can distinguish two parts: creating backup copies of appengine and uploading them to bigquery.

You can use scheduled backups to periodically back up your data warehouse ( https://cloud.google.com/appengine/articles/scheduled_backups ).

Script BigQuery (https://developers.google.com/apps-script/advanced/bigquery#load_csv_data) AppEngine cron, .

+3

. - gcloud beta datastore export.

script: https://github.com/chees/datastore2bigquery cron.

. , : https://www.youtube.com/watch?v=dGyQCE3bWkU

+1

@Jordan , :

1)

2)

3) Open the Big Query Web web interface and download it using the path to the Google Cloud file.

A complete tutorial with images can be found in this post .

0
source

You can use the following code. It mainly uses App Engine Cron jobs and BigQuery APIs.

https://github.com/wenzhe/appengine_datastore_bigquery

-1
source

More articles:


All Articles