Geek Answers Handbook

Google Cloud Dataproc by Google Datalab

Can I run code located on Google Cloud Datalab in Dataproc clusters?

The idea is to use the great interactivity and Datalab interface to run Apache Spark code.

+4
source share
2 answers

This is our radar, but not yet fully included as an init action for the Dataproc cluster.

Thank. Dinesh Kulkarni Product Manager, Datalab and Machine Learning, GCP

+3
source

Now, maybe just create a dataproc cluster using the following command:

gcloud dataproc clusters create $CLUSTERNAME \
    --project $PROJECT \
    --num-workers $WORKERS \
    --bucket $BUCKET \
    --metadata startup-script-url=gs://$BUCKET/setup/setup_env.sh,BUCKET=$BUCKET \
    --master-machine-type $VMMASTER \
    --worker-machine-type $VMWORKER \
    --initialization-actions \
         gs://dataproc-initialization-actions/datalab/datalab.sh \
    --scopes cloud-platform

, script: https://github.com/kanjih-ciandt/script-dataproc-datalab/tree/master

+1

More articles:


All Articles