Geek Answers Handbook

Google Cloud Storage - How to download a file from Python 3?

How can I upload a file to Google Cloud Storage from Python 3? After all, Python 2, if it does not lend itself to Python 3.

I looked and looked, but did not find a solution that really works. I tried boto , but when I try to generate the necessary .boto file through gsutil config -e , it continues to say that I need to configure authentication through gcloud auth login . However, I did the last several times without helping.

+22
source share
4 answers

Use the standard gcloud library, which supports both Python 2 and Python 3.

Example file upload to cloud storage

 from gcloud import storage from oauth2client.service_account import ServiceAccountCredentials import os credentials_dict = { 'type': 'service_account', 'client_id': os.environ['BACKUP_CLIENT_ID'], 'client_email': os.environ['BACKUP_CLIENT_EMAIL'], 'private_key_id': os.environ['BACKUP_PRIVATE_KEY_ID'], 'private_key': os.environ['BACKUP_PRIVATE_KEY'], } credentials = ServiceAccountCredentials.from_json_keyfile_dict( credentials_dict ) client = storage.Client(credentials=credentials, project='myproject') bucket = client.get_bucket('mybucket') blob = bucket.blob('myfile') blob.upload_from_filename('myfile')
+41
source

Simple feature to upload files to gcloud trash.

 from google.cloud import storage def upload_to_bucket(blob_name, path_to_file, bucket_name): """ Upload data to a bucket""" # Explicitly use service account credentials by specifying the private key # file. storage_client = storage.Client.from_service_account_json( 'creds.json') #print(buckets = list(storage_client.list_buckets()) bucket = storage_client.get_bucket(bucket_name) blob = bucket.blob(blob_name) blob.upload_from_filename(path_to_file) #returns a public url return blob.public_url

You can create a credential file using this link: https://cloud.google.com/storage/docs/reference/libraries?authuser=1#client-libraries-install-python

Asynchronous example:

 import asyncio import aiohttp # pip install aiofile from aiofile import AIOFile # pip install gcloud-aio-storage from gcloud.aio.storage import Storage BUCKET_NAME = '<bucket_name>' FILE_NAME = 'requirements.txt' async def async_upload_to_bucket(blob_name, file_obj, folder='uploads'): """ Upload csv files to bucket. """ async with aiohttp.ClientSession() as session: storage = Storage(service_file='./creds.json', session=session) status = await storage.upload(BUCKET_NAME, f'{folder}/{blob_name}', file_obj) #info of the uploaded file # print(status) return status['selfLink'] async def main(): async with AIOFile(FILE_NAME, mode='r') as afp: f = await afp.read() url = await async_upload_to_bucket(FILE_NAME, f) print(url) # Python 3.6 loop = asyncio.get_event_loop() loop.run_until_complete(main()) # Python 3.7+ # asyncio.run(main())
+12
source

Imports the Google Cloud client library ( credentials needed)

 from google.cloud import storage import os os.environ["GOOGLE_APPLICATION_CREDENTIALS"]="C:/Users/siva/Downloads/My First Project-e2d95d910f92.json"

Creates a client instance

 storage_client = storage.Client() buckets = list(storage_client.list_buckets()) bucket = storage_client.get_bucket("ad_documents")//your bucket name blob = bucket.blob('/chosen-path-to-object/{name-of-object}') blob.upload_from_filename('D:/Download/02-06-53.pdf') print(buckets)
+4
source

When installing the Google Cloud Storage API:

pip install google-cloud

will ModuleNotFoundError :

  from google.cloud import storage ModuleNotFoundError: No module named 'google'

Make sure you install as in the Cloud Storage Client Library docs :

pip install --upgrade google-cloud-storage

+1
source

More articles:


All Articles