Geek Answers Handbook

How to set up a structure for backing up elasticsearch data in Google Compute Engine?

I have an e elasticsearch environment configured on GCE (Google Compute Engine) with two nodes, so there are two virtual machines and I need to create a backup strategy for this. At first I thought that I could use elasticsearch to back up all my data to a given storage, as the API supports several ways to store a snapshot.

  • Common file system such as NAS
  • Amazon s3
  • HDFS (Hadoop Distributed File System)
  • Sky blue cloud

I tried using the shared file system option, but this requires the storage location to be shared between nodes. Is there a way I can do this on GCE?

curl -XPUT http://xxxx:9200/_snapshot/backup -d '{ "type": "fs", "settings": { "compress" : true, "location": "/elasticsearch/backup" }

} '

nested: RepositoryVerificationException[[backup] store location [/elasticsearch/backup] is not shared between node

I know that for storing backups there is an AWS plugin for elasticsearch search . Is there a plugin for Google Cloud Storage? Is it possible to do this?

If any of the above options is not possible, is there any other recommended strategy for backing up my data?

+8
elasticsearch google-compute-engine google-cloud-platform
source share
3 answers

Elasticsearch now has a Google Cloud Storage Plugin , so this is supported.

+2
source share

You may be able to use the S3 plugin with Google Cloud Storage through interaction. See here for more details.

Alternatively, you can simply create a regular backup on the file system and then upload it to the cloud using gsutil.

0
source share

I have the same problem with my ES cluster (5 nodes) in Google Cloud. We cannot use local backups on the disk itself, as mentioned above, since not every node has all the data in my case.

It seems to me that the only way is to create a small machine with a large disk and install this disk as a shared disk on all 5 ES-nodes that I have in the same path so that we can use the option "General file system".

0
source share

More articles:


All Articles