16

Copy OpenSearch index data to S3

 2 years ago
source link: https://dev.to/laysauchoa/copy-opensearch-index-data-to-s3-27bb
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

opensearch (2 Part Series)

It is a good practice to perform backups of your OpenSearch data to another storage service. This way you can access your data and restore it if something unexpected happens to it.

In this article, you can find out how to dump your OpenSearch data to
an:

  • AWS S3 bucket

To copy the index data, we will be using elasticsearch-dump
tool. You can read the [instructions on GitHub (https://github.com/elasticsearch-dump/elasticsearch dump/blob/master/README.md) on how to install it. From this library, we will use elasticdump command to copy the input index data to a specific output.

Make sure to have elasticsearch-dump tool installed for the next steps.

Copying data from OpenSearch to AWS S3

Prerequisites

  • OpenSearch cluster as the input
  • AWS S3 bucket as the output

You need to collect the following information about your OpenSearch cluster and your AWS service:

OpenSearch:

  • SERVICE_URI: OpenSearch service URI.
  • INPUT_INDEX_NAME: the index that you aim to copy from your input source.

AWS S3:

  • AWS credentials (ACCESS_KEY_ID and SECRET_ACCESS_KEY).
  • S3 file path. This includes bucket name and file name, for e.g. s3://${BUCKET_NAME}/${FILE_NAME}.json

Find more information about AWS credentials in the AWS docs.

Export OpenSearch index data to S3

Use elasticsearch-dump command to copy the data from your OpenSearch cluster to your AWS S3 bucket. Use your
OpenSearch SERVICE_URI for the input . For the output, choose an AWS S3 file path including the file name that you want for your document.

elasticdump \
--s3AccessKeyId "${ACCESS_KEY_ID}" \
--s3SecretAccessKey "${SECRET_ACCESS_KEY}" \
--input=SERVICE_URI/INPUT_INDEX_NAME --output "s3://${BUCKET_NAME}/${FILE_NAME}.json"

Enter fullscreen mode

Exit fullscreen mode

That is how you can copy your OpenSearch data to a S3 bucket. 🙋🏻‍♀️

Image description


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK