DEV Community

Cover image for Elasticsearch Snapshots to Minio
Ahmet Can Aydemir
Ahmet Can Aydemir

Posted on • Edited on • Originally published at ahmetcanaydemir.com

Elasticsearch Snapshots to Minio

Minio is a high-performance, self-hosted AWS S3 compatible object storage solution.

I will explain how we can store Elasticsearch snapshots in Minio.

Prerequisites

  • Minio
  • Elasticsearch

Creating Bucket

First, we need a bucket to store Easticsearch snapshots.

You can create a bucket at Minio UI Console. Click Buckets from the left menu and click Create Bucket.

Creating Bucket


Creating a Service Account

A service account is also required to establish a Minio connection with Elasticsearch.

You can create a service account at Minio UI Console. Click Service Accounts from the left menu and click Create Service Account.

Save the Secret Key and Access Key, we will use this information during the plugin installation.

Creating Service Account


Elasticsearch S3 Repository Plugin Installation

Since Minio is compatible with Amazon AWS S3, we will use the S3 repository plugin. To install the plugin, connect to all Elasticsearch nodes and follow the steps below.

First of all, install repository-s3 plugin with following command:

/usr/share/elasticsearch/bin/elasticsearch-plugin install repository-s3
Enter fullscreen mode Exit fullscreen mode

An S3 client named default will be created after the plugin is installed. Actions to be taken can be performed by this client. We will define the access_key and secret_key information to the default client for accessing Minio.

/usr/share/elasticsearch/bin/elasticsearch-keystore add s3.client.default.secret_key
# Waiting for secret key

/usr/share/elasticsearch/bin/elasticsearch-keystore add s3.client.default.access_key
# Waiting for access key
Enter fullscreen mode Exit fullscreen mode

After this stage, the Elasticsearch service should be restarted.

systemctl restart elasticsearch
Enter fullscreen mode Exit fullscreen mode

Adding Elasticsearch Repository

A repository named my-minio-repository can be created by sending the following request:

PUT _snapshot/my-minio-repository {
   "type": "s3",
   "settings": {
      "client": "default",
      "bucket": "elasticsearch-snapshots",
      "endpoint": "minio.example.com",
      "path_style_access": "true",
      "protocol": "http"
   } 
}
Enter fullscreen mode Exit fullscreen mode

Manual Snapshot

A snapshot named my-first-snapshot can be taken manually by sending the following request:

PUT /_snapshot/my-minio-repository/my-first-snapshot?wait_for_completion=true
Enter fullscreen mode Exit fullscreen mode

Taking Snapshots with Policy

If desired, the snapshot process can be automated using snapshot policy. For example, the policy below will save snapshots of all indexes to the Minio repository at 07:30 every day.

PUT _slm/policy/daily-snapshots
{
    "name": "<daily-snapshot-{now/d}>",
    "schedule": "0 30 7 * * ?",
    "repository": "my-minio-repository"
}
Enter fullscreen mode Exit fullscreen mode

Restoring snapshots

The my-first-snapshot backup can be restored with the following request:

POST /_snapshot/my-minio-repository/my-first-snapshot/_restore
Enter fullscreen mode Exit fullscreen mode

References

Snapshot and restore

Top comments (0)