DEV Community

Oshan Wisumperuma
Oshan Wisumperuma

Posted on

Elasticsearch Tune for indexing speed

Increase the refresh interval and disable replicas

Screenshot from 2020-11-18 10-33-54

  • On some projects, indexing happens at a different pace, not while the application is in use. In my current project, we follow a similar approach. So I'm disabling the refresh interval because we are not going to search while indexing is on. Also, I'll enable replication when all the indexing complete.
curl --location --request PUT 'localhost:9200/<index>/_settings' \
--header 'Content-Type: application/json' \
--data-raw '{
    "settings": {
        "index": {
            "number_of_replicas": "0",
            "refresh_interval": "600s"
        }
    }
}'
Enter fullscreen mode Exit fullscreen mode

call GET endpoint to confirm the new variables.

Screenshot from 2020-11-18 10-33-29

curl --location --request GET 'localhost:9200/<index>/_settings'
Enter fullscreen mode Exit fullscreen mode
  • collecting these values initially will be helpful, so you can reset the configurations after indexing is complete.

bulk insert

Screenshot from 2020-11-18 10-45-23

  • adjust the batch size of the elasticsearch-model import function argument.

API: https://rubydoc.info/gems/elasticsearch-model/Elasticsearch/Model/Importing/ClassMethods#import-instance_method

Top comments (0)