Upgrade to 1.x.x

Open Distro for Elasticsearch 1.0.0 uses Elasticsearch 7.0.1, which has numerous breaking changes from 6.x.x. This page includes several important considerations when upgrading from Open Distro for Elasticsearch 0.x.x to 1.x.x.

Table of contents

1. Security index
2. Security YAML format
3. Discovery
4. Cluster and index names
5. Default shard count
6. Shards per node limit
7. Search response hits
8. Mapping types

Security index

0.x.x versions of the .opendistro_security index do not automatically work with the new version. Instead, you must back up your configuration, upgrade, and then use the -migrate option of the new securityadmin.sh to migrate your configuration YAML files to the new format and reinitialize the index:

# Example backup command for 0.x.x
./securityadmin.sh -r -cd ~/my-backup-dir -icl -nhnv -cacert /etc/elasticsearch/root-ca.pem -cert /etc/elasticsearch/kirk.pem -key /etc/elasticsearch/kirk-key.pem
# Example migration command for 1.x.x
./securityadmin.sh -migrate ~/my-backup-dir -nhnv -cacert /etc/elasticsearch/root-ca.pem -cert /etc/elasticsearch/kirk.pem -key /etc/elasticsearch/kirk-key.pem

If you use the Docker installation, create copies of the YAML files outside of the container before and after the migration so that you can pass them to the cluster.

Security YAML format

1.x.x versions of the Security configuration YAML files use a slightly different file format than 0.x.x versions. The -migrate option of the new securityadmin.sh helps you move from the old format to the new format.

Discovery

Node discovery settings have changed in 1.x.x. Instead of discovery.zen.ping.unicast.hosts and discovery.zen.hosts_provider, use discovery.seed_hosts and discovery.seed_providers. For an example, see Sample Docker Compose file.

Cluster and index names

Cluster and index names can no longer contain the : character.

Default shard count

Indices now default to one shard rather than five.

Shards per node limit

Clusters now default to a limit of 1,000 shards per data node, which you can change using the cluster.max_shards_per_node setting. Primary and replica shards both count towards this limit, but any shards that are part of a closed index do not.

Search response hits

hits.total, returned as part of search responses, is now an object, not an integer. If you created monitors using the Alerting plugin, you probably need to update them to use the new response format (hits.total.value rather than hits.total).

For example, this is the old format:

{
  "hits": {
    "total": 5
  }
}

This is the new format:

{
  "hits": {
    "total": {
      "value": 5,
      "relation": "eq"
    }
  }
}

Possible values for relation are eq (value is accurate) and gte (value is a lower bound).

If you want to use the old behavior, add rest_total_hits_as_int=true as a parameter in the search request:

GET _search?rest_total_hits_as_int=true
{
  "query": {
    "match": {
      "title": "wind"
    }
  }
}

To update your monitors:

1. Open Kibana.
2. Choose Alerting, Monitors, and a monitor.
3. Select a trigger and choose Edit.
4. Change any occurrences of ctx.results[0].hits.total to ctx.results[0].hits.total.value, and then choose Update.

Mapping types

Like Elasticsearch 6.x, indices can contain only one mapping type, but that type must now be named _doc. As a result, certain requests that used to require a mapping type no longer do.

For example, this is an old call to the bulk API:

POST _bulk
{ "index": { "_index" : "<index>", "_type" : "_doc", "_id" : "<id>" } }
{ "A JSON": "document" }

This is a new call:

POST _bulk
{ "index": { "_index" : "<index>", "_id" : "<id>" } }
{ "A JSON": "document" }