Troubleshoot
This section contains a list of issues and workarounds.
Java error during startup
You might see [ERROR][c.a.o.s.s.t.OpenDistroSecuritySSLNettyTransport] [odfe-node1] SSL Problem Insufficient buffer remaining for AEAD cipher fragment (2). Needs to be more than tag size (16)
when starting Open Distro for Elasticsearch. This problem is a known issue with Java and doesn’t affect the operation of the cluster.
Kibana fails to start
If you encounter the error FATAL Error: Request Timeout after 30000ms
during startup, try running Kibana on a more powerful machine. We recommend four CPU cores and 8 GB of RAM.
Can’t open Kibana on Windows
Kibana doesn’t support Microsoft Edge and many versions of Internet Explorer. We recommend using Firefox or Chrome.
Can’t update by script when FLS, DLS, or field masking is active
The security plugin blocks the update by script operation (POST <index>/_update/<id>
) when field-level security, document-level security, or field masking are active. You can still update documents using the standard index operation (PUT <index>/_doc/<id>
).
Illegal reflective access operation in logs
This is a known issue with Performance Analyzer that shouldn’t affect functionality.
Multi-tenancy issues in Kibana
If you’re testing multiple users in Kibana and encounter unexpected changes in tenant, use Google Chrome in an Incognito window or Firefox in a Private window.
Beats
If you encounter compatibility issues when attempting to connect Beats to Open Distro for Elasticsearch, make sure you’re using the Apache 2.0 distribution of Beats, not the default distribution, which uses a proprietary license.
As of version 6.7, the default distribution of Beats includes a license check and fails to connect to the Apache 2.0 distribution of Elasticsearch.
Try this minimal output configuration for using Beats with the security plugin:
output.elasticsearch:
hosts: ["localhost:9200"]
protocol: https
username: "admin"
password: "admin"
ssl.certificate_authorities:
- /full/path/to/root-ca.pem
ssl.certificate: "/full/path/to/client.pem"
ssl.key: "/full/path/to/client-key.pem"
Even if you use the OSS version, Beats might check for a proprietary plugin on the Elasticsearch server and throw an error during setup. To disable the check, try adding these settings:
setup.ilm.enabled: false
setup.ilm.check_exists: false
Logstash
If you’re having trouble connecting Logstash to Open Distro for Elasticsearch, try this minimal output configuration, which works with the security plugin:
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "logstash-index-test"
user => "admin"
password => "admin"
ssl => true
cacert => "/full/path/to/root-ca.pem"
ilm_enabled => false
}
}
Dependency error during upgrade
If you run sudo yum upgrade
and receive a dependency error, Elasticsearch likely has a new minor version that the Open Distro for Elasticsearch plugins don’t support yet. You can install a specific, supported version of Elasticsearch to resolve the issue.
A temporary solution is to add the --skip-broken
option to upgrade the rest of your system:
sudo yum upgrade --skip-broken
Elasticsearch fails to start on Java 8 (RPM install)
If Elasticsearch fails to start and you’re using Java 8, verify that you set the symbolic link (symlink) correctly in step 6 of the RPM installation. If Java is installed to a non-standard path, try looking for tools.jar
using the following command:
ls /usr/lib/jvm/java-1.8.0-openjdk-*/lib/tools.jar
Then you can delete the old symlink and create a new one to the corrected path:
sudo rm /usr/share/elasticsearch/lib/tools.jar
sudo ln -s /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-0.amzn2.x86_64/lib/tools.jar /usr/share/elasticsearch/lib/