Elasticsearch Security Best Practices

Securing Elasticsearch comes down to five things: keep the built-in security enabled, encrypt both the transport and HTTP layers with TLS, give every user and application a least-privilege role (down to fields and documents where needed), use short-lived API keys for applications, and turn on audit logging. This guide covers each one with configuration examples.

How do you set up authentication and authorization?

Keep security features enabled

Security is on by default: xpack.security.enabled defaults to true. On the first start of a new node, Elasticsearch also generates TLS certificates, writes the TLS settings to elasticsearch.yml, and sets a password for the elastic superuser. If you manage certificates yourself, the explicit settings look like this:

# elasticsearch.yml
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.http.ssl.enabled: true

Never run a shared or production cluster with xpack.security.enabled: false.

User Management

Create role-based access control:

# Create a custom role
curl -X POST "localhost:9200/_security/role/log_reader" -H 'Content-Type: application/json' -d'
{
  "cluster": ["monitor"],
  "indices": [
    {
      "names": ["logs-*"],
      "privileges": ["read", "view_index_metadata"]
    }
  ]
}'

# Create a user with the role
curl -X POST "localhost:9200/_security/user/john_doe" -H 'Content-Type: application/json' -d'
{
  "password": "secure_password",
  "roles": ["log_reader"],
  "full_name": "John Doe",
  "email": "john.doe@company.com"
}'

How do you encrypt traffic?

Transport Layer Security (TLS)

Configure TLS for node-to-node communication:

# elasticsearch.yml
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

HTTP SSL/TLS

Secure client connections:

xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: elastic-certificates.p12

How do you restrict network access?

IP Filtering

Restrict access by IP address on self-managed clusters. Allow rules take precedence over deny rules, and the HTTP filter settings default to the transport values unless you set them explicitly:

xpack.security.transport.filter.allow: "192.168.1.0/24"
xpack.security.transport.filter.deny: "_all"
xpack.security.http.filter.allow: "192.168.1.0/24"
xpack.security.http.filter.deny: "_all"

On Kubernetes (ECK), Elastic recommends Kubernetes network policies over IP filters, because pod IP addresses change.

Firewall Configuration

Configure firewalls to:

Allow only necessary ports (9200 for HTTP, 9300 for transport)
Restrict access to management interfaces
Block unnecessary outbound connections

How do you protect sensitive fields and documents?

Field-Level Security

Restrict access to sensitive fields. Denied fields in except must be a subset of the granted fields, so grant everything and carve out what's sensitive:

{
  "cluster": ["monitor"],
  "indices": [
    {
      "names": ["customer_data"],
      "privileges": ["read"],
      "field_security": {
        "grant": ["*"],
        "except": ["ssn", "credit_card"]
      }
    }
  ]
}

Or list only the fields a role may see ("grant": ["name", "email", "address"]) with no except. Remember that multi-fields are separate fields: deny ssn and ssn.* if you have subfields.

Document-Level Security

Filter documents based on user context with a templated role query:

{
  "indices": [
    {
      "names": ["sales_data"],
      "privileges": ["read"],
      "query": {
        "template": {
          "source": {
            "term": { "department": "{{_user.metadata.department}}" }
          }
        }
      }
    }
  ]
}

How do you enable audit logging?

Enable Audit Logging

Track security events. Audit events are written to a dedicated <clustername>_audit.json file on each node (the old index output and the xpack.security.audit.outputs setting were removed in 7.0; ship the file with a log shipper if you want it searchable):

xpack.security.audit.enabled: true
xpack.security.audit.logfile.events.include: [
  "access_denied", "access_granted", "anonymous_access_denied",
  "authentication_failed", "connection_denied", "tampered_request",
  "run_as_denied", "run_as_granted", "security_config_change"
]

That list is the default, so you only need events.include to narrow or widen it. Avoid _all: it gets very verbose. In recent releases xpack.security.audit.enabled is a dynamic setting; in 9.4 and earlier it was static and needed a restart.

Monitor Audit Events

Set up monitoring for:

Failed authentication attempts
Privilege escalation attempts
Unusual access patterns
Configuration changes

How should applications authenticate?

Create API Keys

Use API keys for application access:

curl -X POST "localhost:9200/_security/api_key" -H 'Content-Type: application/json' -d'
{
  "name": "my-api-key",
  "expiration": "1d",
  "role_descriptors": {
    "role-a": {
      "cluster": ["monitor"],
      "index": [
        {
          "names": ["index-a*"],
          "privileges": ["read"]
        }
      ]
    }
  }
}'

API Key Best Practices

Use short expiration times
Implement key rotation
Monitor API key usage
Revoke unused keys promptly

What should you monitor?

Key Metrics to Monitor

Authentication success/failure rates
Privilege escalation attempts
Unusual query patterns
Network connection anomalies
Resource usage spikes

Alerting Rules

Set up alerts for:

Multiple failed login attempts
Access to sensitive indices
Configuration changes
Unusual data access patterns

What about compliance?

GDPR Compliance

Implement data anonymization
Enable data deletion capabilities
Maintain audit trails
Ensure data portability

SOC 2 Compliance

Access controls and monitoring
Data encryption at rest and in transit
Incident response procedures
Regular security assessments

Security Hardening Checklist

Infrastructure Level

Application Level

Operational Level

Conclusion

Implementing comprehensive security measures is essential for protecting your Elasticsearch deployment. Regular security assessments and staying updated with the latest security features will help maintain a robust security posture.

Remember that security is an ongoing process, not a one-time setup. Continuously monitor, assess, and improve your security measures to stay ahead of evolving threats.

Sources

Elastic documentation, Security settings
Elastic documentation, Automatic security setup
Elastic documentation, IP filtering
Elastic documentation, Controlling access at the document and field level
Elastic documentation, Auditing settings
Elasticsearch 7.0 release notes, Remove index audit output type

Elasticsearch Security Best Practices for Enterprise