Troubleshooting ELKStack

diagnostic steps, in order:

1. Check the service state

bash

sudo systemctl status elasticsearch.service --no-pager

Look for:

  • Active: failed vs Active: inactive (dead)
  • The last few log lines in the status output
  • Whether it is set to start on boot (; enabled; or ; disabled;)

2. Check recent logs from systemd

bash

sudo journalctl -u elasticsearch.service -n 100 --no-pager

Look for Java errors, OOM Killer, failed to start, permission denied, or config file syntax errors.

3. Check Elasticsearch’s own logs

bash

sudo tail -n 50 /var/log/elasticsearch/*.log 2>/dev/null || echo "No log files found"

If the service started partially and crashed, the Java stack trace will usually be here.

4. Check disk space

bash

df -h

Elasticsearch will refuse to start or will shut itself down if disk usage crosses high or flood watermarks (typically 90% and 95%). Pay attention to the root (/) partition.

5. Check for OOM Killer or memory issues

bash

free -h
sudo dmesg | grep -i "kill.*elasticsearch\|out of memory" | tail -20

If the system ran out of RAM, the kernel may have killed the Elasticsearch Java process.

6. Check JVM / config syntax

bash

sudo systemctl cat elasticsearch.service
sudo cat /etc/elasticsearch/jvm.options | grep -v "^#" | grep -v "^$"
sudo cat /etc/elasticsearch/elasticsearch.yml | grep -v "^#" | grep -v "^$"

Common issues:

  • -Xms and -Xmx set higher than available RAM
  • Corrupt YAML in elasticsearch.yml
  • Wrong cluster name or node settings after a restart

7. Try a manual start and watch live

bash

sudo systemctl start elasticsearch.service
sleep 5
sudo systemctl status elasticsearch.service --no-pager
sudo journalctl -u elasticsearch.service --since "1 minute ago" --no-pager

Common root causes

  1. Disk full or over watermark – clean up old indices or expand disk.
  2. OOM / memory pressure – reduce JVM heap in jvm.options or add RAM.
  3. Corrupt or incompatible indices on restart after an unclean shutdown.
  4. Config file edited incorrectly and service fails to parse it on start.

Run steps 1 through 3 first and paste the output here if you want a deeper analysis.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top