Introduction

Logs contain evidence of attacks, errors, and suspicious behaviour. Centralised logging makes analysis easier. This guide sets up ELK Stack for security monitoring.

1. Install Elasticsearch

Add Elastic GPG key and repository, then sudo apt install elasticsearch. Start and enable.

2. Install Logstash

sudo apt install logstash. Create configuration files (input from syslog, filebeats).

3. Install Kibana

sudo apt install kibana. Access on port 5601, secure with nginx proxy and basic auth.

4. Install Filebeat on Production VPS

Filebeat ships logs to Logstash/Elasticsearch. Configure it to send /var/log/auth.log, /var/log/syslog, etc.

5. Create Logstash Pipeline for Security Logs

Filter SSH, fail2ban, and web logs. Use grok patterns to parse timestamps and IPs.

6. Build Kibana Dashboards

Create visualisations for failed logins, banned IPs, unusual activity times.

7. Alerting with ElastAlert or Kibana Alerts

Set up alerts for repeated failed logins or new user creation.

8. Centralised Logging for Multiple VPS

Use a single ELK server (can be small VPS) and ship logs from all your VPS.

9. Log Retention and Rotation

Set Elasticsearch ILM policies to delete old logs after 90 days.

10. Secure the ELK Stack

Enable Elasticsearch security (built‑in), use TLS between components, firewall Kibana.

11. Monitoring System Logs

Look for kernel errors, out‑of‑memory events, disk errors – signs of instability or attack.

12. Web Server Log Analysis

Detect 404 scans, SQL injection attempts, suspicious user agents.

13. SIEM Alternative: Wazuh

Wazuh provides a ready‑to‑use open source SIEM with built‑in rules and dashboards.

14. Logstash Performance

Use multiple pipelines, filter only necessary logs to reduce load.

15. Off‑site Log Backup

Ship logs to a different data centre for disaster recovery.

16. Regulatory Compliance (PCI, HIPAA)

Centralised logs help meet audit requirements (log retention, review).

17. User Activity Monitoring

Auditd + ELK can track command history and file access.

18. Incident Investigation Workflow

Use Kibana timeline and filters to quickly find relevant entries during an incident.

19. Elastic Stack Security Analytics

Explore the SIEM app (Elastic Security) for preconfigured detection rules.

20. Resource Requirements

ELK stack consumes RAM (4GB+); run on a separate VPS if resources are tight.

Conclusion

Centralised logging transforms raw logs into actionable intelligence. Start with Filebeat + Elasticsearch + Kibana, then add alerting.