ELK Stack Configuration and Example Code

The Statistical Dashboard has some dependencies on the Apache Proxy access logs, with the possibilities of using Tomcat access logs as well as the Apache Proxy error logs.   These logs require parsing and shipping to an Elasticsearch intense for parsing into indexed documents which can be served up, in turn, to an application framework such as Spring Boot, which can support a UI for dashboard display.  We are providing configuration files and sample code as prototyping support should we use this approach for a dashboard back end.

ELK Stack

We built our prototypes with a minimal approach using the following applications as the tech stack.  We may add Logstash if we see a need for its filtering capabilities

• Filebeats
• Elasticsearch
• Kabana

Elasticsearch Configuration - default

Kabana Configuration - default

Filebeat Configuration

filebeat.yml

Apache Module Configuration

# Module: apache
# Docs: https://www.elastic.co/guide/en/beats/filebeat/7.9/filebeat-module-apache.html

- module: apache
  # Access logs
  access:
    enabled: true
    var.paths: ["/Users/bauerhs/Desktop/access.target.log"]
    ##The type:access_log will help us point these logs to the right direction
    #input:
     # processors:
       # - add_fields:
           # target: fields
            #fields:
             # codec: plain
              #type: access_log
~