The Statistical Dashboard has some dependencies on the Apache Proxy access logs, with the possibilities of using Tomcat access logs as well as the Apache Proxy error logs. These logs require parsing and shipping to an Elasticsearch intense for parsing into indexed documents which can be served up, in turn, to an application framework such as Spring Boot, which can support a UI for dashboard display. We are providing configuration files and sample code as prototyping support should we use this approach for a dashboard back end.
ELK Stack
We built our prototypes with a minimal approach using the following applications as the tech stack. We may add Logstash if we see a need for its filtering capabilities
- Filebeats
- Elasticsearch
- Kabana
Elasticsearch Configuration - default
Kabana Configuration - default
Filebeat Configuration
Last login: Wed May 19 08:42:37 on console
-------------------------------------------------------------------------------
***WARNING***
-------------------------------------------------------------------------------
YOU ARE ACCESSING A U.S. GOVERNMENT (USG) INFORMATION SYSTEM (IS) THAT IS PROVIDED FOR USG AUTHORIZED USE ONLY. BY USING THIS IS, YOU CONSENT TO THE FOLLOWING CONDITIONS:
- THE USG ROUTINELY MONITORS COMMUNICATIONS OCCURRING ON THIS IS, AND ANY DEVICE ATTACHED TO THIS IS, FOR PURPOSES INCLUDING, BUT NOT LIMITED TO, PENETRATION TESTING, COMSEC MONITORING, NETWORK DEFENSE, QUALITY CONTROL, EMPLOYEE MISCONDUCT, LAW ENFORCEMENT, AND COUNTERINTELLIGENCE INVESTIGATIONS.
- AT ANY TIME, THE USG MAY INSPECT AND/OR SEIZE DATA STORED ON THIS IS AND ANY DEVICE ATTACHED TO THIS IS.
- COMMUNICATIONS OCCURRING ON OR DATA STORED ON THIS IS, OR ANY DEVICE ATTACHED TO THIS IS, ARE NOT PRIVATE. THEY ARE SUBJECT TO ROUTINE MONITORING AND SEARCH.
- ANY COMMUNICATIONS OCCURRING ON OR DATA STORED ON THIS IS, OR ANY DEVICE ATTACHED TO THIS IS, MAY BE DISCLOSED OR USED FOR ANY USG-AUTHORIZED PURPOSE.
- SECURITY PROTECTIONS MAY BE UTILIZED ON THIS IS TO PROTECT CERTAIN INTERESTS THAT ARE IMPORTANT TO THE USG. FOR EXAMPLE, PASSWORDS, ACCESS CARDS, ENCRYPTION OR BIOMETRIC ACCESS CONTROLS PROVIDE SECURITY FOR THE BENEFIT OF THE USG. THESE PROTECTIONS ARE NOT PROVIDED FOR YOUR BENEFIT OR PRIVACY AND MAY BE MODIFIED OR ELIMINATED AT THE USG DISCRETION.
-------------------------------------------------------------------------------
NCI-02196725-ML:~ bauerhs$ cd /usr/local/etc/filebeat/
NCI-02196725-ML:filebeat bauerhs$ vi filebeat.yml
# Set to true to enable instrumentation of filebeat.
#enabled: false
# Environment in which filebeat is running on (eg: staging, production, etc.)
#environment: ""
# APM Server hosts to report instrumentation results to.
#hosts:
# - http://localhost:8200
# API Key for the APM Server(s).
# If api_key is set then secret_token will be ignored.
#api_key:
# Secret token for the APM Server(s).
#secret_token:
# ================================= Migration ==================================
# This allows to enable 6.7 migration aliases
#migration.6_to_7.enabled: true
