Configuring ELK components in Unica Discover 12.1.2

When we are asked what is Elasticsearch used for ? Some of you may say it is an indexing tool, or an analytics solution, maybe some will answer it is an unstructured database, a Big data software or even some may think it’s like Google search with autosuggest on your complex data. It is actually all of the above and more…

But… What is Elasticsearch?

Throughout the years it has evolved to become a complete solution known as the ELK stack, a simple, fast and scalable ecosystem that had become popular for exploring your complex data. The ELK Stack is a collection of three open-source products — Elasticsearch, Logstash, and Kibana. ELK is one of the few new-age frameworks which is capable of handling Big Data demands and scale. ELK Stack is designed to allow users to take data from any source in any format and search, analyze, and visualize that data in real-time. Logstash works towards parsing and transforming event data which is then passed on to Elasticsearch to be stored, searched and indexed. Kibana then accesses Elasticsearch DB to Explore, Visualize and Share.

ELK has been utilized in the latest Unica Discover 12.1.2 to expose Discover logs, windows logs (Widows Event Viewer) and also to use ElasticSearch to search the log files. There is documentation on ELK Configuration that will help you learn more about the ELK stack in Unica Discover.

In this blog, I’d like to consolidate the information from our documentation and provide clear steps on how to configure ELK in Unica Discover.

Steps for configuring ELK components in Unica Discover 

1. After Unica Discover 12.1.2 upgrade/installation, the “System Logs” will appear on the portal under “Discover.”

2. Steps to configure ELK components on the portal.

NOTE: If you are a new user or an existing user, you must install the new RPM for ELK to install Kibana Dashboard and Elastic search. For more information on installing the RPM, see the topic “ELK Installation” in Unica Discover Installation and Deployment Manual in our documentation.

2.1 Configuring FileBeat Service

Goto Discover->Managed Service and on the right panel in WorldView tab expand FileBeat Service, Highlight FileBeat configuration file and click on “View/Edit (Raw)

In the open window, you will see the Filebeat config file (filebeat.yml). The following changes need to be done:

In “Filebeat inputs” session in 
#Change to true to enable this input configuration.
Change:
 enabled: false
To 
enabled: true

Under “# Paths that should be crawled and fetched. Glob based paths.”

Change the path to the Logs directory 
from:
c:\programdata\elasticsearch\logs\
to the location of the Logs directory in your Discover installation. 
Example: - C:\Discover\Logs\*

 Under # —————————- Elasticsearch Output —————————-

Change the hosts entry from localhost to the IP of the Linux machine where ELK has been installed.
Example: 
hosts: ["192.168.56.217:9200"]


Click “Save and “Add Tasks and Submit” on the appearing pop-up window.

2.2 Configuring WinLogBeat Service

• Goto Discover->Managed Service and on the right panel in WorldView tab,

• Expand WinLogBeat Service, Highlight WinLogBeat configuration file and click on “View/Edit (Raw) (shown in Figure 2)

• In the open window, you will see the WinLogBeat config file (filebeat.yml)

Under # —————————- Elasticsearch Output —————————-

Change the hosts entry from localhost to the IP of the Linux machine where ELK has been installed.
Example:
hosts: ["192.168.56.217:9200"]

Click “Save and “Add Tasks and Submit” on the appearing pop-up window

2.3 Configuring Discover global configuration settings

Goto Discover->Managed Service and on the right panel in WorldView tab expand Discover, Highlight Discover global configuration settings and click on “View/Edit.”

In the open window, the following parameters need to be configured:
Elastic Search SSL (0 or 1)

• Elastic Search Server (IP address of the Linux machine where ELK is installed)
• Elastic Search Server Port (default is 9200)
• Kibana Server Port (default is 5601)

You can look at the example of the configuration below:

Click “Save and “Add Tasks and Submit” on the appearing pop-up window. This will restart Discover Portal services, and you’ll need to wait a minute or so before trying to login into the portal.

3. Starting Discover Filebeat and WinLogBeat Services
Goto Discover->Managed Service and on the right panel in WorldView tab Highlight FileBeat Service and click on “Stop” to stop service or “Restart” to start/restart service.
Repeat above for WinLogBeat Service

4. Checking if your ELK is configured and working properly
Login to Portal and go to Discover->System Logs.
When you click on Search, another tab will open in your browser, showing you the ElasticSearh dashboard where you can perform the search against all logs.

You can also have different statistics when clicking on Discover and Windows Dashboard. To learn more about ELK configurations in Unica Discover, you can reach out to us and we will be happy to help.