DEV Community

Turan Kılıç for Açıklab

Posted on • Updated on

Elasticsearch - Winlogbeat Creating Query for an Index to Get a Specific String

Hello there, on this post I will share requesting method to get specific string from an API in Elastic - Winlogbeat.

I've searched to find an example of the query on internet but didn't find anything. Then I created a query by myself and I wanted to share it with you. So let's begin :)

System's Set Up and Configurations

We are using 2 type of beats which are Filebeat (for Linux clients) and Winlogbeat (for Windows clients).

Filebeat collects the logs of Linux client and sends them to elastic search on port 5044. Winlogbeat also collects the logs of Windows client and sends to elastic search on port 5043. The schema of system is:

System Schema of Beat-Logstash

Beat Configurations

Beat config files are the files that taking place on client machines to send the collected data to elastic server.

Filebeat Configuration File:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/*.log
    - /var/log/*/*.log

output.logstash:
  hosts: ["x.x.x.x:5044"]
Enter fullscreen mode Exit fullscreen mode

Winlogbeat Configuration File:

winlogbeat.event_logs:
  - name: Application
    fields:
      log_type: application

  - name: System
    fields:
      log_type: system

  - name: Security
    fields:
      log_type: security

output.logstash:
  hosts: ["x.x.x.x:5043"]
Enter fullscreen mode Exit fullscreen mode

Configuration Files on Elastic Server

These beats are welcomed by some config files on elastic server. The config files must be under /etc/logstash/conf.d/ directory.

filebeat.conf:

# INPUT HERE
input {
   beats {
      port => 5044
      client_inactivity_timeout => 1200
    }
  }

# FILTER HERE
filter{
}

#OUTPUT HERE
output {
      elasticsearch {
        index => "linux"
      }
}
Enter fullscreen mode Exit fullscreen mode

winlogbeat.conf:

# INPUT HERE
input {
   beats {
      port => 5043
      client_inactivity_timeout => 1200
    }
  }

# FILTER HERE
filter{
}

#OUTPUT HERE
output {
      elasticsearch {
        index => "win10"
      }
}

Enter fullscreen mode Exit fullscreen mode

With the help of this config files and adjustment, we are able to index Linux logs as "linux", Windows logs as "win10" on elastic server.

REST APIs of Elasticsearch

If we want to create a request to an index, we have to know which index we are messing. As you know Elasticsearch has some REST APIs. You can list your indexes with _cat API like this:

GET http://your_host_ip:9200/_cat/indices
Enter fullscreen mode Exit fullscreen mode

Response of this request is:

yellow open win10                             004M_Z8HRhOOF9LLgP4Vkw 1 1     245     0 665.9kb 665.9kb
yellow open filebeat-7.17.1-2022.03.02        vWCH-_71QAubBvHXVtgz2g 1 1     392     0 158.8kb 158.8kb
green  open .apm-agent-configuration          QTZfTFNZSXOGcH0P2588YQ 1 0       0     0    226b    226b
yellow open filebeat-7.17.1-2022.03.01        MCb65OGTTECbn9xw1tJ87w 1 1    7271     0   1.1mb   1.1mb
Enter fullscreen mode Exit fullscreen mode

As you can see above, I have 4 indices (to make it short) and win10 index has been created with Winlogbeat. That's why I named it as "win10".

What's Inside of an Index

If we wanted to look inside of an index to see what he has, the search API comes to help. We can see the beats with this request schema:

GET http://host_ip:9200/your_index/_search 
Enter fullscreen mode Exit fullscreen mode

In my case, the request is:

GET http://x.x.x.x:9200/win10/_search
Enter fullscreen mode Exit fullscreen mode

Response of this request should looks like this:

"took": 721,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 299,
            "relation": "eq"
        },
        "max_score": 1.0,
        "hits": [
            {
                "_index": "win10",
        ...
Enter fullscreen mode Exit fullscreen mode

There are lots of things in the response, How are we going to choose what actually we want?
Well you can do it with "q=" parameter in request. In my case I want to get response of beats that its winlog.event_id is equals to "4624" (which refers to "An account was successfully logged on")
So I need to use this request:

GET http://x.x.x.x:9200/win10/_search?q=query.bool.must.filter.bool.should.match.winlog.event_id="4624"
Enter fullscreen mode Exit fullscreen mode

Because the winlog.event_id is under lots of things. Here:

"query": {
    "bool": {
      "must": [],
      "filter": [
        {
          "bool": {
            "should": [
              {
                "match": {
                  "winlog.event_id": "4624"
Enter fullscreen mode Exit fullscreen mode

If I want to reach winlog.event, then I need to write whole path seperated by dots. You need to adjust it for your needings.

That's all,
Hope it would be helpful for you :)

Top comments (0)