feat: add support for Sigma Correlation Event count

[fukusuket]

What Changed

• Closed Sigma correlations support: Event Count #1337

Limitations of current implementation

The following points are not supported because they require more time to implement.
I will consider addressing them as separate PR depending on priority.

• Improving count rule's output #1342
• Multiple yaml in one file
• Multiple Condition(e.g. 100 > count > 1)
• Recursive Correlation rule

I would appreciate it if you could check it out when you have time🙏

[fukusuket]

Correlation(Valid rule)

I compared following rule,

Correlation rule

title: Correlation TEST
id: 0e95725d-7320-415d-80f7-004da920fc11
level: informational
correlation:
  type: event_count
  rules:
    - e87bd730-df45-4ae9-85de-6c75369c5d29 # Logon Failure (Wrong Password)
    - 8afa97ce-a217-4f7c-aced-3e320a57756d # Logon Failure (User Does Not Exist)
  group-by:
    - Computer
    - SubStatus
  timespan: 5m
  condition:
    gte: 3

ant put following 2 rules in same directory.

• https://github.com/Yamato-Security/hayabusa-rules/blob/4cc21ba67015c98e0a2340cd28c6e441a64759f6/hayabusa/builtin/Security/LogonLogoff/Logon/Sec_4625_Low_LogonFail_WrongPW.yml#L8
• https://github.com/Yamato-Security/hayabusa-rules/blob/4cc21ba67015c98e0a2340cd28c6e441a64759f6/hayabusa/builtin/Security/LogonLogoff/Logon/Sec_4625_Low_LogonFail_NonExistantUser.yml#L8

Agg rule

author: TEST
date: 2024/06/03
modified: 2024/06/03

title: Aggregation TEST
description: Aggregation TEST

id: e87bd730-df45-4ae9-85de-6c75369c5d20
level: informational
status: stable
logsource:
  product: windows
  service: security
detection:
  selection_basic:
    Channel: Security
    EventID: 4625
  selection_error:
    SubStatus:
      - "0xc000006a" #Wrong password
      - "0xc0000064" #Username does not exist error
  condition: selection_basic and selection_error | count() by Computer,SubStatus >= 3
  timeframe: 5m
falsepositives:
  - user mistypes password
tags:
references:
ruletype: Hayabusa

Result

then I confirmed that the result is same.

% ./hayabusa csv-timeline -d ../hayabusa-sample-evtx -o timeline.csv -r test -w -C
% cat timeline.csv | grep -e "Correlation" -e "Aggregation"
"2016-09-20 01:50:06.513 +09:00","Aggregation TEST","info","-","-","-","-","Count:3558 ¦ Computer:DESKTOP-M5SN04R ¦ SubStatus:0xc000006a","-"
"2016-09-20 01:50:06.513 +09:00","Correlation TEST","info","-","-","-","-","Count:3558 ¦ Computer:DESKTOP-M5SN04R ¦ SubStatus:0xc000006a","-"
"2021-05-20 21:49:52.315 +09:00","Aggregation TEST","info","-","-","-","-","Count:5 ¦ Computer:fs01.offsec.lan ¦ SubStatus:0xc0000064","-"
"2021-05-20 21:49:52.315 +09:00","Correlation TEST","info","-","-","-","-","Count:5 ¦ Computer:fs01.offsec.lan ¦ SubStatus:0xc0000064","-"
"2021-05-22 05:43:22.562 +09:00","Aggregation TEST","info","-","-","-","-","Count:5 ¦ Computer:fs01.offsec.lan ¦ SubStatus:0xc000006a","-"
"2021-05-22 05:43:22.562 +09:00","Correlation TEST","info","-","-","-","-","Count:5 ¦ Computer:fs01.offsec.lan ¦ SubStatus:0xc000006a","-"

[fukusuket]

Correlation(Invalid rule)

missing condition block rule.

title: Many failed logins to the same computer
id: 0e95725d-7320-415d-80f7-004da920fc11
level: informational
correlation:
  type: event_count
  rules:
    - e87bd730-df45-4ae9-85de-6c75369c5d29 # Logon Failure (Wrong Password)
    - 8afa97ce-a217-4f7c-aced-3e320a57756d # Logon Failure (User Does Not Exist)
  group-by:
    - Computer
    - SubStatus
  timespan: 5m

parse error msg appeared in console.

% ./hayabusa csv-timeline -d ../hayabusa-sample-evtx -o timeline.csv -r test -w -C -q
Start time: 2024/06/03 11:03

Total event log files: 585
Total file size: 137.2 MB

Loading detection rules. Please wait.

Rule parsing errors: 1
...

and output error log.

% cat ./logs/errorlog-20240603_110350.log for details.
user input: ./hayabusa csv-timeline -d ../hayabusa-sample-evtx -o timeline.csv -r test -w -C -q
[WARN] Failed to parse rule. (FilePath : test/test.yml) Failed to parse condition

[fukusuket]

hayabusa-sample-evtx

I confirmed that there are no differences in the result files as follows

csv-timeline

% ./hayabusa-old csv-timeline -d ../hayabusa-sample-evtx -w -D -u -n -q -C -o old.csv
% ./hayabusa-new csv-timeline -d ../hayabusa-sample-evtx -w -D -u -n -q -C -o new.csv
% diff new.csv old.csv
%

json-timeline

% ./hayabusa-old json-timeline -d ../hayabusa-sample-evtx -w -D -u -n -q -C -o old.json
% ./hayabusa-new json-timeline -d ../hayabusa-sample-evtx -w -D -u -n -q -C -o new.json
% diff new.json old.json
%

Integration-Test

https://github.com/Yamato-Security/hayabusa/actions/runs/9343097325

[hitenkoku]

LGTM

[YamatoSecurity]

@fukusuket LGTM! Thanks so much!