log-metrics command

[jw4n6]

Currently I use awk to print only the log source, first and last timestamps:

./hayabusa-2.18.0-lin-x64-gnu/hayabusa-2.18.0-lin-x64-gnu eid-metrics -f dc01-winevt/logs/Security.evtx -U | awk '/Evtx File Path:/ || /First Timestamp:/ || /Last Timestamp:/ { print $0 }'

Evtx File Path: dc01-winevt/logs/Security.evtx
First Timestamp: 2022-08-31 17:36:13.532 +00:00
Last Timestamp: 2023-01-25 17:19:22.776 +00:00

Would it be possible to implement an option or filter that could print something similar to the below example as a table instead of using awk?

Evtx File Path, First Timestamp, Last Timestamp

dc01-winevt/logs/Security.evtx, 2022-08-31 17:36:13.532 +00:00, 2023-01-25 17:19:22.776 +00:00

[YamatoSecurity]

@jw4n6 Thanks for the suggestion.
This would probably be done best with a new command log-metrics that displays
Filename | Computers | Event Count | First Timestamp | Last Timestamp | Channels | Providers in a table in the terminal or CSV output.
In the case of multiple channels and/or providers we separate by broken pipe character.

@fukusuket Would you be interested in implementing this? (Sorry, I know you've got a lot of issues assigned at the moment)

[fukusuket]

@YamatoSecurity
Yes! I would love to implement it!💪

[fukusuket]

@YamatoSecurity
I have several questions!

Question1 option

The following are the same options as the eid-metrics command. Are there any options you would like to add/remove?
(From an implementation perspective, having the same options allows for logic reuse 🥷)

 ./hayabusa log-metrics -h
Hayabusa v2.19.0 - Dev Build
Yamato Security (https://github.com/Yamato-Security/hayabusa - @SecurityYamato)

Usage:
  hayabusa.exe log-metrics <INPUT> [OPTIONS]

Input:
  -d, --directory <DIR>  Directory of multiple .evtx files
  -f, --file <FILE>      File path to one .evtx file
  -l, --live-analysis    Analyze the local C:\Windows\System32\winevt\Logs folder

General Options:
  -C, --clobber                        Overwrite files when saving
  -h, --help                           Show the help menu
  -J, --JSON-input                     Scan JSON formatted logs instead of .evtx (.json or .jsonl)
  -Q, --quiet-errors                   Quiet errors mode: do not save error logs
  -x, --recover-records                Carve evtx records from slack space (default: disabled)
  -c, --rules-config <DIR>             Specify custom rule config directory (default: ./rules/config)
  -t, --threads <NUMBER>               Number of threads (default: optimal number for performance)
      --target-file-ext <FILE-EXT...>  Specify additional evtx file extensions (ex: evtx_data)

Filtering:
      --exclude-computer <COMPUTER...>  Do not scan specified computer names (ex: ComputerA) (ex: ComputerA,ComputerB)
      --include-computer <COMPUTER...>  Scan only specified computer names (ex: ComputerA) (ex: ComputerA,ComputerB)
      --timeline-offset <OFFSET>        Scan recent events based on an offset (ex: 1y, 3M, 30d, 24h, 30m)

Output:
  -o, --output <FILE>  Save the Metrics in CSV format (ex: metrics.csv)

Display Settings:
      --no-color  Disable color output
  -q, --quiet     Quiet mode: do not display the launch banner
  -v, --verbose   Output verbose information

Time Format:
      --European-time     Output timestamp in European time format (ex: 22-02-2022 22:00:00.123 +02:00)
  -O, --ISO-8601          Output timestamp in original ISO-8601 format (ex: 2022-02-22T10:10:10.1234567Z) (Always UTC)
      --RFC-2822          Output timestamp in RFC 2822 format (ex: Fri, 22 Feb 2022 22:00:00 -0600)
      --RFC-3339          Output timestamp in RFC 3339 format (ex: 2022-02-22 22:00:00.123456-06:00)
      --US-military-time  Output timestamp in US military time format (ex: 02-22-2022 22:00:00.123 -06:00)
      --US-time           Output timestamp in US time format (ex: 02-22-2022 10:00:00.123 PM -06:00)
  -U, --UTC               Output time in UTC format (default: local time)

[fukusuket]

Question2 Terminal output layout

Aligning with other metrics commands will result in a table layout. Would it be better to output in a pipe layout?🤔

Pipe layout(and Broken pipe)

Filename|Computers|Event Count|First Timestamp|Last Timestamp|Channels|Providers
Security.evtx|WIN-SERVER01¦ WIN-DC01|12345|2024-11-01 12:34:56|2024-11-10 08:45:23|Security|Microsoft-Windows-Security-Auditing
System.evtx|WIN-SERVER02¦ WIN-WEB01|23456|2024-10-29 11:22:33|2024-11-10 09:01:45|System|Microsoft-Windows-Kernel-Power¦ Microsoft-Windows-Winlogon
Application.evtx|WIN-SERVER03|9876|2024-10-15 08:12:09|2024-11-09 18:59:00|Application|Microsoft-Windows-Application-Experience
ForwardedEvents.evtx|WIN-SERVER01¦ WIN-WEB01|45678|2024-11-01 07:23:11|2024-11-10 07:45:10|ForwardedEvents|Sysmon¦ Microsoft-Windows-PowerShell
CustomLog.evtx|WIN-CUSTOM01|1234|2024-11-03 14:23:56|2024-11-10 06:12:34|CustomChannel|CustomProvider

Table layout

+------------------------+--------------------------+-------------+---------------------+---------------------+----------------------+------------------------+
| Filename               | Computers                | Event Count | First Timestamp     | Last Timestamp      | Channels             | Providers              |
+------------------------+--------------------------+-------------+---------------------+---------------------+----------------------+------------------------+
| Security.evtx          | WIN-SERVER01, WIN-DC01   | 12,345      | 2024-11-01 12:34:56 | 2024-11-10 08:45:23 | Security             | Microsoft-Windows-Security-Auditing |
| System.evtx            | WIN-SERVER02, WIN-WEB01  | 23,456      | 2024-10-29 11:22:33 | 2024-11-10 09:01:45 | System               | Microsoft-Windows-Kernel-Power, Microsoft-Windows-Winlogon |
| Application.evtx       | WIN-SERVER03             | 9,876       | 2024-10-15 08:12:09 | 2024-11-09 18:59:00 | Application          | Microsoft-Windows-Application-Experience |
| ForwardedEvents.evtx   | WIN-SERVER01, WIN-WEB01  | 45,678      | 2024-11-01 07:23:11 | 2024-11-10 07:45:10 | ForwardedEvents      | Sysmon, Microsoft-Windows-PowerShell |
| CustomLog.evtx         | WIN-CUSTOM01             | 1,234       | 2024-11-03 14:23:56 | 2024-11-10 06:12:34 | CustomChannel        | CustomProvider        |
+------------------------+--------------------------+-------------+---------------------+---------------------+----------------------+------------------------+

[fukusuket]

Question3 Abbreviation

Should I output the following fields in abbreviated form?

• Channel
• Provider

That's all the questions I have at the moment! I would appreciate it if you could check them when you have time.

[YamatoSecurity]

@fukusuket All very good questions!

1. I would like to keep the same options that we have for eid-metrics. I believe that all the options are still applicable to log-metrics but let me know if you think there are any options we do not need.

2. I think the table view is much easier to read than separating by | characters. We may need to do line wrapping for some columns to be prevent the table from breaking if it gets too big though.

3. I would like to do channel and provider abbreviation by default but add a -a --disable-abbreviations Disable abbreviations option under General Options to preserve the names as-is. I'd like to add this opiton to eid-metrics and csv-timeline and json-timeline as well as sometimes it is necessary to look up the original Channel and Provider names. We could create a separate issue for this though.

[fukusuket]

@YamatoSecurity
Thank you for comment! I'll implement above specification!

2. ... We may need to do line wrapping for some columns to be prevent the table from breaking if it gets too big though.

I will check to see if there is a good crate!

3. ... I would like to do channel and provider abbreviation by default but add a -a --disable-abbreviations Disable

Sounds good! It seems better to address this as a separate issue!