Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

log-metrics command #1474

Closed
jw4n6 opened this issue Nov 6, 2024 · 7 comments · Fixed by #1484
Closed

log-metrics command #1474

jw4n6 opened this issue Nov 6, 2024 · 7 comments · Fixed by #1484
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@jw4n6
Copy link

jw4n6 commented Nov 6, 2024

Currently I use awk to print only the log source, first and last timestamps:

./hayabusa-2.18.0-lin-x64-gnu/hayabusa-2.18.0-lin-x64-gnu eid-metrics -f dc01-winevt/logs/Security.evtx -U | awk '/Evtx File Path:/ || /First Timestamp:/ || /Last Timestamp:/ { print $0 }'

Evtx File Path: dc01-winevt/logs/Security.evtx
First Timestamp: 2022-08-31 17:36:13.532 +00:00
Last Timestamp: 2023-01-25 17:19:22.776 +00:00

Would it be possible to implement an option or filter that could print something similar to the below example as a table instead of using awk?

Evtx File Path, First Timestamp, Last Timestamp

dc01-winevt/logs/Security.evtx, 2022-08-31 17:36:13.532 +00:00, 2023-01-25 17:19:22.776 +00:00

@jw4n6 jw4n6 changed the title [enhancement] eid-metrics additional options/filtering for only viewing log source and timestamps eid-metrics additional options/filtering for only viewing log source and timestamps Nov 6, 2024
@jw4n6 jw4n6 changed the title eid-metrics additional options/filtering for only viewing log source and timestamps eid-metrics additional options/filtering for only viewing log source and timestamps [enhancement] Nov 6, 2024
@jw4n6 jw4n6 changed the title eid-metrics additional options/filtering for only viewing log source and timestamps [enhancement] eid-metrics additional options/filtering for only viewing log source and timestamps Nov 6, 2024
@YamatoSecurity YamatoSecurity changed the title eid-metrics additional options/filtering for only viewing log source and timestamps log-metrics command Nov 6, 2024
@YamatoSecurity
Copy link
Collaborator

YamatoSecurity commented Nov 7, 2024

@jw4n6 Thanks for the suggestion.
This would probably be done best with a new command log-metrics that displays
Filename | Computers | Event Count | First Timestamp | Last Timestamp | Channels | Providers in a table in the terminal or CSV output.
In the case of multiple channels and/or providers we separate by broken pipe character.

@fukusuket Would you be interested in implementing this? (Sorry, I know you've got a lot of issues assigned at the moment)

@YamatoSecurity YamatoSecurity added the enhancement New feature or request label Nov 7, 2024
@fukusuket
Copy link
Collaborator

@YamatoSecurity
Yes! I would love to implement it!💪

@fukusuket fukusuket added this to the 2.19.0 milestone Nov 8, 2024
@fukusuket
Copy link
Collaborator

@YamatoSecurity
I have several questions!

Question1 option

The following are the same options as the eid-metrics command. Are there any options you would like to add/remove?
(From an implementation perspective, having the same options allows for logic reuse 🥷)

 ./hayabusa log-metrics -h
Hayabusa v2.19.0 - Dev Build
Yamato Security (https://github.com/Yamato-Security/hayabusa - @SecurityYamato)

Usage:
  hayabusa.exe log-metrics <INPUT> [OPTIONS]

Input:
  -d, --directory <DIR>  Directory of multiple .evtx files
  -f, --file <FILE>      File path to one .evtx file
  -l, --live-analysis    Analyze the local C:\Windows\System32\winevt\Logs folder

General Options:
  -C, --clobber                        Overwrite files when saving
  -h, --help                           Show the help menu
  -J, --JSON-input                     Scan JSON formatted logs instead of .evtx (.json or .jsonl)
  -Q, --quiet-errors                   Quiet errors mode: do not save error logs
  -x, --recover-records                Carve evtx records from slack space (default: disabled)
  -c, --rules-config <DIR>             Specify custom rule config directory (default: ./rules/config)
  -t, --threads <NUMBER>               Number of threads (default: optimal number for performance)
      --target-file-ext <FILE-EXT...>  Specify additional evtx file extensions (ex: evtx_data)

Filtering:
      --exclude-computer <COMPUTER...>  Do not scan specified computer names (ex: ComputerA) (ex: ComputerA,ComputerB)
      --include-computer <COMPUTER...>  Scan only specified computer names (ex: ComputerA) (ex: ComputerA,ComputerB)
      --timeline-offset <OFFSET>        Scan recent events based on an offset (ex: 1y, 3M, 30d, 24h, 30m)

Output:
  -o, --output <FILE>  Save the Metrics in CSV format (ex: metrics.csv)

Display Settings:
      --no-color  Disable color output
  -q, --quiet     Quiet mode: do not display the launch banner
  -v, --verbose   Output verbose information

Time Format:
      --European-time     Output timestamp in European time format (ex: 22-02-2022 22:00:00.123 +02:00)
  -O, --ISO-8601          Output timestamp in original ISO-8601 format (ex: 2022-02-22T10:10:10.1234567Z) (Always UTC)
      --RFC-2822          Output timestamp in RFC 2822 format (ex: Fri, 22 Feb 2022 22:00:00 -0600)
      --RFC-3339          Output timestamp in RFC 3339 format (ex: 2022-02-22 22:00:00.123456-06:00)
      --US-military-time  Output timestamp in US military time format (ex: 02-22-2022 22:00:00.123 -06:00)
      --US-time           Output timestamp in US time format (ex: 02-22-2022 10:00:00.123 PM -06:00)
  -U, --UTC               Output time in UTC format (default: local time)

@fukusuket
Copy link
Collaborator

fukusuket commented Nov 10, 2024

Question2 Terminal output layout

Aligning with other metrics commands will result in a table layout. Would it be better to output in a pipe layout?🤔

Pipe layout(and Broken pipe)

Filename|Computers|Event Count|First Timestamp|Last Timestamp|Channels|Providers
Security.evtx|WIN-SERVER01¦ WIN-DC01|12345|2024-11-01 12:34:56|2024-11-10 08:45:23|Security|Microsoft-Windows-Security-Auditing
System.evtx|WIN-SERVER02¦ WIN-WEB01|23456|2024-10-29 11:22:33|2024-11-10 09:01:45|System|Microsoft-Windows-Kernel-Power¦ Microsoft-Windows-Winlogon
Application.evtx|WIN-SERVER03|9876|2024-10-15 08:12:09|2024-11-09 18:59:00|Application|Microsoft-Windows-Application-Experience
ForwardedEvents.evtx|WIN-SERVER01¦ WIN-WEB01|45678|2024-11-01 07:23:11|2024-11-10 07:45:10|ForwardedEvents|Sysmon¦ Microsoft-Windows-PowerShell
CustomLog.evtx|WIN-CUSTOM01|1234|2024-11-03 14:23:56|2024-11-10 06:12:34|CustomChannel|CustomProvider

Table layout

+------------------------+--------------------------+-------------+---------------------+---------------------+----------------------+------------------------+
| Filename               | Computers                | Event Count | First Timestamp     | Last Timestamp      | Channels             | Providers              |
+------------------------+--------------------------+-------------+---------------------+---------------------+----------------------+------------------------+
| Security.evtx          | WIN-SERVER01, WIN-DC01   | 12,345      | 2024-11-01 12:34:56 | 2024-11-10 08:45:23 | Security             | Microsoft-Windows-Security-Auditing |
| System.evtx            | WIN-SERVER02, WIN-WEB01  | 23,456      | 2024-10-29 11:22:33 | 2024-11-10 09:01:45 | System               | Microsoft-Windows-Kernel-Power, Microsoft-Windows-Winlogon |
| Application.evtx       | WIN-SERVER03             | 9,876       | 2024-10-15 08:12:09 | 2024-11-09 18:59:00 | Application          | Microsoft-Windows-Application-Experience |
| ForwardedEvents.evtx   | WIN-SERVER01, WIN-WEB01  | 45,678      | 2024-11-01 07:23:11 | 2024-11-10 07:45:10 | ForwardedEvents      | Sysmon, Microsoft-Windows-PowerShell |
| CustomLog.evtx         | WIN-CUSTOM01             | 1,234       | 2024-11-03 14:23:56 | 2024-11-10 06:12:34 | CustomChannel        | CustomProvider        |
+------------------------+--------------------------+-------------+---------------------+---------------------+----------------------+------------------------+

@fukusuket
Copy link
Collaborator

fukusuket commented Nov 10, 2024

Question3 Abbreviation

Should I output the following fields in abbreviated form?

  • Channel
  • Provider

That's all the questions I have at the moment! I would appreciate it if you could check them when you have time.

@YamatoSecurity
Copy link
Collaborator

@fukusuket All very good questions!

  1. I would like to keep the same options that we have for eid-metrics. I believe that all the options are still applicable to log-metrics but let me know if you think there are any options we do not need.

  2. I think the table view is much easier to read than separating by | characters. We may need to do line wrapping for some columns to be prevent the table from breaking if it gets too big though.

  3. I would like to do channel and provider abbreviation by default but add a -a --disable-abbreviations Disable abbreviations option under General Options to preserve the names as-is. I'd like to add this opiton to eid-metrics and csv-timeline and json-timeline as well as sometimes it is necessary to look up the original Channel and Provider names. We could create a separate issue for this though.

@fukusuket
Copy link
Collaborator

fukusuket commented Nov 10, 2024

@YamatoSecurity
Thank you for comment! I'll implement above specification!

  1. ... We may need to do line wrapping for some columns to be prevent the table from breaking if it gets too big though.

I will check to see if there is a good crate!

  1. ... I would like to do channel and provider abbreviation by default but add a -a --disable-abbreviations Disable

Sounds good! It seems better to address this as a separate issue!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants