feat: add `--disable-abbreviations` option #1497

fukusuket · 2024-11-15T02:04:50Z

What Changed

Closed Add --disable-abbreviations option #1485
- Since the -a option was already in use, I temporarily set it to the -b option.

The command to which this option is added

csv-timeline
json-timeline
eid-metrics
log-metrics
search

Evidence

Integration-Test

https://github.com/Yamato-Security/hayabusa/actions/runs/11848925669

I would appreciate it if you could check it out when you have time🙏

fukusuket · 2024-11-15T02:07:04Z

csv-timeline

help

fukusuke@MacBookAir hayabusa-2.19.0-mac-aarch64 % ./hayabusa csv-timeline -h
Hayabusa v2.19.0 - Dev Build
Yamato Security (https://github.com/Yamato-Security/hayabusa - @SecurityYamato)

Usage:
  hayabusa.exe csv-timeline <INPUT> [OPTIONS]

Input:
  -d, --directory <DIR>  Directory of multiple .evtx files
  -f, --file <FILE>      File path to one .evtx file
  -l, --live-analysis    Analyze the local C:\Windows\System32\winevt\Logs folder

General Options:
  -C, --clobber                        Overwrite files when saving
  -b, --disable-abbreviations          Disable abbreviations
  -h, --help                           Show the help menu
  -J, --JSON-input                     Scan JSON formatted logs instead of .evtx (.json or .jsonl)
  -w, --no-wizard                      Do not ask questions. Scan for all events and alerts
  -Q, --quiet-errors                   Quiet errors mode: do not save error logs
  -x, --recover-records                Carve evtx records from slack space (default: disabled)
  -r, --rules <DIR/FILE>               Specify a custom rule directory or file (default: ./rules)
  -c, --rules-config <DIR>             Specify custom rule config directory (default: ./rules/config)
  -s, --sort-events                    Sort events before saving the file. (warning: this uses much more memory!)
  -t, --threads <NUMBER>               Number of threads (default: optimal number for performance)
      --target-file-ext <FILE-EXT...>  Specify additional evtx file extensions (ex: evtx_data)

Filtering:
  -E, --EID-filter                      Scan only common EIDs for faster speed (./rules/config/target_event_IDs.txt)
  -A, --enable-all-rules                Enable all rules regardless of loaded evtx files (disable channel filter for rules)
  -D, --enable-deprecated-rules         Enable rules with a status of deprecated
  -n, --enable-noisy-rules              Enable rules set to noisy (./rules/config/noisy_rules.txt)
  -u, --enable-unsupported-rules        Enable rules with a status of unsupported
  -e, --exact-level <LEVEL>             Only load rules with a specific level (informational, low, medium, high, critical)
      --exclude-category <CATEGORY...>  Do not load rules with specified logsource categories (ex: process_creation,pipe_created)
      --exclude-computer <COMPUTER...>  Do not scan specified computer names (ex: ComputerA) (ex: ComputerA,ComputerB)
      --exclude-eid <EID...>            Do not scan specific EIDs for faster speed (ex: 1) (ex: 1,4688)
      --exclude-status <STATUS...>      Do not load rules according to status (ex: experimental) (ex: stable,test)
      --exclude-tag <TAG...>            Do not load rules with specific tags (ex: sysmon)
      --include-category <CATEGORY...>  Only load rules with specified logsource categories (ex: process_creation,pipe_created)
      --include-computer <COMPUTER...>  Scan only specified computer names (ex: ComputerA) (ex: ComputerA,ComputerB)
      --include-eid <EID...>            Scan only specified EIDs for faster speed (ex: 1) (ex: 1,4688)
      --include-status <STATUS...>      Only load rules with specific status (ex: experimental) (ex: stable,test)
      --include-tag <TAG...>            Only load rules with specific tags (ex: attack.execution,attack.discovery)
  -m, --min-level <LEVEL>               Minimum level for rules to load (default: informational)
  -P, --proven-rules                    Scan with only proven rules for faster speed (./rules/config/proven_rules.txt)
  -a, --scan-all-evtx-files             Scan all evtx files regardless of loaded rules (disable channel filter for evtx files)
      --time-offset <OFFSET>            Scan recent events based on an offset (ex: 1y, 3M, 30d, 24h, 30m)
      --timeline-end <DATE>             End time of the event logs to load (ex: "2022-02-22 23:59:59 +09:00")
      --timeline-start <DATE>           Start time of the event logs to load (ex: "2020-02-22 00:00:00 +09:00")

Output:
  -G, --GeoIP <MAXMIND-DB-DIR>       Add GeoIP (ASN, city, country) info to IP addresses
  -H, --HTML-report <FILE>           Save Results Summary details to an HTML report (ex: results.html)
  -M, --multiline                    Output event field information in multiple rows
  -F, --no-field-data-mapping        Disable field data mapping
      --no-pwsh-field-extraction     Disable field extraction of PowerShell classic logs
  -o, --output <FILE>                Save the timeline in CSV format (ex: results.csv)
  -p, --profile <PROFILE>            Specify output profile
  -R, --remove-duplicate-data        Duplicate field data will be replaced with "DUP"
  -X, --remove-duplicate-detections  Remove duplicate detections (default: disabled)

Display Settings:
      --no-color            Disable color output
  -N, --no-summary          Do not display Results Summary for faster speed
  -q, --quiet               Quiet mode: do not display the launch banner
  -v, --verbose             Output verbose information
  -T, --visualize-timeline  Output event frequency timeline (terminal needs to support unicode)

Time Format:
      --European-time     Output timestamp in European time format (ex: 22-02-2022 22:00:00.123 +02:00)
  -O, --ISO-8601          Output timestamp in original ISO-8601 format (ex: 2022-02-22T10:10:10.1234567Z) (Always UTC)
      --RFC-2822          Output timestamp in RFC 2822 format (ex: Fri, 22 Feb 2022 22:00:00 -0600)
      --RFC-3339          Output timestamp in RFC 3339 format (ex: 2022-02-22 22:00:00.123456-06:00)
      --US-military-time  Output timestamp in US military time format (ex: 02-22-2022 22:00:00.123 -06:00)
      --US-time           Output timestamp in US time format (ex: 02-22-2022 10:00:00.123 PM -06:00)
  -U, --UTC               Output time in UTC format (default: local time)

--disable-abbreviations option

% ./hayabusa csv-timeline -d ../hayabusa-sample-evtx/EVTX-to-MITRE-Attack/Antivirus -q -w -b
Start time: 2024/11/15 11:10

Total event log files: 2
Total file size: 2.2 MB

Loading detection rules. Please wait.

Excluded rules: 26
Noisy rules: 12 (Disabled)

Deprecated rules: 216 (5.01%) (Disabled)
Experimental rules: 373 (8.65%)
Stable rules: 241 (5.59%)
Test rules: 3,700 (85.77%)
Unsupported rules: 42 (0.97%) (Disabled)

Correlation rules: 3 (0.07%)
Correlation referenced rules: 3 (0.07%)

Hayabusa rules: 175
Sigma rules: 4,139
Total detection rules: 4,314

Creating the channel filter. Please wait.

Evtx files loaded after channel filter: 2
Detection rules enabled after channel filter: 29

Output profile: standard

Scanning in progress. Please wait.

Timestamp · RuleTitle · Level · Computer · Channel · EventID · RecordID · Details · ExtraFieldInfo
2020-12-11 21:28:01.299 +09:00 · Defender Alert (High) · high · WIN10-client01.offsec.lan · Microsoft-Windows-Windows Defender/Operational · 1116 · 171 · Threat: HackTool:Win64/Mikatz!dha ¦ Severity: High ¦ Type: Tool ¦ User: OFFSEC\admmig ¦ Path: file:_C:\Users\admmig\Documents\mimidrv.sys ¦ Proc: C:\Windows\explorer.exe · Action ID: 9 ¦ Action Name: Not Applicable ¦ Additional Actions ID: 0 ¦ Additional Actions String: No additional actions required ¦ Category ID: 34 ¦ Detection ID: {82C6A580-0C4C-48BD-A0AC-6D3DE58FDABB} ¦ Detection Time: 2020-12-11T12:28:01.177Z ¦ Engine Version: AM: 1.1.17600.5, NIS: 1.1.17600.5 ¦ Error Code: 0x00000000 ¦ Error Description: The operation completed successfully. ¦ Execution ID: 1 ¦ Execution Name: Suspended ¦ FWLink: https://go.microsoft.com/fwlink/?linkid=37020&name=HackTool:Win64/Mikatz!dha&threatid=2147705511&enterprise=0 ¦ Origin ID: 1 ¦ Origin Name: Local machine ¦ Post Clean Status: 0 ¦ Pre Execution Status: 0 ¦ Product Name: Microsoft Defender Antivirus ¦ Product Version: 4.18.2011.6 ¦ Security intelligence Version: AV: 1.327.2245.0, AS: 1.327.2245.0, NIS: 1.327.2245.0 ¦ Severity ID: 4 ¦ Source ID: 3 ¦ Source Name: Real-Time Protection ¦ State: 1 ¦ Status Code: 1 ¦ Threat ID: 2147705511 ¦ Type ID: 0 ¦ Type Name: Concrete

fukusuket · 2024-11-15T02:13:26Z

json-timeline

help

% ./hayabusa json-timeline -h
Hayabusa v2.19.0 - Dev Build
Yamato Security (https://github.com/Yamato-Security/hayabusa - @SecurityYamato)

Usage:
  hayabusa.exe json-timeline <INPUT> [OPTIONS]

Input:
  -d, --directory <DIR>  Directory of multiple .evtx files
  -f, --file <FILE>      File path to one .evtx file
  -l, --live-analysis    Analyze the local C:\Windows\System32\winevt\Logs folder

General Options:
  -C, --clobber                        Overwrite files when saving
  -b, --disable-abbreviations          Disable abbreviations
  -h, --help                           Show the help menu
  -J, --JSON-input                     Scan JSON formatted logs instead of .evtx (.json or .jsonl)
  -w, --no-wizard                      Do not ask questions. Scan for all events and alerts
  -Q, --quiet-errors                   Quiet errors mode: do not save error logs
  -x, --recover-records                Carve evtx records from slack space (default: disabled)
  -r, --rules <DIR/FILE>               Specify a custom rule directory or file (default: ./rules)
  -c, --rules-config <DIR>             Specify custom rule config directory (default: ./rules/config)
  -s, --sort-events                    Sort events before saving the file. (warning: this uses much more memory!)
  -t, --threads <NUMBER>               Number of threads (default: optimal number for performance)
      --target-file-ext <FILE-EXT...>  Specify additional evtx file extensions (ex: evtx_data)

Filtering:
  -E, --EID-filter                      Scan only common EIDs for faster speed (./rules/config/target_event_IDs.txt)
  -A, --enable-all-rules                Enable all rules regardless of loaded evtx files (disable channel filter for rules)
  -D, --enable-deprecated-rules         Enable rules with a status of deprecated
  -n, --enable-noisy-rules              Enable rules set to noisy (./rules/config/noisy_rules.txt)
  -u, --enable-unsupported-rules        Enable rules with a status of unsupported
  -e, --exact-level <LEVEL>             Only load rules with a specific level (informational, low, medium, high, critical)
      --exclude-category <CATEGORY...>  Do not load rules with specified logsource categories (ex: process_creation,pipe_created)
      --exclude-computer <COMPUTER...>  Do not scan specified computer names (ex: ComputerA) (ex: ComputerA,ComputerB)
      --exclude-eid <EID...>            Do not scan specific EIDs for faster speed (ex: 1) (ex: 1,4688)
      --exclude-status <STATUS...>      Do not load rules according to status (ex: experimental) (ex: stable,test)
      --exclude-tag <TAG...>            Do not load rules with specific tags (ex: sysmon)
      --include-category <CATEGORY...>  Only load rules with specified logsource categories (ex: process_creation,pipe_created)
      --include-computer <COMPUTER...>  Scan only specified computer names (ex: ComputerA) (ex: ComputerA,ComputerB)
      --include-eid <EID...>            Scan only specified EIDs for faster speed (ex: 1) (ex: 1,4688)
      --include-status <STATUS...>      Only load rules with specific status (ex: experimental) (ex: stable,test)
      --include-tag <TAG...>            Only load rules with specific tags (ex: attack.execution,attack.discovery)
  -m, --min-level <LEVEL>               Minimum level for rules to load (default: informational)
  -P, --proven-rules                    Scan with only proven rules for faster speed (./rules/config/proven_rules.txt)
  -a, --scan-all-evtx-files             Scan all evtx files regardless of loaded rules (disable channel filter for evtx files)
      --time-offset <OFFSET>            Scan recent events based on an offset (ex: 1y, 3M, 30d, 24h, 30m)
      --timeline-end <DATE>             End time of the event logs to load (ex: "2022-02-22 23:59:59 +09:00")
      --timeline-start <DATE>           Start time of the event logs to load (ex: "2020-02-22 00:00:00 +09:00")

Output:
  -G, --GeoIP <MAXMIND-DB-DIR>       Add GeoIP (ASN, city, country) info to IP addresses
  -H, --HTML-report <FILE>           Save Results Summary details to an HTML report (ex: results.html)
  -L, --JSONL-output                 Save the timeline in JSONL format (ex: -L -o results.jsonl)
  -F, --no-field-data-mapping        Disable field data mapping
      --no-pwsh-field-extraction     Disable field extraction of PowerShell classic logs
  -o, --output <FILE>                Save the timeline in JSON format (ex: results.json)
  -p, --profile <PROFILE>            Specify output profile
  -R, --remove-duplicate-data        Duplicate field data will be replaced with "DUP"
  -X, --remove-duplicate-detections  Remove duplicate detections (default: disabled)

Display Settings:
      --no-color            Disable color output
  -N, --no-summary          Do not display Results Summary for faster speed
  -q, --quiet               Quiet mode: do not display the launch banner
  -v, --verbose             Output verbose information
  -T, --visualize-timeline  Output event frequency timeline (terminal needs to support unicode)

Time Format:
      --European-time     Output timestamp in European time format (ex: 22-02-2022 22:00:00.123 +02:00)
  -O, --ISO-8601          Output timestamp in original ISO-8601 format (ex: 2022-02-22T10:10:10.1234567Z) (Always UTC)
      --RFC-2822          Output timestamp in RFC 2822 format (ex: Fri, 22 Feb 2022 22:00:00 -0600)
      --RFC-3339          Output timestamp in RFC 3339 format (ex: 2022-02-22 22:00:00.123456-06:00)
      --US-military-time  Output timestamp in US military time format (ex: 02-22-2022 22:00:00.123 -06:00)
      --US-time           Output timestamp in US time format (ex: 02-22-2022 10:00:00.123 PM -06:00)
  -U, --UTC               Output time in UTC format (default: local time)

--disable-abbreviations option

% ./hayabusa json-timeline -d ../hayabusa-sample-evtx/EVTX-to-MITRE-Attack/Antivirus -q -w -b
Start time: 2024/11/15 11:12

Total event log files: 2
Total file size: 2.2 MB

Loading detection rules. Please wait.

Excluded rules: 26
Noisy rules: 12 (Disabled)

Deprecated rules: 216 (5.01%) (Disabled)
Experimental rules: 373 (8.65%)
Stable rules: 241 (5.59%)
Test rules: 3,700 (85.77%)
Unsupported rules: 42 (0.97%) (Disabled)

Correlation rules: 3 (0.07%)
Correlation referenced rules: 3 (0.07%)

Hayabusa rules: 175
Sigma rules: 4,139
Total detection rules: 4,314

Creating the channel filter. Please wait.

Evtx files loaded after channel filter: 2
Detection rules enabled after channel filter: 29

Output profile: standard

Scanning in progress. Please wait.

{
    "Timestamp": "2020-12-11 21:28:01.299 +09:00",
    "RuleTitle": "Defender Alert (High)",
    "Level": "high",
    "Computer": "WIN10-client01.offsec.lan",
    "Channel": "Microsoft-Windows-Windows Defender/Operational",
    "EventID": 1116,
    "RecordID": 171,
    "Details": {
        "Threat": "HackTool:Win64/Mikatz!dha",
        "Severity": "High",
        "Type": "Tool",
        "User": "OFFSEC\\admmig",
        "Path": "file:_C:\\Users\\admmig\\Documents\\mimidrv.sys",
        "Proc": "C:\\Windows\\explorer.exe"
    },
    "ExtraFieldInfo": {
        "Action ID": 9,
        "Action Name": "Not Applicable",
        "Additional Actions ID": 0,
        "Additional Actions String": "No additional actions required",
        "Category ID": 34,
        "Detection ID": "{82C6A580-0C4C-48BD-A0AC-6D3DE58FDABB}",
        "Detection Time": "2020-12-11T12:28:01.177Z",
        "Engine Version": "AM: 1.1.17600.5, NIS: 1.1.17600.5",
        "Error Code": "0x00000000",
        "Error Description": "The operation completed successfully.",
        "Execution ID": 1,
        "Execution Name": "Suspended",
        "FWLink": "https://go.microsoft.com/fwlink/?linkid=37020&name=HackTool:Win64/Mikatz!dha&threatid=2147705511&enterprise=0",
        "Origin ID": 1,
        "Origin Name": "Local machine",
        "Post Clean Status": 0,
        "Pre Execution Status": 0,
        "Product Name": "Microsoft Defender Antivirus",
        "Product Version": "4.18.2011.6",
        "Security intelligence Version": "AV: 1.327.2245.0, AS: 1.327.2245.0, NIS: 1.327.2245.0",
        "Severity ID": 4,
        "Source ID": 3,
        "Source Name": "Real-Time Protection",
        "State": 1,
        "Status Code": 1,
        "Threat ID": 2147705511,
        "Type ID": 0,
        "Type Name": "Concrete"
    }
}

fukusuket · 2024-11-15T02:18:58Z

eid-metrics

help

% ./hayabusa eid-metrics -h
Hayabusa v2.19.0 - Dev Build
Yamato Security (https://github.com/Yamato-Security/hayabusa - @SecurityYamato)

Usage:
  hayabusa.exe eid-metrics <INPUT> [OPTIONS]

Input:
  -d, --directory <DIR>  Directory of multiple .evtx files
  -f, --file <FILE>      File path to one .evtx file
  -l, --live-analysis    Analyze the local C:\Windows\System32\winevt\Logs folder

General Options:
  -C, --clobber                        Overwrite files when saving
  -b, --disable-abbreviations          Disable abbreviations
  -h, --help                           Show the help menu
  -J, --JSON-input                     Scan JSON formatted logs instead of .evtx (.json or .jsonl)
  -Q, --quiet-errors                   Quiet errors mode: do not save error logs
  -x, --recover-records                Carve evtx records from slack space (default: disabled)
  -c, --rules-config <DIR>             Specify custom rule config directory (default: ./rules/config)
  -t, --threads <NUMBER>               Number of threads (default: optimal number for performance)
      --target-file-ext <FILE-EXT...>  Specify additional evtx file extensions (ex: evtx_data)

Filtering:
      --exclude-computer <COMPUTER...>  Do not scan specified computer names (ex: ComputerA) (ex: ComputerA,ComputerB)
      --include-computer <COMPUTER...>  Scan only specified computer names (ex: ComputerA) (ex: ComputerA,ComputerB)
      --time-offset <OFFSET>            Scan recent events based on an offset (ex: 1y, 3M, 30d, 24h, 30m)

Output:
  -o, --output <FILE>  Save the Metrics in CSV format (ex: metrics.csv)

Display Settings:
      --no-color  Disable color output
  -q, --quiet     Quiet mode: do not display the launch banner
  -v, --verbose   Output verbose information

Time Format:
      --European-time     Output timestamp in European time format (ex: 22-02-2022 22:00:00.123 +02:00)
  -O, --ISO-8601          Output timestamp in original ISO-8601 format (ex: 2022-02-22T10:10:10.1234567Z) (Always UTC)
      --RFC-2822          Output timestamp in RFC 2822 format (ex: Fri, 22 Feb 2022 22:00:00 -0600)
      --RFC-3339          Output timestamp in RFC 3339 format (ex: 2022-02-22 22:00:00.123456-06:00)
      --US-military-time  Output timestamp in US military time format (ex: 02-22-2022 22:00:00.123 -06:00)
      --US-time           Output timestamp in US time format (ex: 02-22-2022 10:00:00.123 PM -06:00)
  -U, --UTC               Output time in UTC format (default: local time)

--disable-abbreviations option

 ./hayabusa eid-metrics -d ../hayabusa-sample-evtx/YamatoSecurity -q -b
Generating Event ID Metrics

Start time: 2024/11/15 11:17

Total event log files: 15
Total file size: 1044.5 KB

Currently scanning for event ID metrics. Please wait.

[00:00:00] 15 / 15   [========================================] 100%

Scanning finished.

Total Event Records: 203

First Timestamp: 2020-01-19 03:14:29.831 +09:00
Last Timestamp: 2024-11-04 22:59:32.624 +09:00

╭───────┬───────┬────────────────────┬──────┬─────────────────────────────────────────────────────────╮
│ Total ┆   %   ┆       Channel      ┆  ID  ┆                          Event                          │
╞═══════╪═══════╪════════════════════╪══════╪═════════════════════════════════════════════════════════╡
│ 27    ┆ 13.3% ┆ security           ┆ 4624 ┆ Logon success                                           │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 26    ┆ 12.8% ┆ security           ┆ 4672 ┆ Admin logon                                             │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 25    ┆ 12.3% ┆ security           ┆ 4634 ┆ Account logoff                                          │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 20    ┆ 9.9%  ┆ windows powershell ┆ 600  ┆ Unknown                                                 │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 14    ┆ 6.9%  ┆ microsoft-windows- ┆ 1    ┆ Process Creation                                        │
│       ┆       ┆ sysmon/operational ┆      ┆                                                         │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 10    ┆ 4.9%  ┆ microsoft-windows- ┆ 11   ┆ File Creation or Overwrite                              │
│       ┆       ┆ sysmon/operational ┆      ┆                                                         │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 10    ┆ 4.9%  ┆ microsoft-windows- ┆ 5    ┆ Process Terminated                                      │
│       ┆       ┆ sysmon/operational ┆      ┆                                                         │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 8     ┆ 3.9%  ┆ microsoft-windows- ┆ 312  ┆ Unknown                                                 │
│       ┆       ┆ terminalservices-g ┆      ┆                                                         │
│       ┆       ┆ ateway/operational ┆      ┆                                                         │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 8     ┆ 3.9%  ┆ security           ┆ 4769 ┆ Kerberos service ticket requested                       │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 4     ┆ 2.0%  ┆ security           ┆ 1102 ┆ Audit log cleared                                       │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 4     ┆ 2.0%  ┆ microsoft-windows- ┆ 27   ┆ Executable File Write Blocked                           │
│       ┆       ┆ sysmon/operational ┆      ┆                                                         │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 3     ┆ 1.5%  ┆ security           ┆ 4627 ┆ Unknown                                                 │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 3     ┆ 1.5%  ┆ windows powershell ┆ 400  ┆ Unknown                                                 │

fukusuket · 2024-11-15T02:20:27Z

log-metrics

help

% ./hayabusa log-metrics -h
Hayabusa v2.19.0 - Dev Build
Yamato Security (https://github.com/Yamato-Security/hayabusa - @SecurityYamato)

Usage:
  hayabusa.exe log-metrics <INPUT> [OPTIONS]

Input:
  -d, --directory <DIR>  Directory of multiple .evtx files
  -f, --file <FILE>      File path to one .evtx file
  -l, --live-analysis    Analyze the local C:\Windows\System32\winevt\Logs folder

General Options:
  -C, --clobber                        Overwrite files when saving
  -b, --disable-abbreviations          Disable abbreviations
  -h, --help                           Show the help menu
  -J, --JSON-input                     Scan JSON formatted logs instead of .evtx (.json or .jsonl)
  -Q, --quiet-errors                   Quiet errors mode: do not save error logs
  -x, --recover-records                Carve evtx records from slack space (default: disabled)
  -c, --rules-config <DIR>             Specify custom rule config directory (default: ./rules/config)
  -t, --threads <NUMBER>               Number of threads (default: optimal number for performance)
      --target-file-ext <FILE-EXT...>  Specify additional evtx file extensions (ex: evtx_data)

Filtering:
      --exclude-computer <COMPUTER...>  Do not scan specified computer names (ex: ComputerA) (ex: ComputerA,ComputerB)
      --include-computer <COMPUTER...>  Scan only specified computer names (ex: ComputerA) (ex: ComputerA,ComputerB)
      --time-offset <OFFSET>            Scan recent events based on an offset (ex: 1y, 3M, 30d, 24h, 30m)

Output:
  -M, --multiline      Output event field information in multiple rows for CSV output
  -o, --output <FILE>  Save the Metrics in CSV format (ex: metrics.csv)

Display Settings:
      --no-color  Disable color output
  -q, --quiet     Quiet mode: do not display the launch banner
  -v, --verbose   Output verbose information

Time Format:
      --European-time     Output timestamp in European time format (ex: 22-02-2022 22:00:00.123 +02:00)
  -O, --ISO-8601          Output timestamp in original ISO-8601 format (ex: 2022-02-22T10:10:10.1234567Z) (Always UTC)
      --RFC-2822          Output timestamp in RFC 2822 format (ex: Fri, 22 Feb 2022 22:00:00 -0600)
      --RFC-3339          Output timestamp in RFC 3339 format (ex: 2022-02-22 22:00:00.123456-06:00)
      --US-military-time  Output timestamp in US military time format (ex: 02-22-2022 22:00:00.123 -06:00)
      --US-time           Output timestamp in US time format (ex: 02-22-2022 10:00:00.123 PM -06:00)
  -U, --UTC               Output time in UTC format (default: local time)

-disable-abbreviations option

% ./hayabusa log-metrics -d ../hayabusa-sample-evtx/EVTX-to-MITRE-Attack/Antivirus -q -b
Start time: 2024/11/15 11:19

Total event log files: 2
Total file size: 2.2 MB

Currently scanning for log metrics. Please wait.

[00:00:00] 2 / 2   [========================================] 100%

Scanning finished.                                                                                                                             ╭──────────────────────┬─────────────────────┬────────┬─────────────────────┬─────────────────────┬─────────────────────┬─────────────────────╮
│ Filename             ┆ Computers           ┆ Events ┆ First Timestamp     ┆ Last Timestamp      ┆ Channels            ┆ Providers           │
╞══════════════════════╪═════════════════════╪════════╪═════════════════════╪═════════════════════╪═════════════════════╪═════════════════════╡
│ ID1151-Defender      ┆ WIN10-client01.offs ┆ 455    ┆ 2020-06-23          ┆ 2020-12-11          ┆ Microsoft-Windows-W ┆ Microsoft-Windows-W │
│ health status.evtx   ┆ ec.lan              ┆        ┆ 18:35:56.825 +09:00 ┆ 20:33:56.921 +09:00 ┆ indows Defender/Ope ┆ indows Defender     │
│                      ┆                     ┆        ┆                     ┆                     ┆ rational            ┆                     │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ID1116-1117-Defender ┆ WIN10-client01.offs ┆ 6      ┆ 2020-12-11          ┆ 2020-12-11          ┆ Microsoft-Windows-W ┆ Microsoft-Windows-W │
│ threat detected.evtx ┆ ec.lan              ┆        ┆ 21:28:01.299 +09:00 ┆ 21:28:44.317 +09:00 ┆ indows Defender/Ope ┆ indows Defender     │
│                      ┆                     ┆        ┆                     ┆                     ┆ rational            ┆                     │
╰──────────────────────┴─────────────────────┴────────┴─────────────────────┴─────────────────────┴─────────────────────┴─────────────────────╯
Elapsed time: 00:00:00.019

fukusuket · 2024-11-15T02:22:26Z

search

help

% ./hayabusa search -h
Hayabusa v2.19.0 - Dev Build
Yamato Security (https://github.com/Yamato-Security/hayabusa - @SecurityYamato)

Usage:
  hayabusa.exe search <INPUT> <--keywords "<KEYWORDS>" OR --regex "<REGEX>"> [OPTIONS]

Display Settings:
      --no-color  Disable color output
  -q, --quiet     Quiet mode: do not display the launch banner
  -v, --verbose   Output verbose information

General Options:
  -C, --clobber                        Overwrite files when saving
  -b, --disable-abbreviations          Disable abbreviations
  -h, --help                           Show the help menu
  -Q, --quiet-errors                   Quiet errors mode: do not save error logs
  -x, --recover-records                Carve evtx records from slack space (default: disabled)
  -c, --rules-config <DIR>             Specify custom rule config directory (default: ./rules/config)
  -t, --threads <NUMBER>               Number of threads (default: optimal number for performance)
      --target-file-ext <FILE-EXT...>  Specify additional evtx file extensions (ex: evtx_data)

Input:
  -d, --directory <DIR>  Directory of multiple .evtx files
  -f, --file <FILE>      File path to one .evtx file
  -l, --live-analysis    Analyze the local C:\Windows\System32\winevt\Logs folder

Filtering:
  -a, --and-logic             Search keywords with AND logic (default: OR)
  -F, --filter <FILTER...>    Filter by specific field(s)
  -i, --ignore-case           Case-insensitive keyword search
  -k, --keyword <KEYWORD...>  Search by keyword(s)
  -r, --regex <REGEX>         Search by regular expression
      --time-offset <OFFSET>  Scan recent events based on an offset (ex: 1y, 3M, 30d, 24h, 30m)

Output:
  -J, --JSON-output    Save the search results in JSON format (ex: -J -o results.json)
  -L, --JSONL-output   Save the search results in JSONL format (ex: -L -o results.jsonl)
  -M, --multiline      Output event field information in multiple rows for CSV output
  -o, --output <FILE>  Save the search results in CSV format (ex: search.csv)

Time Format:
      --European-time     Output timestamp in European time format (ex: 22-02-2022 22:00:00.123 +02:00)
  -O, --ISO-8601          Output timestamp in original ISO-8601 format (ex: 2022-02-22T10:10:10.1234567Z) (Always UTC)
      --RFC-2822          Output timestamp in RFC 2822 format (ex: Fri, 22 Feb 2022 22:00:00 -0600)
      --RFC-3339          Output timestamp in RFC 3339 format (ex: 2022-02-22 22:00:00.123456-06:00)
      --US-military-time  Output timestamp in US military time format (ex: 02-22-2022 22:00:00.123 -06:00)
      --US-time           Output timestamp in US time format (ex: 02-22-2022 10:00:00.123 PM -06:00)
  -U, --UTC               Output time in UTC format (default: local time)

-disable-abbreviations option

% ./hayabusa search -d ../hayabusa-sample-evtx -k mimikatz -q -b
Searching...

Start time: 2024/11/15 11:21

Total event log files: 598
Total file size: 139.2 MB

Currently searching. Please wait.

[00:00:01] 598 / 598   [========================================] 100%

Scanning finished.                                                                                                                             Timestamp · EventTitle · Hostname · Channel · Event ID · Record ID · AllFieldInfo · EvtxFile
2019-03-18 04:37:11.661 +09:00 · Process Access · PC04.example.corp · Microsoft-Windows-Sysmon/Operational · 10 · 4807 · CallTrace: C:\Windows\SYSTEM32\ntdll.dll+4595c|C:\Windows\system32\KERNELBASE.dll+8185|C:\Users\IEUser\Desktop\mimikatz_trunk\Win32\mimikatz.exe+5c5a9|C:\Users\IEUser\Desktop\mimikatz_trunk\Win32\mimikatz.exe+5c86c|C:\Users\IEUser\Desktop\mimikatz_trunk\Win32\mimikatz.exe+5cbd2|C:\Users\IEUser\Desktop\mimikatz_trunk\Win32\mimikatz.exe+5c4ff|C:\Users\IEUser\Desktop\mimikatz_trunk\Win32\mimikatz.exe+3b3d3 ¦ GrantedAccess: 0x1010 ¦ RuleName:  ¦ SourceImage: C:\Users\IEUser\Desktop\mimikatz_trunk\Win32\mimikatz.exe ¦ SourceProcessGUID: 365ABB72-A1E3-5C8E-0000-0010CEF72200 ¦ SourceProcessId: 3588 ¦ SourceThreadId: 2272 ¦ TargetImage: C:\Windows\system32\lsass.exe ¦ TargetProcessGUID: 365ABB72-0886-5C8F-0000-001030560000 ¦ TargetProcessId: 476 ¦ UtcTime: 2019-03-17 19:37:11.641 · ../hayabusa-sample-evtx/EVTX-ATTACK-SAMPLES/Credential Access/sysmon_10_lsass_mimikatz_sekurlsa_logonpasswords.evtx

YamatoSecurity

@fukusuket LGTM! Just one thing, I think this option will be better placed under Output instead of General Options as that is where other similar options are. Could you just move this?

fukusuket · 2024-11-15T04:42:54Z

@YamatoSecurity
Thank you for checking! Exactly! I moved this options to Output!

YamatoSecurity

@fukusuket LGTM! Thanks so much!

fukusuket added 2 commits November 15, 2024 11:01

feat: add disable-abbreviation option

c5f5a2a

test: add command

d7ff898

fukusuket self-assigned this Nov 15, 2024

fukusuket added the enhancement New feature or request label Nov 15, 2024

fukusuket added this to the 2.19.0 milestone Nov 15, 2024

fukusuket marked this pull request as ready for review November 15, 2024 02:20

fukusuket requested a review from YamatoSecurity November 15, 2024 02:21

update changelog

3b75add

YamatoSecurity requested changes Nov 15, 2024

View reviewed changes

chg: move disable-abbreviations options to Output

2c4961c

YamatoSecurity approved these changes Nov 15, 2024

View reviewed changes

YamatoSecurity merged commit 77f4b0e into main Nov 15, 2024
5 checks passed

fukusuket deleted the 1485-option-disable-abbriviation branch November 15, 2024 06:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add `--disable-abbreviations` option #1497

feat: add `--disable-abbreviations` option #1497

fukusuket commented Nov 15, 2024 •

edited

Loading

fukusuket commented Nov 15, 2024 •

edited

Loading

fukusuket commented Nov 15, 2024

fukusuket commented Nov 15, 2024

fukusuket commented Nov 15, 2024

fukusuket commented Nov 15, 2024

YamatoSecurity left a comment

fukusuket commented Nov 15, 2024 •

edited

Loading

YamatoSecurity left a comment

feat: add --disable-abbreviations option #1497

feat: add --disable-abbreviations option #1497

Conversation

fukusuket commented Nov 15, 2024 • edited Loading

What Changed

The command to which this option is added

Evidence

Integration-Test

fukusuket commented Nov 15, 2024 • edited Loading

csv-timeline

help

--disable-abbreviations option

fukusuket commented Nov 15, 2024

json-timeline

help

--disable-abbreviations option

fukusuket commented Nov 15, 2024

eid-metrics

help

--disable-abbreviations option

fukusuket commented Nov 15, 2024

log-metrics

help

-disable-abbreviations option

fukusuket commented Nov 15, 2024

search

help

-disable-abbreviations option

YamatoSecurity left a comment

Choose a reason for hiding this comment

fukusuket commented Nov 15, 2024 • edited Loading

YamatoSecurity left a comment

Choose a reason for hiding this comment

feat: add `--disable-abbreviations` option #1497

feat: add `--disable-abbreviations` option #1497

fukusuket commented Nov 15, 2024 •

edited

Loading

fukusuket commented Nov 15, 2024 •

edited

Loading

fukusuket commented Nov 15, 2024 •

edited

Loading