Ingest Custom Logs LogStash

Installing Logstash

Follow the Official Installing Logstash instructions.

• Install Java

sudo apt-get install default-jre

• Download and install the Public Signing Key

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

• Run sudo apt-get update. You can install it with:

sudo apt-get update && sudo apt-get install logstash

Installing the Microsoft Log Analytics output plugin for Logstash

Follow the installtion instructions from Azure Sentinel Github Dataconnectors for microsoft-logstash-output-azure-loganalytics

bin/plugin install microsoft-logstash-output-azure-loganalytics
# or
bin/logstash-plugin install microsoft-logstash-output-azure-loganalytics  (Newer versions of Logstash)

Configuration File Example

Check the example configuration file. You can also find more configuration examples in the Azure Sentinel Community github under parsers

output {
    microsoft-logstash-output-azure-loganalytics {
        workspace_id => "<OMS WORKSPACE ID>"
        workspace_key => "<CLIENT AUTH KEY>"
        custom_log_table_name => "<LOG TYPE NAME>"
        key_names  => ['key1','key2','key3'..] ## list of Key names
        flush_items => <FLUSH_ITEMS_NUM>
        plugin_flush_interval => <FLUSH INTERVAL TIME(sec)>
    }
}

Ingest custom logs using Microsoft Log Analytics output plugin for Logstash

bin/logstash -f config-file.conf

Scalable SYSLOG CEF Collection using Logstash and VMSS

If you are looking for ARM template to do scalable syslog CEF collection using Logstash, refer the DataConnector for more instructions.

Scalable SYSLOG CEF Collection using Logstash and VMSS