Elastic Butler was formed at Oi telecomunication company as an open source alternative to Elastic Stack Alerting tool.
Whereas other services issue notifications based on pre-determined parameters, Elastic Butler allows the user to create their own. Users define the pattern they desire a notification for, known as a recipes, as well as how they are notified, known as notification types.
Such examples include, but are not limited to:
- Notify me by mail if there are more than 10 fail login attempts in the last 20 minutes
- Notify me by if we sell more than 1000 iphones in the last day
Elastic Butler will store the recipes created, along with the execution result at the Elastic Search Index. Prior to launching Elastic Butler, check the configuration file config/env.json to set up the desired configurations (i.e. - recipes).
Once complete, launch Electric Butler.
NOTE: The user can also use sandbox to perform the first test. Sandbox provides users with an elasticearch and Kibana instances running on the Docker platform.
npm startButler uses the @timestamp field to do the "period" filter. Make sure your index has this field.
A file config/env.json exists to allow you to provide configuration for this application. This includes many things like sender configuration and data store configuration.
Elastic butler will store the recipes and the execution result at a Elastic Search index this means you can build dashboards and visualizations about your alerts! You could even alert on your alerts if you wanted to.
A recipe describe the operation of monitoring. This is how a recipe looks like:
{
"name": "test-recipe",
"application": "test",
"active": true,
"elasticsearch": "http://localhost:9200",
"kibana": "http://localhost:5601",
"interval": 10,
"search": {
"index": "shakespeare",
"query": "\"with love\"",
"limit": 10,
"period": "10 m"
},
"action": {
"type": "gmail",
"to": "[email protected]",
"subject": "[#hits#] hits at [#application# #recipe#]",
"body": "<p>Your recipe results:</p> #detail#"
}
}To create your first recipe POST it to the store.recipeIndex ( elastic_butler_recipe by default )
POST http://localhost:9200/elastic_butler_recipe/_doc/
{
"name": "test-recipe",
"application": "test",
"active": true,
"elasticsearch": "http://localhost:9200",
"kibana": "http://localhost:5601",
"interval": 1,
"search": {
"index": "shakespeare",
"query": "with love",
"limit": 10,
"period": "10 m"
},
"action": {
"type": "gmail",
"to": "[email protected]",
"subject": "[#hits#] hits at [#application# #recipe#]",
"body": "<p>Your recipe results:</p> #detail#"
}
}or just execute the setup:
node setup.js
- name: Recipe name
- application: Monitored application name
- elasticsearch: Elastic-search url with port information (Ex: localhost:9200)
- search: This is the object to specify the search
- search.index: Search index name. If you set dateOnIndex at config/env.json, butler will add the current date (-YYYY-MM-DD) at the end of index name.
- search.query: Elastic search query. (Ex.: code:"500" && "EXTERNAL API ERROR")
- search.limit: Limit of hits until action be executed
- search.period: Period in minutes that the occurrence will be searched. (Ex: 20 m) "At last 20 minutes"
- action: This is the object to specify the action. For default butler has 2 action types (gmail and twiliosms). Butler will search for this type at worker/senders folder. You can create your own sender.
This action is part of butler default solution, and uses a gmail account to send the notification.
- action.type: gmail
- action.to: Recipient's email
- action.subject: Mail subject. Tags #hits#, #application# and #recipe# will be replaced with recipe data
- action.body: Mail body. Tag #detail# will be replaced with search result data
{
"name" : "test-recipe",
"application" : "test",
"active" : false,
"elasticsearch" : "http://localhost:9200",
"kibana" : "http://localhost:5601",
"interval" : 10,
"action" : {
"body" : "<p>Your recipe results:</p> #detail#",
"subject" : "[#hits#] hits at [#application# #recipe#]",
"to" : "[email protected]",
"type" : "gmail"
},
"search" : {
"index" : "shakespeare",
"query" : "\"with love\"",
"limit" : "10",
"period" : "60 m"
}
}This action is part of butler default solution, and uses a twilio account to send sms.
- action.type: twiliosms
- action.to: Recipient's phone number (Ex: +5521999998888)
- action.body: Sms body. Tags #hits#, #application# and #recipe# will be replaced with recipe data
{
"name" : "test-recipe-sms",
"application" : "test",
"active" : true,
"elasticsearch" : "http://localhost:9200",
"kibana" : "http://localhost:5601",
"interval" : 10,
"action" : {
"body" : "#application# #recipe# => #hits# hits",
"to" : "+5521999998888",
"type" : "twiliosms"
},
"search" : {
"index" : "shakespeare",
"query" : "\"with love\"",
"limit" : "10",
"period" : "6000 m"
}
}This action is part of butler default solution, and uses a slack webhook to send a message.
This action allows you to specify multiple slack webhooks to alert different rooms depending on your recipe.
- Go to https://{yourteam}.slack.com/apps
- Install Incoming Webhooks.
- Generate a new webhook for your recipe.
- Copy the url and include it in the recipe.
- action.type: slack
- action.webhookUrl: https://hooks.slack.com/services/...
- action.username: Elastic Butler
- action.iconEmoji: ghost
- action.detailField: text_entry
- action.message: Message body. Tags #hits#, #application#, #recipe#, and #detail# will be replaced with recipe/hit data
{
"name" : "test-recipe",
"application" : "test",
"active" : true,
"elasticsearch" : "http://localhost:9200",
"kibana" : "http://localhost:5601",
"interval" : 1,
"action" : {
"type" : "slack",
"message": "[#application#] recieved [#hits#] hits for [#recipe#] \n\n #detail#",
"webhookUrl": "https://hooks.slack.com/services/...",
"username": "Elastic Butler",
"iconEmoji": "ghost",
"detailField": "text_entry"
},
"search" : {
"index" : "shakespeare",
"query" : "\"with love\"",
"limit" : "0",
"period" : "10 m"
}
}For test propouses you can use our sandbox to create an initial test environment:
cd _sandbox
sudo ./sandbox.sh up -dOur sandbox will use docker-compose to run elastic-search and kibana containers.
After containers are running, you should need import some sample data:
Add sample index mapping at http://localhost:5601/app/kibana#/dev_tools/console
PUT /shakespeare
{
"mappings": {
"doc": {
"properties": {
"speaker": {
"type": "keyword"
},
"play_name": {
"type": "keyword"
},
"line_id": {
"type": "integer"
},
"speech_number": {
"type": "integer"
}
}
}
}
}Add the @timestamp ingest to automatic include timestamp information in you data.
PUT _ingest/pipeline/timestamp
{
"description" : "Adds a timestamp field at the current time",
"processors" : [ {
"set" : {
"field": "@timestamp",
"value": "{{_ingest.timestamp}}"
}
} ]
}Now you can import some sample data to your index:
cd _sandbox/sample_data
curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/shakespeare/doc/_bulk?pretty&pipeline=timestamp' --data-binary @shakespeare_6.0.jsonTo run butler on a docker container you need to adjust config/env.json:
- Edit store configuration url and options
- Edit senders configurations
Than just run:
cd _docker
sudo ./up.shIf you want to see container logs:
sudo docker logs butlerTo stop butler:
cd _docker
sudo ./down.shYou can create your own butler sender. To do this you have to create a sender class at worker/senders folder with the name of your type:
// worker/senders/sms.js
class SmsSender {
constructor() {
this.config = require('../../config');
}
// Send recipe result by sms
send(recipe, searchResult) {
let message = this._getInfo(recipe.action.body, recipe, searchResult);
// send the sms notification here...
};
_getInfo(text, recipe, searchResult) {
return text
.replace('#application#', recipe.application)
.replace('#recipe#', recipe.name)
.replace('#hits#', searchResult.hits.total);
}
};
module.exports = new SmsSender();Than you can create a recipe using your new sender:
{
"name": "Error 500 on process recipe",
"application": "My Application",
"active": true,
"elasticsearch": "http://localhost:9200",
"kibana": "http://localhost:5601",
"interval": 10,
"search": {
"index": "shakespeare",
"query": "\"process\" && code: \"500\"",
"limit": 10,
"period": "10 m"
},
"action": {
"type": "sms",
"to": "2199999-8888",
"body": "#application# #recipe# => #hits# hits"
}
}