Skip to content

yanming-zhang/log-pilot

 
 

Repository files navigation

log-pilot

CircleCI Go Report Card

log-pilot is an awesome docker log tool. With log-pilot you can collect logs from docker hosts and send them to your centralized log system such as elasticsearch, graylog2, awsog and etc. log-pilot can collect not only docker stdout but also log file that inside docker containers.

Try it

Prerequisites:

  • docker-compose >= 1.6
  • Docker Engine >= 1.10
# download log-pilot project
git clone [email protected]:AliyunContainerService/log-pilot.git
# build log-pilot image
cd log-pilot/ && ./build-image.sh
# quick start
cd quickstart/ && ./run

Then access kibana under the tips. You will find that tomcat's has been collected and sended to kibana.

Create index: kibana

Query the logs: kibana

Quickstart

Run pilot

docker run --rm -it \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v /etc/localtime:/etc/localtime \
    -v /:/host:ro \
    --cap-add SYS_ADMIN \
    registry.cn-hangzhou.aliyuncs.com/acs/log-pilot:0.9.5-filebeat

Run applications whose logs need to be collected

Open a new terminal, run the application. With tomcat for example:

docker run -it --rm  -p 10080:8080 \
    -v /usr/local/tomcat/logs \
    --label aliyun.logs.catalina=stdout \
    --label aliyun.logs.access=/usr/local/tomcat/logs/localhost_access_log.*.txt \
    tomcat

Now watch the output of log-pilot. You will find that log-pilot get all tomcat's startup logs. If you access tomcat with your broswer, access logs in /usr/local/tomcat/logs/localhost_access_log.\*.txt will also be displayed in log-pilot's output.

More Info: Fluentd Plugin and Filebeat Plugin

Feature

  • Support both fluentd plugin and filebeat plugin. You don't need to create new fluentd or filebeat process for every docker container.
  • Support both stdout and log files. Either docker log driver or logspout can only collect stdout.
  • Declarative configuration. You need do nothing but declare the logs you want to collect.
  • Support many log management: elastichsearch, graylog2, awslogs and more.
  • Tags. You could add tags on the logs collected, and later filter by tags in log management.

Build log-pilot

Prerequisites:

  • Go >= 1.6
go get github.com/AliyunContainerService/log-pilot
cd $GOPATH/github.com/AliyunContainerService/log-pilot
# This will create a new docker image named log-pilot:latest
./build-image.sh

Contribute

You are welcome to make new issues and pull reuqests.

Packages

No packages published

Languages

  • Go 52.8%
  • Shell 24.9%
  • Ruby 11.9%
  • Smarty 4.8%
  • Makefile 3.6%
  • Python 2.0%