Using Docker logs

I recently switched to using Docker volumes for logs., to reduce installation steps and coupling to infrastructure. I created a single shared volume by stack.

However this implied that logs from stopped containers were never removed… because logs rotation was not doing its work anymore.

I then understood better why 12 factors app practices recommends to only log to STDOUT, and let Docker handle with log storage and removal. And I decide to adapt.

  • Stop JSON logging on files
  • Replace human readable logging on STDOUT by json logging
  • Change filebeat configuration to container mode

At the same time, I benefited from this change in many ways:

  • Traefik, metricbeat and filebeat logs are also captured
  • Elasticsearch and kibana logs are captured on dev
  • All logs have been switched to JSON
  • Filebeat allow enriching logs with Docker metadata, allowing to know easily the origin of the log
  • Filebeat processors and scripting allow reshaping the logs to have a common format for all sources 🙂 Thanks Elastic devs!

It is all in place and deployed! More observability than ever.

Leave a Reply

Your email address will not be published. Required fields are marked *