Logging

I like logging. Logging is one way to monitor your application. Strictly speaking, logs are not for monitoring your application, but more for tracing and debugging processes throughout your application.

The following are some notes on logging, loggers, logging tools, log stores and stashes.

Logging Basics

Logs originate from the stdout of your application. When your application runs, the stdout of the application is stored into a text file, this file is the log file.

Our goal when running an application in production is to keep these logs in a place where we can view, parse, and even search the logs for useful information about data, the application state, or just in general what is happening.

Logging Transports

There aren’t strictly logging-specific protocols, but there are “transports”. These are, basically, various ways that you can get your logs transferred from your application stdout to your log store.

• syslog -
• udp -
• tcp -
• http -

Log Stores

These aren’t all directly the same thing, but various log aggregation tools and platforms outside of the log aggregation story. Basically places to put and view your logs.

• ELK stack - Elasticsearch, Logstash, Kibana. Very popular and powerful. Can be self hosted. Heavy af.
• Loki + Promtail + Grafana (LPG stack)
• splunk
• tick stack - all open source tools.
• AWS Cloudwatch
• My list of logging stuff

Logging Best Practices

• Each log should be a single line of json in the logfile.
• Use a system of transaction id or request id. Each process or request that runs through your application should generate an id, and when you log something this id should be present in the log. This way you can, for example, trace an HTTP request as it runs through your application.
• Don’t log secrets or sensitive application information, like API keys.
• Don’t log sensitive user information, like user passwords or other private information.
• Don’t make your logs too big — you don’t have to log 1000 results in a json object after running a database query.
• Write a useful log message
• Use proper log levels — e.g., error, warn, info, debug. These levels depend on your logging library or implementation.
• Don’t use debug package
• Don’t use the default stdout writer (e.g., console.log for Node)

Node JS Logging Libraries

• bunyan - old but gold. Works with a variety of logging protocols. Also check bunyan-logger-factory
• winston - never used it, but it’s very popular for Node.
• pino - proports to be the fastest Node JS logger.

Go Logging Libraries

• zap
• log - built-in lib