A Few Notes on Logstash


Logstash is a popular logging component so I have been experimenting with it. Its primary role in a logging architecture is message translator. It works like this: it receives a log message from an input source, translates the message from one format to another, and then send the new message to an output source. For example, it can receive a message from a syslog socket, translate the single line formatted message into structured JSON, and send the JSON message to the elasticsearch database.

The good things about logstash are its simple configuration file format and the large number of input and output sources it supports. logstash is at the forefront of new logging tools that promote structured log messages in favor of the old-fashioned, long obsolete, but still popular syslog format. I like that it implicitly assumes a pipe-and-filter model and is designed to be a component of a larger logging architecture.

logstash has decent primary documentation by open source standards. While it doesn’t provide a decent overview or introduction, the fact that it offers users a simple conceptual model (input->filter->output) that lack isn’t too serious.  That simple model is a strong point in its favor.  Offering users an easy-to-grasp conceptual model of your software, whether library or application, is a sign of good design.

The not so good things about logstash are that it lacks maturity and its very odd language choice. logstash is written in JRuby. This means that its written in a version of Ruby that must run inside Java. Ruby I could understand. Java I could understand. But JRuby for a production application? I’m totally baffled by that choice. That is not a choice that favors speed and simplicity. And indeed it is slow to start, resource hungry, and not very fast.

When I say that it lacks maturity I mean that it lacks the basics such as a Linux service init script and an installation package (e.g, deb or RPM). The lack of these basics made it very painful to install. Yes, you can find a variety of init scripts on the Internet but I have yet to find one that works well. Running the logstash jar file initially gave me a weird error about an “ffi” problem. This turned out to be due to the noexec option set on my /tmp partition per the SCAP rules. That took a long time to debug. Needless to say logstash doesn’t include an SELinux policy either.

logstash is one of those applications with strong positives and strong negatives: I’m glad it exists, I’m just not sure I want to use it.



Categories: Software

Tags: ,

2 replies

  1. Adam, have you looked at fluentd? I ran across it today while on the Elasticsearch website, looking at Kibana (again) http://docs.fluentd.org/articles/free-alternative-to-splunk-by-fluentd

    Like

Share Your Ideas

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: