Documentation

Welcome to Logl! We're glad you made it here. Here's what you need to do to get Logl up and running:

Installation

1. Install pip
   Pip is a tool for installing Python packages. If you don't already have pip, you can install it here:
   http://www.pip-installer.org/en/latest/installing.html#
   
   

2. Install PyMongo
   PyMongo, MongoDB's language driver for Python, is a prerequisite for running Logl. You can download and install PymMngo using pip, as described here:
   http://api.mongodb.org/python/current/installation.html
   
   

3. Install MongoDB (which you should all do anyway because it is awesome)
   Help all of us at 10gen out by downloading MongoDB 2.1.2, our latest development release, and helping us test for 2.2!
   http://www.mongodb.org/downloads

4. Install a non-text-based browser
   We recommend Google Chrome or Firefox.

5. Install Logl
   COMING SOON
   
   

Running Logl

In order to run logl, you must also have a mongod running in the background:

$ ./mongod

Logl is runnable from the command line. Once you have a mongod running, navigate to the top-level logl directory and run the following:

$ python logl/logl.py logfile1 logfile2...

Feed Logl your log files as command line arguments. If you don't have any log files to try with Logl, we've provided some samples so you don't have to miss out on the fun:

$ python logl/logl.py test/1.log test/2.log test/3.log test/4.log test/5.log

Once Logl has launched a web page, it will spawn a python server that will run until you shut it down with Ctrl+C. Be sure to kill this server once you are finished with the visualizer. Happy Logling!

Customize Logl

We've tried to build Logl with you in mind, but every person wants something a little different from their log visualizer. So, we built Logl to make it customizable! If you're familiar with Python and JavaScript and there's something specific you'd like to track, you can add in your own filters to catch special kinds of log messages.

1. Write a python filter
   The first round of processing that Logl handles involves a series of filters that parse through the log files. This filters can be found in logl/logl/filters. We've provided an empty filter, template.py, with some structuring to help get you started.
   
   Each filter contains a criteria(msg) method and a process(date, msg) method. Feel free to add other helper methods if you'd like, but be sure to write these two.
   
   criteria() takes a log line and decides whether it fits the given filter. It should return integer codes to indicate a match.
   
   process() packages matching log lines into a documents, which later get stored in a mongod using the db "logl" and a randomly-generated collection name. The structure of these documents is outlined in each filter, and is slightly different for each type of message. If you're writing your own filter, its documents will have to contain the fields that are outlined in template.py. Feel free to add additional fields, if you'd like. Logl.py will add an "origin_server" field to each document after it is processed.
   
   

2. Add logic to event_matchup.py
   Once Logl has finished parsing the log files themselves, it analyzes the documents its filters have stored in the mongod and packages them into discrete "events". An event is something that happens to the server cluster. It can be seen by just the target server (the one that was affected by the change in the state of the world) or can be seen by multiple servers. For example, when a server becomes PRIMARY this is usually seen by all servers in the cluster. By contrast when a server becomes locked by the user, this is a change only seen by the target server.
   
   Event matching is done in logl/logl/post/event_matchup.py. You'll need to add some additional logic here if your new filter has special fields.
   
   -Check out the format of an event object, outlined in this module. Decide if you need to add any additional fields to this dictionary.
   
   -In the next_event() method, if your filter detects events only viewable by the target server, add the name you assigned for its "type" field to the loners[] list at the top.
   
   -Futher down, next_event() makes some checks for specific types of messages. If your filter needs some special handling, add it here.
   
   -If your filter does set some extra fields, but is not a loner[] message, add some checks to the type_check() method. This method checks to see if two messages from different servers are actually talking about the same event (For example, the messages "I am now the PRIMARY" and "That guy over there is the PRIMARY" would usually coincide.)
   
   -Lastly, add some fields to the generate_summary() method. This generates a nice, human-readable summary of the event that later gets displayed in the visualizer.

Bug reports and feature requests

We'd love to hear your input.