“Professional service
that won't make you nervous”

Macintosh  Perl  Linux

|tailfilter
Streaming Firewall logfile filter

Turn hard to read insanity like this:

Aug 2 14:20:59 pcp01501167pcs kernel: IN=eth0 OUT= MAC=00:80:c8:f2:78:da:00:01:5c:22:2b:02:08:00 SRC=68.82.158.253 DST=68.82.178.180 LEN=48 TOS=0x00 PREC=0x80 TTL=119 ID=51893 DF PROTO=TCP SPT=4109 DPT=80 WINDOW=16384 RES=0x00 SYN URGP=0
Aug 2 14:21:02 pcp01501167pcs kernel: IN=eth0 OUT= MAC=00:80:c8:f2:78:da:00:01:5c:22:2b:02:08:00 SRC=68.82.158.253 DST=68.82.178.180 LEN=48 TOS=0x00 PREC=0x80 TTL=119 ID=52204 DF PROTO=TCP SPT=4109 DPT=80 WINDOW=16384 RES=0x00 SYN URGP=0
Aug 2 14:21:08 pcp01501167pcs kernel: IN=eth0 OUT= MAC=00:80:c8:f2:78:da:00:01:5c:22:2b:02:08:00 SRC=68.82.158.253 DST=68.82.178.180 LEN=48 TOS=0x00 PREC=0x80 TTL=119 ID=52661 DF PROTO=TCP SPT=4109 DPT=80 WINDOW=16384 RES=0x00 SYN URGP=0
Aug 2 14:27:05 pcp01501167pcs kernel: IN=eth0 OUT= MAC=00:80:c8:f2:78:da:00:01:5c:22:2b:02:08:00 SRC=68.82.191.202 DST=68.82.178.180 LEN=48 TOS=0x00 PREC=0x00 TTL=123 ID=7470 DF PROTO=TCP SPT=1926 DPT=80 WINDOW=16384 RES=0x00 SYN URGP=0
Aug 2 14:27:08 pcp01501167pcs kernel: IN=eth0 OUT= MAC=00:80:c8:f2:78:da:00:01:5c:22:2b:02:08:00 SRC=68.82.191.202 DST=68.82.178.180 LEN=48 TOS=0x00 PREC=0x00 TTL=123 ID=7694 DF PROTO=TCP SPT=1926 DPT=80 WINDOW=16384 RES=0x00 SYN URGP=0
Aug 2 14:27:14 pcp01501167pcs kernel: IN=eth0 OUT= MAC=00:80:c8:f2:78:da:00:01:5c:22:2b:02:08:00 SRC=68.82.191.202 DST=68.82.178.180 LEN=48 TOS=0x00 PREC=0x00 TTL=123 ID=8121 DF PROTO=TCP SPT=1926 DPT=80 WINDOW=16384 RES=0x00 SYN URGP=0
Aug 2 14:31:58 pcp01501167pcs kernel: IN=eth0 OUT= MAC=00:80:c8:f2:78:da:00:01:5c:22:2b:02:08:00 SRC=217.204.59.46 DST=68.82.178.180 LEN=404 TOS=0x00 PREC=0x00 TTL=108 ID=34486 PROTO=UDP SPT=1147 DPT=1434 LEN=384

... Into this!
Shot of Tailfilter in action. boring, but what did you expect?

Current Version: 0.76 -- see CHANGES file for details.


Download the documentation in PDF format

Download the tailfilter.tgz (which includes both the program and the PDF docs)

View some sample output (long)

Requirements:

  • A version of linux/unix that supports the ipchains/iptables firewalling code.
  • Rules in your firewall that log packet attempts
  • A compatible logfile format
  • SuperUser access, in order to read /var/log/messages or similar (wherever firewall logging is stored)
  • Have the appropriate amount of fun.

Description follows, further documentation is available as part of the package.


SYNOPSIS

Tailfilter is something that started off as a perl one-liner, took on a life of its own, and swiftly grew out of control because I couldn't stop thinking of ways to enhance the original idea.

Essentially, it's a logfile filter that reformats the output from the log and 'pretty-prints' a more legible arrangement that lends itself better to rapid and/or cursory analysis, as well as offering immediate notification of new events as they occur. It additionally caches the results of DNS lookups, as well as the TCP-based services in /etc/services to speed up the info lookups considerably.

To use it, simply do one of the following (or similar):

tail -f /var/log/messages |tailfilter

sudo tail -n 50 -f /var/log/messages |./tailfilter -l 40 -c

(depending on where you keep tailfilter :-) Also see Setup below.


DESCRIPTION

Zen and the Art of Reading Firewall Logs...

As a relative newcomer to Linux, I've found the logging information from a straight tail -f to be a little mysterious looking, and not at all conducive to casual inspection, and wanted to find a way to filter out the information I was interested in. It really did start as a simple Perl one-liner though, if you can believe it. :-)

Basically, tailfilter filters your logfiles for packet log information from the firewall and reformats them to look a little nicer, does a cached hostname lookup on the IP in question, and adds a columnised report layout, (thanks to Perl's format STDOUT command), which just goes one step further to making the information in the logfiles more understandable. Nothing in your logs is actually changed, mind you, just 'filtered' through a little Perl magic. ;-)

Tailfilter was designed to work with the output of iptables/ipchains, and your logfile entries should look like this:

ipchains:

 Jul 28 06:36:37 pcp01487622pcs kernel: Packet log: input DENY eth0 PROTO=6
 68.82.244.101:4434 68.82.41.167:80 L=48 S=0x00 I=23517 F=0x4000 T=119 SYN (#17)

iptables:

 Aug 18 04:05:17 anakin kernel: Blocked incoming port: IN=eth0 OUT= 
 MAC=00:10:dc:21:ad:36:00:00:c5:7d:5d:2c:08:00 SRC=4.2.2.1 DST=216.163.77.13 
 LEN=72 TOS=0x00 PREC=0x00 TTL=242 ID=2208 DF PROTO=UDP SPT=53 DPT=32919 LEN=52

Note on Logfiles:

  1. If you have a different format in /var/log/messages for your firewall entries, this script will require adjustment to account for those differences. Please let me know if you do, and I'll see if there's anything I can do about it.

  2. Important: Your firewall MUST be set to be logging some packets for whatever reasons you decide, or this won't find anything in your logs to parse. i.e. it won't do nuttin'. :-)

    You can do a quick check with this:

    grep "kernel:" /var/log/messages |less

    Look for entries similar to the above two samples.

Caching /etc/services and /etc/protocols for fast port/protocol lookups:

During the reading in of /etc/services and /etc/protocols, it will warn you if it finds any service name that the current regex does not catch (and it will skip that line.) You can study the warning line and tweak the regex accordingly for your system.


COPYRIGHT

Copyright (c) 2001 Scott R. Godin. All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.