The Pig Doktah is Born! - A Snort Performance Monitor Interpretation Tool

After receiving a variety of feedback via email, irc, twitter and even some comments on this blog, I have decided to start an updated and maintained project to parse, display, and ultimately make configuration / tuning suggestions based on the snort performance monitor output.  As you may have guessed by now, this tool is called the Pig Doktah and can be found at http://thepigdoktah.googlecode.com.  The current version reads snort performance monitor output and stores in a hash for quick access, sorting etc...

I am still quite interested in what specific metrics people are most interested, so please don't hesitate to hit me up via comment on this blog, irc (freenode - #snort), email, or twitter.

During the development, and for the foreseeable future, I will have the latest version of the code sending output to this location on an hourly basis: http://rootedyour.com/enhanced/pminfo.htm

Sample output:

-= Tha Pig Doktah 0.1 Dev =-
Copyright (C) 2010 JJ Cummings

Report Info:
    Processed: /var/tmp/snortstat
    First Entry: Wed Sep  1 11:34:05 2010
    Last Entry: Wed Sep  8 09:00:17 2010
    Time Span: 6 days, 21 hours, 26 minutes and 12 seconds

Wirespeed:
    High: 10.613 Mbits/Sec | Sat Sep  4 07:59:48 2010
    Low: 0.006 Mbits/Sec | Sat Sep  4 07:12:47 2010
    Avg: 1.953 Mbits/Sec
  
% Packet Loss:
    High: 10.504% | Sat Sep  4 03:00:00 2010
    Low: 0.000% | Wed Sep  8 08:41:27 2010
    Avg: 1.002%

Additional Info:
    Avg Pkt Size: 803.413 bytes
    Avg Syns/Sec: 0.181
    Avg SynAcks/Sec: 0.124
    Avg Alerts/Sec: 0.001
    Avg Current Cached Sessions: 6671.668

Raw Values:
     alerts avg = 0.001
     alerts high = 0.032
     alerts high_date = Wed Sep  1 12:32:57 2010
     alerts low = 0.000
     alerts low_date = Wed Sep  8 09:00:17 2010
     attrib_hosts_current avg = 0.000
     attrib_hosts_current high = 0.000
     attrib_hosts_current high_date = Wed Sep  8 09:00:17 2010
     attrib_hosts_current low = 0.000
     attrib_hosts_current low_date = Wed Sep  8 09:00:17 2010
     attrib_reloads avg = 0.000
     attrib_reloads high = 0
     attrib_reloads high_date = Wed Sep  8 09:00:17 2010
     attrib_reloads low = 0
     attrib_reloads low_date = Wed Sep  8 09:00:17 2010
     bytes_applayer avg = 0.252
     bytes_applayer high = 1.352
     bytes_applayer high_date = Sat Sep  4 07:59:48 2010
     bytes_applayer low = 0.006
     bytes_applayer low_date = Tue Sep  7 09:13:56 2010
     bytes_ipfrag avg = 0.000
     bytes_ipfrag high = 0
     bytes_ipfrag high_date = Wed Sep  8 09:00:17 2010
     bytes_ipfrag low = 0
     bytes_ipfrag low_date = Wed Sep  8 09:00:17 2010
     bytes_ipreass avg = 2279.291
     bytes_ipreass high = 3660
     bytes_ipreass high_date = Thu Sep  2 13:47:36 2010
     bytes_ipreass low = 368
     bytes_ipreass low_date = Thu Sep  2 10:22:15 2010
     bytes_tcprebuilt avg = 892.669
     bytes_tcprebuilt high = 1458
     bytes_tcprebuilt high_date = Sun Sep  5 15:19:06 2010
     bytes_tcprebuilt low = 136
     bytes_tcprebuilt low_date = Sat Sep  4 00:58:27 2010
     cpu1_idle avg = 95.767
     cpu1_idle high = 99.977
     cpu1_idle high_date = Sat Sep  4 00:58:27 2010
     cpu1_idle low = 69.943
     cpu1_idle low_date = Tue Sep  7 06:20:11 2010
     cpu1_sys avg = 0.051
     cpu1_sys high = 0.287
     cpu1_sys high_date = Sat Sep  4 07:59:48 2010
     cpu1_sys low = 0.000
     cpu1_sys low_date = Wed Sep  8 08:07:19 2010
     cpu1_user avg = 4.183
     cpu1_user high = 29.860
     cpu1_user high_date = Tue Sep  7 06:20:11 2010
     cpu1_user low = 0.023
     cpu1_user low_date = Sat Sep  4 00:58:27 2010
     cpu_count avg = 1.000
     cpu_count high = 1
     cpu_count high_date = Wed Sep  8 09:00:17 2010
     cpu_count low = 1
     cpu_count low_date = Wed Sep  8 09:00:17 2010
     drops avg = 1.002
     drops high = 10.504
     drops high_date = Sat Sep  4 03:00:00 2010
     drops low = 0.000
     drops low_date = Wed Sep  8 08:41:27 2010
     filtered_tcp avg = 3790.598
     filtered_tcp high = 45608
     filtered_tcp high_date = Tue Sep  7 09:24:12 2010
     filtered_tcp low = 85
     filtered_tcp low_date = Wed Sep  1 11:50:25 2010
     filtered_udp avg = 3790.598
     filtered_udp high = 45608
     filtered_udp high_date = Tue Sep  7 09:24:12 2010
     filtered_udp low = 85
     filtered_udp low_date = Wed Sep  1 11:50:25 2010
     frag_auto avg = 0.000
     frag_auto high = 0.000
     frag_auto high_date = Wed Sep  8 09:00:17 2010
     frag_auto low = 0.000
     frag_auto low_date = Wed Sep  8 09:00:17 2010
     frag_complete avg = 0.000
     frag_complete high = 0.000
     frag_complete high_date = Wed Sep  8 09:00:17 2010
     frag_complete low = 0.000
     frag_complete low_date = Wed Sep  8 09:00:17 2010
     frag_current avg = 0.000
     frag_current high = 0
     frag_current high_date = Wed Sep  8 09:00:17 2010
     frag_current low = 0
     frag_current low_date = Wed Sep  8 09:00:17 2010
     frag_delete avg = 0.000
     frag_delete high = 0.000
     frag_delete high_date = Wed Sep  8 09:00:17 2010
     frag_delete low = 0.000
     frag_delete low_date = Wed Sep  8 09:00:17 2010
     frag_faults avg = 0.000
     frag_faults high = 0
     frag_faults high_date = Wed Sep  8 09:00:17 2010
     frag_faults low = 0
     frag_faults low_date = Wed Sep  8 09:00:17 2010
     frag_flushes avg = 0.000
     frag_flushes high = 0.000
     frag_flushes high_date = Wed Sep  8 09:00:17 2010
     frag_flushes low = 0.000
     frag_flushes low_date = Wed Sep  8 09:00:17 2010
     frag_insert avg = 0.000
     frag_insert high = 0.000
     frag_insert high_date = Wed Sep  8 09:00:17 2010
     frag_insert low = 0.000
     frag_insert low_date = Wed Sep  8 09:00:17 2010
     frag_max avg = 0.000
     frag_max high = 0
     frag_max high_date = Wed Sep  8 09:00:17 2010
     frag_max low = 0
     frag_max low_date = Wed Sep  8 09:00:17 2010
     frag_new avg = 0.000
     frag_new high = 0.000
     frag_new high_date = Wed Sep  8 09:00:17 2010
     frag_new low = 0.000
     frag_new low_date = Wed Sep  8 09:00:17 2010
     frag_timeout avg = 0.000
     frag_timeout high = 0
     frag_timeout high_date = Wed Sep  8 09:00:17 2010
     frag_timeout low = 0
     frag_timeout low_date = Wed Sep  8 09:00:17 2010
     kpkts_applayer avg = 121425.178
     kpkts_applayer high = 444882
     kpkts_applayer high_date = Thu Sep  2 22:42:20 2010
     kpkts_applayer low = 5738
     kpkts_applayer low_date = Wed Sep  1 18:55:09 2010
     kpkts_ipfrag avg = 0.000
     kpkts_ipfrag high = 0.000
     kpkts_ipfrag high_date = Wed Sep  8 09:00:17 2010
     kpkts_ipfrag low = 0.000
     kpkts_ipfrag low_date = Wed Sep  8 09:00:17 2010
     kpkts_ipreass avg = 0.022
     kpkts_ipreass high = 0.366
     kpkts_ipreass high_date = Tue Sep  7 06:20:11 2010
     kpkts_ipreass low = 0.000
     kpkts_ipreass low_date = Wed Sep  8 08:31:29 2010
     kpkts_iptcprebuilt avg = 0.273
     kpkts_iptcprebuilt high = 1.646
     kpkts_iptcprebuilt high_date = Thu Sep  2 22:42:20 2010
     kpkts_iptcprebuilt low = 0.006
     kpkts_iptcprebuilt low_date = Tue Sep  7 09:13:56 2010
     kpkts_wire avg = 0.252
     kpkts_wire high = 1.352
     kpkts_wire high_date = Sat Sep  4 07:59:48 2010
     kpkts_wire low = 0.006
     kpkts_wire low_date = Tue Sep  7 09:13:56 2010
     mbits_applayer avg = 803.413
     mbits_applayer high = 1009
     mbits_applayer high_date = Sat Sep  4 08:09:48 2010
     mbits_applayer low = 120
     mbits_applayer low_date = Mon Sep  6 05:52:07 2010
     mbits_ipfrag avg = 2.434
     mbits_ipfrag high = 17.685
     mbits_ipfrag high_date = Tue Sep  7 06:20:11 2010
     mbits_ipfrag low = 0.007
     mbits_ipfrag low_date = Mon Sep  6 17:12:03 2010
     mbits_ipreass avg = 0.000
     mbits_ipreass high = 0.000
     mbits_ipreass high_date = Wed Sep  8 09:00:17 2010
     mbits_ipreass low = 0.000
     mbits_ipreass low_date = Wed Sep  8 09:00:17 2010
     mbits_tcprebuilt avg = 0.482
     mbits_tcprebuilt high = 8.324
     mbits_tcprebuilt high_date = Tue Sep  7 06:20:11 2010
     mbits_tcprebuilt low = 0.000
     mbits_tcprebuilt low_date = Tue Sep  7 01:11:34 2010
     mbps_snort avg = 0.000
     mbps_snort high = 0
     mbps_snort high_date = Wed Sep  8 09:00:17 2010
     mbps_snort low = 0
     mbps_snort low_date = Wed Sep  8 09:00:17 2010
     mbps_wire avg = 1.953
     mbps_wire high = 10.613
     mbps_wire high_date = Sat Sep  4 07:59:48 2010
     mbps_wire low = 0.006
     mbps_wire low_date = Sat Sep  4 07:12:47 2010
     patmatch avg = 320.575
     patmatch high = 556.312
     patmatch high_date = Sun Sep  5 19:37:37 2010
     patmatch low = 2.946
     patmatch low_date = Wed Sep  8 07:11:52 2010
     pktbytes avg = 803.413
     pktbytes high = 1009
     pktbytes high_date = Sat Sep  4 08:09:48 2010
     pktbytes low = 120
     pktbytes low_date = Mon Sep  6 05:52:07 2010
     pkts_blocked avg = 0.229
     pkts_blocked high = 14.322
     pkts_blocked high_date = Sun Sep  5 20:50:12 2010
     pkts_blocked low = 0.109
     pkts_blocked low_date = Sat Sep  4 01:34:34 2010
     pkts_dropped avg = 0.000
     pkts_dropped high = 0
     pkts_dropped high_date = Wed Sep  8 09:00:17 2010
     pkts_dropped low = 0
     pkts_dropped low_date = Wed Sep  8 09:00:17 2010
     pkts_dropped_percentage avg = 0.172
     pkts_dropped_percentage high = 9.096
     pkts_dropped_percentage high_date = Sun Sep  5 20:50:12 2010
     pkts_dropped_percentage low = 0.003
     pkts_dropped_percentage low_date = Wed Sep  1 11:50:25 2010
     pkts_total avg = 2106.252
     pkts_total high = 38320
     pkts_total high_date = Thu Sep  2 22:42:20 2010
     pkts_total low = 0
     pkts_total low_date = Wed Sep  8 08:41:27 2010
     sessions_close avg = 0.000
     sessions_close high = 0.000
     sessions_close high_date = Wed Sep  8 09:00:17 2010
     sessions_close low = 0.000
     sessions_close low_date = Wed Sep  8 09:00:17 2010
     sessions_closed avg = 1024.846
     sessions_closed high = 2980
     sessions_closed high_date = Mon Sep  6 12:37:55 2010
     sessions_closed low = 2
     sessions_closed low_date = Wed Sep  1 11:34:05 2010
     sessions_cur avg = 6671.668
     sessions_cur high = 8173
     sessions_cur high_date = Sun Sep  5 21:10:31 2010
     sessions_cur low = 51
     sessions_cur low_date = Wed Sep  1 11:34:05 2010
     sessions_del avg = 0.177
     sessions_del high = 3.055
     sessions_del high_date = Mon Sep  6 05:52:07 2010
     sessions_del low = 0.000
     sessions_del low_date = Sun Sep  5 19:53:29 2010
     sessions_dropped avg = 0.001
     sessions_dropped high = 0.006
     sessions_dropped high_date = Wed Sep  1 11:50:25 2010
     sessions_dropped low = 0.000
     sessions_dropped low_date = Wed Sep  8 09:00:17 2010
     sessions_est avg = 0.376
     sessions_est high = 11.686
     sessions_est high_date = Sun Sep  5 20:50:12 2010
     sessions_est low = 0.003
     sessions_est low_date = Wed Sep  1 11:50:25 2010
     sessions_init avg = 0.001
     sessions_init high = 0.174
     sessions_init high_date = Tue Sep  7 18:18:34 2010
     sessions_init low = 0.000
     sessions_init low_date = Wed Sep  8 08:46:27 2010
     sessions_max avg = 0.000
     sessions_max high = 0.000
     sessions_max high_date = Wed Sep  8 09:00:17 2010
     sessions_max low = 0.000
     sessions_max low_date = Wed Sep  8 09:00:17 2010
     sessions_midstream avg = 6703.818
     sessions_midstream high = 8175
     sessions_midstream high_date = Sun Sep  5 21:03:29 2010
     sessions_midstream low = 51
     sessions_midstream low_date = Wed Sep  1 11:34:05 2010
     sessions_new avg = 0.165
     sessions_new high = 3.062
     sessions_new high_date = Mon Sep  6 05:52:07 2010
     sessions_new low = 0.016
     sessions_new low_date = Fri Sep  3 20:12:36 2010
     sessions_pruned avg = 579.871
     sessions_pruned high = 953
     sessions_pruned high_date = Sun Sep  5 08:30:47 2010
     sessions_pruned low = 3
     sessions_pruned low_date = Wed Sep  1 11:50:25 2010
     sessions_timedout avg = 5066.950
     sessions_timedout high = 7586
     sessions_timedout high_date = Sun Sep  5 21:22:42 2010
     sessions_timedout low = 31
     sessions_timedout low_date = Wed Sep  1 11:34:05 2010
     sessions_udp_cachedSsns_sec avg = 0.000
     sessions_udp_cachedSsns_sec high = 0
     sessions_udp_cachedSsns_sec high_date = Wed Sep  8 09:00:17 2010
     sessions_udp_cachedSsns_sec low = 0
     sessions_udp_cachedSsns_sec low_date = Wed Sep  8 09:00:17 2010
     sessions_udp_cached_current avg = 0.000
     sessions_udp_cached_current high = 0.000
     sessions_udp_cached_current high_date = Wed Sep  8 09:00:17 2010
     sessions_udp_cached_current low = 0.000
     sessions_udp_cached_current low_date = Wed Sep  8 09:00:17 2010
     sessions_udp_cached_max avg = 0.000
     sessions_udp_cached_max high = 0
     sessions_udp_cached_max high_date = Wed Sep  8 09:00:17 2010
     sessions_udp_cached_max low = 0
     sessions_udp_cached_max low_date = Wed Sep  8 09:00:17 2010
     sessions_udp_cached_sec avg = 0.000
     sessions_udp_cached_sec high = 0
     sessions_udp_cached_sec high_date = Wed Sep  8 09:00:17 2010
     sessions_udp_cached_sec low = 0
     sessions_udp_cached_sec low_date = Wed Sep  8 09:00:17 2010
     stream_fault avg = 13.182
     stream_fault high = 59
     stream_fault high_date = Wed Sep  8 05:04:52 2010
     stream_fault low = 0
     stream_fault low_date = Wed Sep  8 00:51:37 2010
     stream_flush avg = 21.526
     stream_flush high = 365.535
     stream_flush high_date = Tue Sep  7 06:20:11 2010
     stream_flush low = 0.013
     stream_flush low_date = Thu Sep  2 05:44:59 2010
     stream_timeout avg = 239.842
     stream_timeout high = 3578
     stream_timeout high_date = Sun Sep  5 20:50:12 2010
     stream_timeout low = 1
     stream_timeout low_date = Wed Sep  1 11:50:25 2010
     synacks avg = 0.124
     synacks high = 2.771
     synacks high_date = Mon Sep  6 12:42:56 2010
     synacks low = 0.006
     synacks low_date = Sat Sep  4 00:58:27 2010
     syns avg = 0.181
     syns high = 6.072
     syns high_date = Mon Sep  6 05:52:07 2010
     syns low = 0.019
     syns low_date = Fri Sep  3 20:12:36 2010

Snort Performance Stats Tool Info

I have noticed that there are not a ton of tools out there (definitely not many currently maintained) that can be used to parse the information from the snort performance monitor.  As such, I am considering writing one and wanted to see what the interest would be.  If there is indeed interest, then I would also like to know what format(s) and types of information would be most useful to the community.  Of course I know what will be useful to myself, and will likely be writing about that in the near future.  For now, here is some sample output from a quick perl parser that I wrote today.

$ ./pminfo.pl /var/tmp/snortstat

-= Tha Pig Doktah 0.1 Dev =-
Copyright (C) 2010 JJ Cummings

Report Info:
    Processed: /var/tmp/snortstat
    First Entry: Wed Sep  1 11:34:05 2010
    Last Entry: Wed Sep  1 22:27:47 2010
    Time Span: 0 days, 10 hours, 53 minutes and 42 seconds

Wirespeed:
    High: 6.683 Mbits/Sec | Wed Sep  1 12:54:00 2010
    Low: 0.007 Mbits/Sec | Wed Sep  1 18:14:18 2010
    Avg: 0.276 Mbits/Sec
  
% Packet Loss:
    High: 3.817% | Wed Sep  1 20:13:39 2010
    Low: 0.000% | Wed Sep  1 22:22:47 2010
    Avg: 0.095%

Additional Info:
    Avg Pkt Size: 363 bytes
    Avg Syns/Sec: 0.153
    Avg SynAcks/Sec: 0.105
    Avg Alerts/Sec: 0.001
    Avg Current Cached Sessions: 2326

Obviously this is was only as a quick test and does not include all of the important pieces of data.  Please feel free to hit me up in #snort (on freenode),  twitter, email(if'n you knows it), or post a comment here.

Cheers,
JJC