Snort 2.9.0 is teh outed, must haz bakon!!

Snort 2.9.0 introduces:

• Feature rich IPS mode including improvements to Stream for inline deployments. Additionally a common active response API is used for all packet responses, including those from Stream, Respond, or React. A new response module, respond3, supports the syntax of both resp & resp2, including strafing for passive deployments. When Snort is deployed inline, a new preprocessor has been added to handle packet normalization to allow Snort to interpret a packet the same way as the receiving host.
• Use of a Data Acquisition API (DAQ) that supports many different packet access methods including libpcap, netfilterq, IPFW, and afpacket. For libpcap, version 1.0 or higher is now required. The DAQ library can be updated independently from Snort and is a separate module that Snort links. See README.daq for details on using Snort and the new DAQ./li>
• Updates to HTTP Inspect to extract and log IP addresses from X-Forward-For and True-Client-IP header fields when Snort generates events on HTTP traffic.
• A new rule option 'byte_extract' that allows extracted values to be used in subsequent rule options for isdataat, byte_test, byte_jump, and content distance/within/depth/offset.
• Updates to SMTP preprocessor to support MIME attachment decoding across multiple packets.
• Ability to "test" drop rules using Inline Test Mode. Snort will indicate a packet would have been dropped in the unified2 or console event log if policy mode was set to inline.
• Two new rule options to support base64 decoding of certain pieces of data and inspection of the base64 data via subsequent rule options.
• Updates to the Snort packet decoders for IPv6 for improvements to anomaly detection.
• Added a new pattern matcher that supports Intel's Quick Assist Technology for improved performance on supported hardware platforms. Visit http://www.intel.com to find out more about Intel Quick Assist. The following document describes Snort's integration with the Quick Assist Technology: http://download.intel.com/embedded/applications/networksecurity/324029.pdf.
• Reference applications for reading unified2 output that handle all unified2 record formats used by Snort.

Snort 2.9.0 is now available at http://www.snort.org/snort-downloads. Please see the Release Notes and ChangeLog for more details.

The Pig Doktah is Born! - A Snort Performance Monitor Interpretation Tool

After receiving a variety of feedback via email, irc, twitter and even some comments on this blog, I have decided to start an updated and maintained project to parse, display, and ultimately make configuration / tuning suggestions based on the snort performance monitor output.  As you may have guessed by now, this tool is called the Pig Doktah and can be found at http://thepigdoktah.googlecode.com.  The current version reads snort performance monitor output and stores in a hash for quick access, sorting etc...

I am still quite interested in what specific metrics people are most interested, so please don't hesitate to hit me up via comment on this blog, irc (freenode - #snort), email, or twitter.

During the development, and for the foreseeable future, I will have the latest version of the code sending output to this location on an hourly basis: http://rootedyour.com/enhanced/pminfo.htm

Sample output:

-= Tha Pig Doktah 0.1 Dev =-
Copyright (C) 2010 JJ Cummings

Report Info:
    Processed: /var/tmp/snortstat
    First Entry: Wed Sep  1 11:34:05 2010
    Last Entry: Wed Sep  8 09:00:17 2010
    Time Span: 6 days, 21 hours, 26 minutes and 12 seconds

Wirespeed:
    High: 10.613 Mbits/Sec | Sat Sep  4 07:59:48 2010
    Low: 0.006 Mbits/Sec | Sat Sep  4 07:12:47 2010
    Avg: 1.953 Mbits/Sec
  
% Packet Loss:
    High: 10.504% | Sat Sep  4 03:00:00 2010
    Low: 0.000% | Wed Sep  8 08:41:27 2010
    Avg: 1.002%

Additional Info:
    Avg Pkt Size: 803.413 bytes
    Avg Syns/Sec: 0.181
    Avg SynAcks/Sec: 0.124
    Avg Alerts/Sec: 0.001
    Avg Current Cached Sessions: 6671.668

Raw Values:
     alerts avg = 0.001
     alerts high = 0.032
     alerts high_date = Wed Sep  1 12:32:57 2010
     alerts low = 0.000
     alerts low_date = Wed Sep  8 09:00:17 2010
     attrib_hosts_current avg = 0.000
     attrib_hosts_current high = 0.000
     attrib_hosts_current high_date = Wed Sep  8 09:00:17 2010
     attrib_hosts_current low = 0.000
     attrib_hosts_current low_date = Wed Sep  8 09:00:17 2010
     attrib_reloads avg = 0.000
     attrib_reloads high = 0
     attrib_reloads high_date = Wed Sep  8 09:00:17 2010
     attrib_reloads low = 0
     attrib_reloads low_date = Wed Sep  8 09:00:17 2010
     bytes_applayer avg = 0.252
     bytes_applayer high = 1.352
     bytes_applayer high_date = Sat Sep  4 07:59:48 2010
     bytes_applayer low = 0.006
     bytes_applayer low_date = Tue Sep  7 09:13:56 2010
     bytes_ipfrag avg = 0.000
     bytes_ipfrag high = 0
     bytes_ipfrag high_date = Wed Sep  8 09:00:17 2010
     bytes_ipfrag low = 0
     bytes_ipfrag low_date = Wed Sep  8 09:00:17 2010
     bytes_ipreass avg = 2279.291
     bytes_ipreass high = 3660
     bytes_ipreass high_date = Thu Sep  2 13:47:36 2010
     bytes_ipreass low = 368
     bytes_ipreass low_date = Thu Sep  2 10:22:15 2010
     bytes_tcprebuilt avg = 892.669
     bytes_tcprebuilt high = 1458
     bytes_tcprebuilt high_date = Sun Sep  5 15:19:06 2010
     bytes_tcprebuilt low = 136
     bytes_tcprebuilt low_date = Sat Sep  4 00:58:27 2010
     cpu1_idle avg = 95.767
     cpu1_idle high = 99.977
     cpu1_idle high_date = Sat Sep  4 00:58:27 2010
     cpu1_idle low = 69.943
     cpu1_idle low_date = Tue Sep  7 06:20:11 2010
     cpu1_sys avg = 0.051
     cpu1_sys high = 0.287
     cpu1_sys high_date = Sat Sep  4 07:59:48 2010
     cpu1_sys low = 0.000
     cpu1_sys low_date = Wed Sep  8 08:07:19 2010
     cpu1_user avg = 4.183
     cpu1_user high = 29.860
     cpu1_user high_date = Tue Sep  7 06:20:11 2010
     cpu1_user low = 0.023
     cpu1_user low_date = Sat Sep  4 00:58:27 2010
     cpu_count avg = 1.000
     cpu_count high = 1
     cpu_count high_date = Wed Sep  8 09:00:17 2010
     cpu_count low = 1
     cpu_count low_date = Wed Sep  8 09:00:17 2010
     drops avg = 1.002
     drops high = 10.504
     drops high_date = Sat Sep  4 03:00:00 2010
     drops low = 0.000
     drops low_date = Wed Sep  8 08:41:27 2010
     filtered_tcp avg = 3790.598
     filtered_tcp high = 45608
     filtered_tcp high_date = Tue Sep  7 09:24:12 2010
     filtered_tcp low = 85
     filtered_tcp low_date = Wed Sep  1 11:50:25 2010
     filtered_udp avg = 3790.598
     filtered_udp high = 45608
     filtered_udp high_date = Tue Sep  7 09:24:12 2010
     filtered_udp low = 85
     filtered_udp low_date = Wed Sep  1 11:50:25 2010
     frag_auto avg = 0.000
     frag_auto high = 0.000
     frag_auto high_date = Wed Sep  8 09:00:17 2010
     frag_auto low = 0.000
     frag_auto low_date = Wed Sep  8 09:00:17 2010
     frag_complete avg = 0.000
     frag_complete high = 0.000
     frag_complete high_date = Wed Sep  8 09:00:17 2010
     frag_complete low = 0.000
     frag_complete low_date = Wed Sep  8 09:00:17 2010
     frag_current avg = 0.000
     frag_current high = 0
     frag_current high_date = Wed Sep  8 09:00:17 2010
     frag_current low = 0
     frag_current low_date = Wed Sep  8 09:00:17 2010
     frag_delete avg = 0.000
     frag_delete high = 0.000
     frag_delete high_date = Wed Sep  8 09:00:17 2010
     frag_delete low = 0.000
     frag_delete low_date = Wed Sep  8 09:00:17 2010
     frag_faults avg = 0.000
     frag_faults high = 0
     frag_faults high_date = Wed Sep  8 09:00:17 2010
     frag_faults low = 0
     frag_faults low_date = Wed Sep  8 09:00:17 2010
     frag_flushes avg = 0.000
     frag_flushes high = 0.000
     frag_flushes high_date = Wed Sep  8 09:00:17 2010
     frag_flushes low = 0.000
     frag_flushes low_date = Wed Sep  8 09:00:17 2010
     frag_insert avg = 0.000
     frag_insert high = 0.000
     frag_insert high_date = Wed Sep  8 09:00:17 2010
     frag_insert low = 0.000
     frag_insert low_date = Wed Sep  8 09:00:17 2010
     frag_max avg = 0.000
     frag_max high = 0
     frag_max high_date = Wed Sep  8 09:00:17 2010
     frag_max low = 0
     frag_max low_date = Wed Sep  8 09:00:17 2010
     frag_new avg = 0.000
     frag_new high = 0.000
     frag_new high_date = Wed Sep  8 09:00:17 2010
     frag_new low = 0.000
     frag_new low_date = Wed Sep  8 09:00:17 2010
     frag_timeout avg = 0.000
     frag_timeout high = 0
     frag_timeout high_date = Wed Sep  8 09:00:17 2010
     frag_timeout low = 0
     frag_timeout low_date = Wed Sep  8 09:00:17 2010
     kpkts_applayer avg = 121425.178
     kpkts_applayer high = 444882
     kpkts_applayer high_date = Thu Sep  2 22:42:20 2010
     kpkts_applayer low = 5738
     kpkts_applayer low_date = Wed Sep  1 18:55:09 2010
     kpkts_ipfrag avg = 0.000
     kpkts_ipfrag high = 0.000
     kpkts_ipfrag high_date = Wed Sep  8 09:00:17 2010
     kpkts_ipfrag low = 0.000
     kpkts_ipfrag low_date = Wed Sep  8 09:00:17 2010
     kpkts_ipreass avg = 0.022
     kpkts_ipreass high = 0.366
     kpkts_ipreass high_date = Tue Sep  7 06:20:11 2010
     kpkts_ipreass low = 0.000
     kpkts_ipreass low_date = Wed Sep  8 08:31:29 2010
     kpkts_iptcprebuilt avg = 0.273
     kpkts_iptcprebuilt high = 1.646
     kpkts_iptcprebuilt high_date = Thu Sep  2 22:42:20 2010
     kpkts_iptcprebuilt low = 0.006
     kpkts_iptcprebuilt low_date = Tue Sep  7 09:13:56 2010
     kpkts_wire avg = 0.252
     kpkts_wire high = 1.352
     kpkts_wire high_date = Sat Sep  4 07:59:48 2010
     kpkts_wire low = 0.006
     kpkts_wire low_date = Tue Sep  7 09:13:56 2010
     mbits_applayer avg = 803.413
     mbits_applayer high = 1009
     mbits_applayer high_date = Sat Sep  4 08:09:48 2010
     mbits_applayer low = 120
     mbits_applayer low_date = Mon Sep  6 05:52:07 2010
     mbits_ipfrag avg = 2.434
     mbits_ipfrag high = 17.685
     mbits_ipfrag high_date = Tue Sep  7 06:20:11 2010
     mbits_ipfrag low = 0.007
     mbits_ipfrag low_date = Mon Sep  6 17:12:03 2010
     mbits_ipreass avg = 0.000
     mbits_ipreass high = 0.000
     mbits_ipreass high_date = Wed Sep  8 09:00:17 2010
     mbits_ipreass low = 0.000
     mbits_ipreass low_date = Wed Sep  8 09:00:17 2010
     mbits_tcprebuilt avg = 0.482
     mbits_tcprebuilt high = 8.324
     mbits_tcprebuilt high_date = Tue Sep  7 06:20:11 2010
     mbits_tcprebuilt low = 0.000
     mbits_tcprebuilt low_date = Tue Sep  7 01:11:34 2010
     mbps_snort avg = 0.000
     mbps_snort high = 0
     mbps_snort high_date = Wed Sep  8 09:00:17 2010
     mbps_snort low = 0
     mbps_snort low_date = Wed Sep  8 09:00:17 2010
     mbps_wire avg = 1.953
     mbps_wire high = 10.613
     mbps_wire high_date = Sat Sep  4 07:59:48 2010
     mbps_wire low = 0.006
     mbps_wire low_date = Sat Sep  4 07:12:47 2010
     patmatch avg = 320.575
     patmatch high = 556.312
     patmatch high_date = Sun Sep  5 19:37:37 2010
     patmatch low = 2.946
     patmatch low_date = Wed Sep  8 07:11:52 2010
     pktbytes avg = 803.413
     pktbytes high = 1009
     pktbytes high_date = Sat Sep  4 08:09:48 2010
     pktbytes low = 120
     pktbytes low_date = Mon Sep  6 05:52:07 2010
     pkts_blocked avg = 0.229
     pkts_blocked high = 14.322
     pkts_blocked high_date = Sun Sep  5 20:50:12 2010
     pkts_blocked low = 0.109
     pkts_blocked low_date = Sat Sep  4 01:34:34 2010
     pkts_dropped avg = 0.000
     pkts_dropped high = 0
     pkts_dropped high_date = Wed Sep  8 09:00:17 2010
     pkts_dropped low = 0
     pkts_dropped low_date = Wed Sep  8 09:00:17 2010
     pkts_dropped_percentage avg = 0.172
     pkts_dropped_percentage high = 9.096
     pkts_dropped_percentage high_date = Sun Sep  5 20:50:12 2010
     pkts_dropped_percentage low = 0.003
     pkts_dropped_percentage low_date = Wed Sep  1 11:50:25 2010
     pkts_total avg = 2106.252
     pkts_total high = 38320
     pkts_total high_date = Thu Sep  2 22:42:20 2010
     pkts_total low = 0
     pkts_total low_date = Wed Sep  8 08:41:27 2010
     sessions_close avg = 0.000
     sessions_close high = 0.000
     sessions_close high_date = Wed Sep  8 09:00:17 2010
     sessions_close low = 0.000
     sessions_close low_date = Wed Sep  8 09:00:17 2010
     sessions_closed avg = 1024.846
     sessions_closed high = 2980
     sessions_closed high_date = Mon Sep  6 12:37:55 2010
     sessions_closed low = 2
     sessions_closed low_date = Wed Sep  1 11:34:05 2010
     sessions_cur avg = 6671.668
     sessions_cur high = 8173
     sessions_cur high_date = Sun Sep  5 21:10:31 2010
     sessions_cur low = 51
     sessions_cur low_date = Wed Sep  1 11:34:05 2010
     sessions_del avg = 0.177
     sessions_del high = 3.055
     sessions_del high_date = Mon Sep  6 05:52:07 2010
     sessions_del low = 0.000
     sessions_del low_date = Sun Sep  5 19:53:29 2010
     sessions_dropped avg = 0.001
     sessions_dropped high = 0.006
     sessions_dropped high_date = Wed Sep  1 11:50:25 2010
     sessions_dropped low = 0.000
     sessions_dropped low_date = Wed Sep  8 09:00:17 2010
     sessions_est avg = 0.376
     sessions_est high = 11.686
     sessions_est high_date = Sun Sep  5 20:50:12 2010
     sessions_est low = 0.003
     sessions_est low_date = Wed Sep  1 11:50:25 2010
     sessions_init avg = 0.001
     sessions_init high = 0.174
     sessions_init high_date = Tue Sep  7 18:18:34 2010
     sessions_init low = 0.000
     sessions_init low_date = Wed Sep  8 08:46:27 2010
     sessions_max avg = 0.000
     sessions_max high = 0.000
     sessions_max high_date = Wed Sep  8 09:00:17 2010
     sessions_max low = 0.000
     sessions_max low_date = Wed Sep  8 09:00:17 2010
     sessions_midstream avg = 6703.818
     sessions_midstream high = 8175
     sessions_midstream high_date = Sun Sep  5 21:03:29 2010
     sessions_midstream low = 51
     sessions_midstream low_date = Wed Sep  1 11:34:05 2010
     sessions_new avg = 0.165
     sessions_new high = 3.062
     sessions_new high_date = Mon Sep  6 05:52:07 2010
     sessions_new low = 0.016
     sessions_new low_date = Fri Sep  3 20:12:36 2010
     sessions_pruned avg = 579.871
     sessions_pruned high = 953
     sessions_pruned high_date = Sun Sep  5 08:30:47 2010
     sessions_pruned low = 3
     sessions_pruned low_date = Wed Sep  1 11:50:25 2010
     sessions_timedout avg = 5066.950
     sessions_timedout high = 7586
     sessions_timedout high_date = Sun Sep  5 21:22:42 2010
     sessions_timedout low = 31
     sessions_timedout low_date = Wed Sep  1 11:34:05 2010
     sessions_udp_cachedSsns_sec avg = 0.000
     sessions_udp_cachedSsns_sec high = 0
     sessions_udp_cachedSsns_sec high_date = Wed Sep  8 09:00:17 2010
     sessions_udp_cachedSsns_sec low = 0
     sessions_udp_cachedSsns_sec low_date = Wed Sep  8 09:00:17 2010
     sessions_udp_cached_current avg = 0.000
     sessions_udp_cached_current high = 0.000
     sessions_udp_cached_current high_date = Wed Sep  8 09:00:17 2010
     sessions_udp_cached_current low = 0.000
     sessions_udp_cached_current low_date = Wed Sep  8 09:00:17 2010
     sessions_udp_cached_max avg = 0.000
     sessions_udp_cached_max high = 0
     sessions_udp_cached_max high_date = Wed Sep  8 09:00:17 2010
     sessions_udp_cached_max low = 0
     sessions_udp_cached_max low_date = Wed Sep  8 09:00:17 2010
     sessions_udp_cached_sec avg = 0.000
     sessions_udp_cached_sec high = 0
     sessions_udp_cached_sec high_date = Wed Sep  8 09:00:17 2010
     sessions_udp_cached_sec low = 0
     sessions_udp_cached_sec low_date = Wed Sep  8 09:00:17 2010
     stream_fault avg = 13.182
     stream_fault high = 59
     stream_fault high_date = Wed Sep  8 05:04:52 2010
     stream_fault low = 0
     stream_fault low_date = Wed Sep  8 00:51:37 2010
     stream_flush avg = 21.526
     stream_flush high = 365.535
     stream_flush high_date = Tue Sep  7 06:20:11 2010
     stream_flush low = 0.013
     stream_flush low_date = Thu Sep  2 05:44:59 2010
     stream_timeout avg = 239.842
     stream_timeout high = 3578
     stream_timeout high_date = Sun Sep  5 20:50:12 2010
     stream_timeout low = 1
     stream_timeout low_date = Wed Sep  1 11:50:25 2010
     synacks avg = 0.124
     synacks high = 2.771
     synacks high_date = Mon Sep  6 12:42:56 2010
     synacks low = 0.006
     synacks low_date = Sat Sep  4 00:58:27 2010
     syns avg = 0.181
     syns high = 6.072
     syns high_date = Mon Sep  6 05:52:07 2010
     syns low = 0.019
     syns low_date = Fri Sep  3 20:12:36 2010

Snort Performance Stats Tool Info

I have noticed that there are not a ton of tools out there (definitely not many currently maintained) that can be used to parse the information from the snort performance monitor.  As such, I am considering writing one and wanted to see what the interest would be.  If there is indeed interest, then I would also like to know what format(s) and types of information would be most useful to the community.  Of course I know what will be useful to myself, and will likely be writing about that in the near future.  For now, here is some sample output from a quick perl parser that I wrote today.

$ ./pminfo.pl /var/tmp/snortstat

-= Tha Pig Doktah 0.1 Dev =-
Copyright (C) 2010 JJ Cummings

Report Info:
    Processed: /var/tmp/snortstat
    First Entry: Wed Sep  1 11:34:05 2010
    Last Entry: Wed Sep  1 22:27:47 2010
    Time Span: 0 days, 10 hours, 53 minutes and 42 seconds

Wirespeed:
    High: 6.683 Mbits/Sec | Wed Sep  1 12:54:00 2010
    Low: 0.007 Mbits/Sec | Wed Sep  1 18:14:18 2010
    Avg: 0.276 Mbits/Sec
  
% Packet Loss:
    High: 3.817% | Wed Sep  1 20:13:39 2010
    Low: 0.000% | Wed Sep  1 22:22:47 2010
    Avg: 0.095%

Additional Info:
    Avg Pkt Size: 363 bytes
    Avg Syns/Sec: 0.153
    Avg SynAcks/Sec: 0.105
    Avg Alerts/Sec: 0.001
    Avg Current Cached Sessions: 2326

Obviously this is was only as a quick test and does not include all of the important pieces of data.  Please feel free to hit me up in #snort (on freenode),  twitter, email(if'n you knows it), or post a comment here.

Cheers,
JJC

PulledPork 0.4.2 501 error when downloading rules

This issue most typically stems from a missing Perl Module that is required to communicate via SSL using LWP::Simple.  This required Perl Module is Crypt::SSLeay and is not included in the LWP::Simple redistributed package from the Ubuntu 8.x repositories, and will typically fail to install via CPAN on many Ubuntu server installations.  As such you simply need to do the following (on Ubuntu, since this is the only place I have seen it broken):

sudo apt-get install libcrypt-ssleay-perl

Of course if you are not running Ubuntu then you will need to use CPAN or find whatever repackaged garbage that your distro is using to distribute this ;-).

One other cause could be that your root certificates are outdated, so if you have the aforementioned PM installed and are still receiving a 501.. this is likely the cause... google how to update your root certificates for your distro!  Again, for the sake of completeness, this is how you do it on Ubuntu:

sudo apt-get install ca-certificates
sudo update-ca-certificates

I have also added this to the PP FAQ.

Cheers,
JJC

PulledPork 0.4.2 - get it while it's hawt!

This release represents a number of significant enhancements and features (all listed below). Probably the most important to note are the changes from a delimeter of | to : when modifying rule state. We also now automatically determine snort version and OS arch. One of the most useful features, IMHO, is the pcre: rule state modification capability.. see the rule modification configs for more details... but let's say that I wanted to disable ALL MSXX rules because I run a strictly *nix environment... simply place something like pcre:MS\d{2}-\d+ into the disablesid.conf and use that file by specifying -i.

As noted below, there are MANY other changes, fixes, and additions so please don't hesitate to ask questions in irc (freenode #pulledpork) or on the mailing list.

get it here -> http://code.google.com/p/pulledpork

v0.4.2

New Features / changes:

• Capability to modify rules by category (See README.CATEGORIES)
• Capability to modify rules using regular expressions (pcre:) - See sid modification configs
• Capability to use regular expressions in specific rule modifications - See sid modification configs
• Changed the | delimiter for cve,bugtraq etc to :
• Added README.CATEGORIES
• Added README.SHAREDOBJECTS
• Follow flowbit chains
• Moved README files to doc
• Automatically determine arch
• Automatically determine Snort Version
• Added some verbiage surrounding HUP vs Restart vs When/where/who and how
• Added support for new snort.org download scheme of http://snort.org/reg-rules...

Bug Fixes:

• Certain rules specific GID values were not being properly parsed by the modifysid sub.
• Bug #20 fixed, ranges are no longer off by +1 additional rule being enabled
• Enhancement request #21, added more descript information to dropsid.conf and to README
• Fixed flaw that caused certain flowbits to not be set (when GID boundaries were crossed and multiple keys were checked)
• Enhancement request #22 updated the master config file to contain all of the currently available precompiled SO rules
• Remove risky system calls, use handles instead

pulledpork-0.4.2.tar.gz latest hashes:
MD5SUM = d11b9d884f940a0df293718a4d4b3913
SHA256 = 3491b8c3c99c621cfd6467da2c43866f33ede1d096538e4a497cdf52b49ad677

Cheers,
JJC