Hogging the Snort Host Attribute Table

Hogger is a new Snort supportive tool written in Perl, by Parker Crook, that allows you to create a Host Attribute Table from an nmap scan. But first, a little primer; A feature within Snort that has received some traction lately is that of the --enable-targetbased configuration option. This allows you to specify a Host Attribute Table that contains critical information about what your network host topology is (i.e. OS, services etc..). Using this information, snort can then properly reassemble fragments, track streams and a number of other things. All of these items are covered in Joel Esler's recent CSO article that can be found at This URL. This is an excellent article that covers what Host Attribute Tables are and how to use them, so please read the article for a better understanding!

Now that you know all about the Host Attribute Table, let's jump into the purpose and use of hogger. As mentioned previously, hogger was written by Parker Crook to create a Host Attribute Table using the resulting output of an nmap scan. Without further adieu, let's walk through the usage of hogger!

Requirements:

• Perl
• XML::Writer (perl module)
• Nmap
• Hogger

Steps:

1. Install XML::Writer
2. Get hogger
3. Install Nmap
4. Run Nmap with correct options
5. Run hogger against Nmap output file
6. Start your snorting!

1: Installing XML::Writer

$perl -MCPAN -e shell

cpan[1]> install XML::Writer

2: Get Hogger

$wget http://hogger.googlecode.com/files/hogger.tar.gz

$tar xvfz hogger.tar.gz

3: Install Nmap

Use whatever tool that your distribution / OS uses to install Nmap, or get the source from nmap.org and build it yourself!

4: Run Nmap

$mkdir ~/hogger/nmap

$cd ~/hogger/nmap

$nmap -sV -T4 -oN scan.nmap 192.168.1.0/24

Starting Nmap 5.21 ( http://nmap.org ) at 2010-02-25 18:46 UTC

..output suppressed...

5: Run hogger (against scan.nmap)

$cd ~/hogger

$./hogger.pl -c nmap/hostmap.csv -n nmap/scan.nmap -x nmap/host_attrib_table.xml

6: Start your snorting - At this point you can take the newly created host_attrib_table.xml file and place the path to it in your snort.conf, assuming your built snort with the correct option:

attribute_table filename /path/to/host_attrib_table.xml

Now that we have all of this running, let's examine some of the options that are currently available in hogger and dissect our hogger run: "./hogger.pl -c nmap/hostmap.csv -n nmap/scan.nmap -x nmap/host_attrib_table.xml".

Hogger help output:

Usage: ./hogger.pl [-r? -help] -n -c -x

Options:

-c Where the human-readable/modifiable csv file containing host information lives.

-n Where the nmap file containing host information lives.

-r Process the csv file and output to xml for snort, but do not read an nmap file.

-x Where you want to create the host_attribute table.xml (Overwrites existing files)

-help/? Print this information

Starting with the -c flag, this is a file that will be created by hogger if it does not exist, and is simply a csv file that you can modify (for those hosts that nmap either misses or is not as accurate as you would like). A few sample entries in the file (hostmap.csv) that we created in the above test run:

192.168.1.1, Linux, 23|tcp|telnet 53|tcp|domain 443|tcp|ssl/http

192.168.1.2, Linux, 23|tcp|telnet 53|tcp|domain 443|tcp|ssl/http

192.168.1.7, FreeBSD, 22|tcp|ssh 53|tcp|domain 80|tcp|http 3000|tcp|http 3128|tcp|http-proxy 3306|tcp|mysql 5000|tcp|http-proxy 8443|tcp|http

Next we see the -n flag, this is the flag that specifies where the nmap output file (that we previously created using the nmap -oN scan.nmap option). This is the file that hogger reads to create entries in the -c .

The -r flag is fairly straightforward and specifies that you ONLY want to read the csv file specified with the -c flag value.

The final flag that we will discuss is the -x flag, this is a required flag and tells hogger where you want the resulting output (the Host Attribute Table) to be placed. Examples from the output, matching those noted in the -c flag information above:

<SNORT_ATTRIBUTES>

<ATTRIBUTE_TABLE>

<HOST IP="192.168.1.1">

<OPERATING_SYSTEM>

<NAME ATTRIBUTE_VALUE="Linux" CONFIDENCE="90"></NAME>

<FRAG_POLICY>Linux</FRAG_POLICY>

<STREAM_POLICY>linux</STREAM_POLICY>

</OPERATING_SYSTEM>

<SERVICES>

<SERVICE>

<PORT ATTRIBUTE_VALUE=" 23" CONFIDENCE="100"></PORT>

<IPPROTO ATTRIBUTE_VALUE="tcp" CONFIDENCE="100"></IPPROTO>

<PROTOCOL ATTRIBUTE_VALUE="telnet 53" CONFIDENCE="95"></PROTOCOL>

</SERVICE>

</SERVICES>

</HOST>

<HOST IP="192.168.1.2">

<OPERATING_SYSTEM>

<NAME ATTRIBUTE_VALUE="Linux" CONFIDENCE="90"></NAME>

<FRAG_POLICY>Linux</FRAG_POLICY>

<STREAM_POLICY>linux</STREAM_POLICY>

</OPERATING_SYSTEM>

<SERVICES>

<SERVICE>

<PORT ATTRIBUTE_VALUE=" 23" CONFIDENCE="100"></PORT>

<IPPROTO ATTRIBUTE_VALUE="tcp" CONFIDENCE="100"></IPPROTO>

<PROTOCOL ATTRIBUTE_VALUE="telnet 53" CONFIDENCE="95"></PROTOCOL>

</SERVICE>

</SERVICES>

</HOST>

<HOST IP="192.168.1.7">

<OPERATING_SYSTEM>

<NAME ATTRIBUTE_VALUE="FreeBSD" CONFIDENCE="90"></NAME>

<FRAG_POLICY>BSD</FRAG_POLICY>

<STREAM_POLICY>bsd</STREAM_POLICY>

</OPERATING_SYSTEM>

<SERVICES>

<SERVICE>

<PORT ATTRIBUTE_VALUE=" 22" CONFIDENCE="100"></PORT>

<IPPROTO ATTRIBUTE_VALUE="tcp" CONFIDENCE="100"></IPPROTO>

<PROTOCOL ATTRIBUTE_VALUE="ssh 53" CONFIDENCE="95"></PROTOCOL>

</SERVICE>

</SERVICES>

</HOST>

Having said all of this, I am not going to go into detail about the flags used during the Nmap scan, suffice it to say that those are the suggested flags and that the -oN is required to produce the output file for hogger to read.

Overall I think that the concept behind hogger is excellent and that it should provide useful aide to all you snort heads out there! This tool gets a thumbs up from me and should be one that you put into your snort bag of tricks and is also one that I am planning on contributing to.

Cheers,

JJC

pulledpork 0.2.2 and new features

Get it while it's hot @here!

I have received a few requests to build support into pulledpork for the restarting of processes (i.e. snort after downloading new rules or modifying the ruleset using disablesid). In response to this, it is done ^-^. You will note in the pulledpork.conf file that there is a new option at the bottom called pid_path. Simply list the path to your pid files (/var/run/snort_intx.pid,/path/to/another/pid.pid) etc... and specify -H at runtime.. you will be magically pleased (assuming you run pulledpork under a context that has permissions to restart said PID).

I also added a second option "-n" that will allow you to make modifications to the disablesid.conf file and re-execute pulledpork without attempting to download the current ruleset or md5 again (ala tuning exercises...).

Please see the included README for additional info and general guidelines on usage... below is some sample output.

./pulledpork.pl -c ../pulledpork.conf -i disablesid.conf -THn
Prepping files for work....
Done!
Copying rules files....
Done!
Disabling your chosen SID's....
Disabled 1 rules in /usr/local/etc/snort/rules/web-iis.rules
Disabled 2 rules in /usr/local/etc/snort/rules/backdoor.rules
Disabled 1 rules in /usr/local/etc/snort/rules/rpc.rules
Disabled 1 rules in /usr/local/etc/snort/rules/exploit.rules
Done
HangUP Time....
Done!
Fly Piggy Fly!

That's all for now, enjoy!

JJC

Snorby for Snort, a Recipe with Barnyard2 and Unified2

Snorby, an all new frontend (yes, it's still Beta) for snort has recently emerged. As such I decided that I would take a look and give my thoughts as well as a quick recipe to get it running fairly quickly using barnyard2. During my testing of Snorby, I talked with the creator (mephux) about his plans for Snorby and also worked through a couple of bugs, that he jumped on right away.

Note: This posting details how to get Snorby working with apache and passenger, NOT Webrick.. if you want that please read the details of how to do so at the Snorby site.

Recipe Components:

• FreeBSD 8.0R
• apache22
• ruby-gems
• ruby-iconv
  
• prawn (gem)
  
• rake (gem)
  
• mysql (gem)
• rails (gem)
  
• passenger (formerly modrails)
• mysql
• snort
• barnyard2
• git

Ok, let's get the dependencies and such out of the way. I am making several assumptions in writing this... the least of which is that you know how to use google if you can't figure something out... also that you already have the base of some of these items installed (ala, FreeBSD, apache, snort). If not, I have previous posts that discuss the setup of said items, and I am again going to drop the google bomb!

We need ruby-gems to get passenger running and ultimately Snorby:

$ cd /usr/ports/devel/git/ && sudo make install clean
...I deselect all of the options, I just want regular old git for this exercise
...output suppressed
$ cd /usr/ports/devel/ruby-gems/ && sudo make install clean
...output suppressed
$ sudo gem install prawn --no-rdoc --no-ri
...output suppressed
$ sudo gem install rake --no-rdoc --no-ri
...output suppressed
$ sudo gem install rails --no-rdoc --no-ri
...output suppressed
$ sudo gem install mysql --no-rdoc --no-ri
...output suppressed
$ sudo gem install passenger --no-rdoc --no-ri
...output suppressed
$ sudo passenger-install-apache2-module
...run through the setup and perform the steps that are noted to activate the passenger capabilities with apache.. ala vi httpd.conf and add the 3 lines that you are told to.
$ cd /usr/local/www/ && sudo git clone git://github.com/mephux/Snorby.git
...output suppressed/usr/ports/converters/ruby-iconv
$ cd /usr/ports/converters/ruby-iconv && sudo make install clean

At this point you are ready to modify your database and email configuration for Snorby. If you have not done so, you should create a snort database (I have called mine snort and created a user "snorby" with password "snorby".. ok that's not really the password but for this writeup it is! This user has full access (not grant) to the snort database. I have also created the apt tables in this database using the create_mysql sql that is included in both Snorby and Snort!

$ sudo cp /usr/local/www/Snorby/config/database.yml.example /usr/local/www/Snorby/config/database.yml
$ sudo cp /usr/local/www/Snorby/config/email.yml.example /usr/local/www/Snorby/config/email.yml

Now choose your preferred editor and modify the /usr/local/www/Snorby/config/database.yml file.. we are only concerned with the production info... you can also modify the email.yml but don't have to for our current purposes.

Install additional gem requirements and setup Snorby to run!

$ cd /usr/local/www/Snorby && sudo rake gems:install
...output suppressed
$ cd /usr/local/www/Snorby && sudo rake snorby:setup RAILS_ENV=production
...output suppressed

At this point you are ready to tell apache all about Snorby, so lets modify our vhost or apache config again. Simply add the following under the vhost of your choice, you need to be sure that RewriteEngine On and RewriteOptions inherit are specified in this vhost (or in scope of your config):

DocumentRoot /usr/local/www/Snorby/public

RailsBaseURI /

<directory "/usr/local/www/Snorby/public">
AllowOverride All
Order deny,allow
Allow from all
</directory>

Once this is complete, restart apache and you will get the login for Snorby when you browse to that vhost. The default username is snorby and password is admin.

We are now ready to modify our snort config to output unified2, modify your snort.conf and comment out your old output plugins or simply replace them with the following:

output unified2: filename snortunified2.log, limit 128

Note that unified2 contains all log and alert data, so no longer do you need two files! And now it's time for barnyard2. Go ahead and fetch the latest version from securixlive.com, configure with "--with-mysql" option. Once that is done copy the barnyard.conf to /usr/local/etc/snort/ and let's go ahead and edit that file, putting in the mysql information that you used with Snorby earlier and making sure that we have our input specified as unified2. You should go through and make sure that all of the paths to the map and ref files are specified correctly. Once that's done, you are ready to fire it up!

sudo barnyard2 -c /usr/local/etc/snort/barnyard2.conf -d /var/log/snort -f snortunified2.log -w /var/log/snort/barnyard2.waldo -D

You should now be receiving events in the snort mysql database and seeing them in Snorby.

Please note that there are a number of security considerations that I did not take into account (ala running all this stuff under root) so please take that into consideration.

Overall, I give Snorby a good rating, it certainly has lots of eye candy at this point. Mephux promises that much of the functionality that everyone wants is coming shortly... I would say that Snorby has a good start and promises to be a decent usable frontend for viewing snort events. Is it a sguil, certainly not... but it does look like it will be a decent alternative to BASE.

Cheers,
JJC

How to block robots.. before they hit robots.txt - ala: mod_security

As many of you know, robots (in their many forms) can be quite pesky when it comes to crawling your site, indexing things that you don't want indexed. Yes, there is the standard of putting a robots.txt in your webroot, but that is often not highly effective. This is due to a number of facts... the least of which is not that robots tend to be poorly written to begin with and thus simply ignore the robots.txt anyway.

This comes up because a friend of mine that runs a big e-com site recently asked me.. "J, how can I block everything from these robots, I simply don't want them crawling our site." My typical response to this was "you know that you will then block these search engines and keep them from indexing your site"... to whit "yes, none of our sales are organic, they all come from referring partners and affiliate programs".... That's all that I needed to know... as long as it doesn't break anything that they need heh.

After puting some thought into it, and deciding that there was no really easy way to do this on a firewall, I decided that the best way to do it was to create some mod_security rules that looked for known robots and returned a 404 whenever any such monster hit the site. This made the most sense because they are running an Apache reverse proxy in front of their web application servers with mod_security (and some other fun).

A quick search on the internet found the robotstxt.org site that contained a listing (http://www.robotstxt.org/db/all.txt) of quite a few common robots. Looking through this file, all that I really cared about was the robots-useragent value. As such, I quickly whipped up the following perl that automaticaly creates a file named modsecurity_crs_36_all_robots.conf. Simply place this file in the apt path (for me /usr/local/etc/apache/Includes/mod_security2/) and restart your apache... voila.. now only (for the most part) users can browse your webserver. I'll not get into other complex setups, but you could do this on a per directory level also, from your httpd.conf, and mimic robots.txt (except the robots can't ignore the 404 muahahaha).

#####################Begin Perl#######################

#!/usr/bin/perl

##
## Quick little routine to pull the user-agent string out of the
## all.txt file from the robots project, with the intention of creating
## regular expression block rules so that they can no longer crawl
## against the rules!
## Copyright JJ Cummings 2009
## cummingsj@gmail.com
##

use strict;
use warnings;
use File::Path;

my ($line,$orig);
my $c = 1000000;
my $file = "all.txt";
my $write = "modsecurity_crs_36_all_robots.conf";
open (DATA,"<$file");
my @lines = ;
close (DATA);

open (WRITE,">$write");
print WRITE "#\n#\tQuick list of known robots that are parsable via http://www.robotstxt.org/db/all.txt\n";
print WRITE "#\tgenerated by robots.pl written by JJ Cummings \n\n";
foreach $line(@lines){
if ($line=~/robot-useragent:/i){
$line=~s/robot-useragent://;
$line=~s/^\s+//;
$line=~s/\s+$//;
$orig=$line;
$line=~s/\//\\\//g;
#$line=~s/\s/\\ /g;
$line=~s/\./\\\./g;
$line=~s/\!/\\\!/g;
$line=~s/\?/\\\?/g;
$line=~s/\$/\\\$/g;
$line=~s/\+/\\\+/g;
$line=~s/\|/\\\|/g;
$line=~s/\{/\\\{/g;
$line=~s/\}/\\\}/g;
$line=~s/\(/\\\(/g;
$line=~s/\)/\\\)/g;
$line=~s/\*/\\\*/g;
$line=~s/X/\./g;
$line=lc($line);
chomp($line);
if (($line ne "") && ($line !~ "no") && ($line !~ /none/i)) {
$c++;
$orig=~s/'//g;
$orig=~s/`//g;
chomp($orig);
print WRITE "SecRule REQUEST_HEADERS:User-Agent \"$line\" \\\n";
print WRITE "\t\"phase:2,t:none,t:lowercase,deny,log,auditlog,status:404,msg:'Automated Web Crawler Block Activity',id:'$c',tag:'AUTOMATION/BOTS',severity:'2'\"\n";
}
}
}
close (WRITE);
$c=$c-1000000;
print "$c total robots\n";

#####################End Perl#######################

To use the above, you have to save the all.txt file to the same directory as the perl.. and of course have +w permissions so that the perl can create the apt new file. This is a pretty basic routine... I wrote it in about 5 minutes (with a few extra minutes for tweaking of the ruleset format output (displayed below). So please, feel free to modify / enhance / whatever to fit your own needs as best you deem. **yes, I did shrink it so that it would format correctly here**

#####################Begin Example Output#######################

SecRule REQUEST_HEADERS:User-Agent "abcdatos botlink\/1\.0\.2 \(test links\)" \
"phase:2,t:none,t:lowercase,deny,log,auditlog,status:404,msg:'Automated Web Crawler Block Activity',id:'1000001',tag:'AUTOMATION/BOTS',severity:'2'"
SecRule REQUEST_HEADERS:User-Agent "'ahoy\! the homepage finder'" \
"phase:2,t:none,t:lowercase,deny,log,auditlog,status:404,msg:'Automated Web Crawler Block Activity',id:'1000002',tag:'AUTOMATION/BOTS',severity:'2'"
SecRule REQUEST_HEADERS:User-Agent "alkalinebot" \
"phase:2,t:none,t:lowercase,deny,log,auditlog,status:404,msg:'Automated Web Crawler Block Activity',id:'1000003',tag:'AUTOMATION/BOTS',severity:'2'"
SecRule REQUEST_HEADERS:User-Agent "anthillv1\.1" \
"phase:2,t:none,t:lowercase,deny,log,auditlog,status:404,msg:'Automated Web Crawler Block Activity',id:'1000004',tag:'AUTOMATION/BOTS',severity:'2'"
SecRule REQUEST_HEADERS:User-Agent "appie\/1\.1" \
"phase:2,t:none,t:lowercase,deny,log,auditlog,status:404,msg:'Automated Web Crawler Block Activity',id:'1000005',tag:'AUTOMATION/BOTS',severity:'2'"

#####################End Example Output#######################

And that folks, is how you destroy robots that you don't like.. you can modify the error that returns to fit whatever suits you best.. 403, 404.....

Cheers,
JJC

openpacket.org

I recently took over managing and maintaining OpenPacket.org from Richard Bejtlich of TaoSecurity. I would like to extend my thanks to Richard for his time and efforts in getting OpenPacket.org off the ground.

The mission of OpenPacket.org is to provide quality network traffic traces to researchers, analysts, and other members of the digital security community. One of the most difficult problems facing researchers, analysts, and others is understanding traffic carried by networks. At present there is no central repository of traces from which a student of network traffic could draw samples. OpenPacket.org provides one possible solution to this problem.

Analysts looking for network traffic of a particular type can visit OpenPacket.org, query the OpenPacket.org capture repo for matching traces, and download those packets in their original format (e.g., Libpcap, etc.). The analyst will be able to process and analyze that traffic using tools of their choice, like Tcpdump, Snort, Ethereal, and so on.

Analysts who collect their own traffic will be able to submit it to the OpenPacket.org database after they register.

Anonymous users can download any trace that's published. Only registered users can upload. This system provides a level of accountability for trace uploads.

Our moderators will review the trace to ensure it does not contain any sensitive information that should not be posted publicly. Besides appearing on the site, once a trace has been published you can receive notice of it via this published trace RSS feed.

If you have any doubt regarding the publication of a trace, do not try to submit it. When moderators are unsure of the nature of a trace, we will reject it. OpenPacket.org is not a vehicle for publishing enterprise data as contained in network traffic.

In the upcoming months you will see significant changes and improvements to the OpenPacket.org site. Many of these suggestions are the result of user feedback, so please keep it coming and stay tuned as updates are released!

JJC

"Block the Bad" OSS IPS with Content Filtration and Transparent Proxy Acceleration pt 1.

In this two part series I will discuss and demonstrate the creation of an inline security and content filtration system built on FreeBSD 7.0R. What is a security and content filtration system you might ask? Simply put it is a system that has the capabilities of an IPS with the included benefit of advanced content filtration (things like blacklists, page content scoring "keywords etc", greylists, whitelists and so on...).

This first part, entitled "block the bad" will deal with the IPS aspect of the system that includes some new "or newly revisited" ways of utilizing snortsam with barnyard rather than directly patching snort. This is good for a variety of reasons that include the capability to keep your snort version updated without having to continually re-patch it for snortsam, and not having to load snort down with more work than what it was intended "SNIFFING J00r PAket F00".

Some things in the below documented barnyard snortsam plugin have been hacked together, and I am sure that more capable individuals "rotorhead, Obiwan..." will write a non-hacked-together plugin in the near future. But this will get you up and rolling for now.

A few assumptions are made before we get started... the first is that you have already built snort (2.8.1 is the latest as of the time I wrote this), and if not that you can follow the directions to do so on a previous posting of mine. The second assumes if you want to see output such as BASE, you read and followed that entire posting. The third assumption is that you know how to modify your kernel options and ultimately make and install a new kernel. The fourth and final assumption is that if any of the previous assumptions are not true, you know how to use google.

Now, to the heart of the subject at hand, we will be using the following for the remainder of the excercise:

1. Snort 2.8.1 (see above)
2. barnyard 0.20.0 (with a modified snortsam plugin)
3. snortsam 2.52
4. ipf
5. ipfw (this will come into play in the next part re: content filtering, but can also be used to block by entire source or destination *not protocol/port* hence ipf)

So, for our first step (since we have snort built/running) let's get our barnyard patched so that we have the snortsam plugin. If you previously built barnyard and still have all of the source, that's great... but remember to make clean before we do anything. For my purposes I'll be demonstrating with a freshly downloaded barnyard. You will need autotools "cd /usr/ports/devel/autotools/ && make install clean" to finish the patch work.

[jj@Azazel /usr/home/jj]$ wget http://www.snort.org/dl/barnyard/barnyard-0.2.0.tar.gz

2008-04-13 18:14:39 (537 KB/s) - `barnyard-0.2.0.tar.gz' saved [161543/161543]

[jj@Azazel /usr/home/jj]$ tar xvfz barnyard-0.2.0.tar.gz
x barnyard-0.2.0/

[jj@Azazel /usr/home/jj]$ wget http://www.snortsam.net/files/barnyard-plugin/barnyard-snortsam-patch.gz

2008-04-13 18:16:37 (148 KB/s) - `barnyard-snortsam-patch.gz' saved [27149/27149]

[jj@Azazel /usr/home/jj]$ gunzip barnyard-snortsam-patch.gz
[jj@Azazel /usr/home/jj]$ cd barnyard-0.2.0
[jj@Azazel /usr/home/jj/barnyard-0.2.0]$ patch -p1 < ../barnyard-snortsam-patch
Hmm... Looks like a unified diff to me...
...
Hunk #1 succeeded at 1.
Hunk #2 succeeded at 33.
Hunk #3 succeeded at 54.
...
done
[jj@Azazel /usr/home/jj/barnyard-0.2.0]$ ./autojunk.sh
configure.in:147: warning: underquoted definition of SN_CHECK_DECL
configure.in:147: run info '(automake)Extending aclocal'
configure.in:147: or see http://sources.redhat.com/automake/automake.html#Extending-aclocal
autoheader-2.61: WARNING: Using auxiliary files such as `acconfig.h', `config.h.bot'
autoheader-2.61: WARNING: and `config.h.top', to define templates for `config.h.in'
autoheader-2.61: WARNING: is deprecated and discouraged.
autoheader-2.61:
autoheader-2.61: WARNING: Using the third argument of `AC_DEFINE' and
autoheader-2.61: WARNING: `AC_DEFINE_UNQUOTED' allows one to define a template without
autoheader-2.61: WARNING: `acconfig.h':
autoheader-2.61:
autoheader-2.61: WARNING: AC_DEFINE([NEED_FUNC_MAIN], 1,
autoheader-2.61: [Define if a function `main' is needed.])
autoheader-2.61:
autoheader-2.61: WARNING: More sophisticated templates can also be produced, see the
autoheader-2.61: WARNING: documentation.
[jj@Azazel /usr/home/jj/barnyard-0.2.0]$

Now that we have the main part of the patch completed we need to make a few quick modifications to "src/output-plugins/op_alert_fwsam.c" so that it handles the barnyard output properly and loads the sid-msg.map file via a hard coded path (line 191). I threw a patch out there so that you don't need to do this manually, located here: http://www.redsphereglobal.com/data/tools/security/patches/barnyard-snortsam-hack.gz.

[jj@Azazel /usr/home/jj]$ wget http://www.redsphereglobal.com/data/tools/security/patches/barnyard-snortsam-hack.gz

2008-04-13 18:52:54 (1.15 MB/s) - `barnyard-snortsam-hack.gz' saved [641/641]

[jj@Azazel /usr/home/jj]$ gunzip barnyard-snortsam-hack.gz
[jj@Azazel /usr/home/jj]$ cd barnyard-0.2.0
[jj@Azazel /usr/home/jj/barnyard-0.2.0]$ patch -p1 < ../barnyard-snortsam-hack Hmm... Looks like a unified diff to me... The text leading up to this was: ...
Patching file src/output-plugins/op_alert_fwsam.c using Plan A...
Hunk #1 succeeded at 188.
Hunk #2 succeeded at 815.
done
[jj@Azazel /usr/home/jj/barnyard-0.2.0]$

This patch or "hack" has assumed that the location of your sid-msg.map is at /usr/local/etc/snort/sid-msg.map if this is not the case, you will need to edit /src/output-plugins/op_alert_fwsam.c around line 191 and specify the correct path. At this point you can configure barnyard and build as you normally would.

[jj@Azazel /usr/home/jj/barnyard-0.2.0]$./configure --enable-mysql
[jj@Azazel /usr/home/jj/barnyard-0.2.0]$make
[jj@Azazel /usr/home/jj/barnyard-0.2.0]$sudo make install

Your barnyard is now ready and we will cover the config file and startup after we get ipf and snortsam up and running.

The next step is to add the following to our Kernel so that we have ipf and ipfw enabled and running by default at boot.

# IPFW support

options IPFIREWALL #Enable IPFW directly in the kernel
options IPFIREWALL_FORWARD #Enable the Ip Forwarding function of IPFW
options IPFIREWALL_VERBOSE
options IPFIREWALL_DEFAULT_TO_ACCEPT
options IPDIVERT #allow this host to divert packets to/through different ints and routes

# IPF Support - default is to accept
options IPFILTER
options IPFILTER_LOG

Once these have been added please build your kernel, install and reboot. At this point we are ready to fetch and make snortsam.

[jj@Azazel /usr/home/jj]$ wget http://www.snortsam.net/files/snortsam/snortsam-src-2.52.tar.gz

2008-04-13 19:17:28 (497 KB/s) - `snortsam-src-2.52.tar.gz' saved [1075606/1075606]

[jj@Azazel /usr/home/jj]$ tar xvfz snortsam-src-2.52.tar.gz
x snortsam
[jj@Azazel /usr/home/jj]$ cd snortsam
[jj@Azazel /usr/home/jj/snortsam]$ sh ./makesnortsam.sh
-------------------------------------------------------------------------------
Building SnortSam (release)
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Building SnortSam (debug)
-------------------------------------------------------------------------------
Done.
[jj@Azazel /usr/home/jj/snortsam]$sudo cp snortsam* /usr/local/bin/

That's it for the snortsam build, now we are ready to configure everything and fire it up for a test! The first thing that we will configure is our snortsam. There is a good amount of documentation under snortsam/docs/README.conf that covers basic configuration. For our purposes we will create the file /etc/snortsam.conf and place the following in it.

defaultkey secrets
port 6783
accept 192.168.1.0/24
keyinterval 30 minutes
ipf bge0

This configuration specifies a default key of "secrets" and that the snortsam daemon should listen on port 6783 for connectoins from the 192.168.1.0/24 network. The configuration also specifies that the connection between the client (barnyard) and snortsam daemon will be rekeyed every 30 minutes and that ipf will be used on bge0 locally.

On to the barnyard configuration, this file will be barnyard-snortsam.conf located at /usr/local/etc/. The only line that needs to be in this file is the one that calls the snortsam plugin for barnyard and specifies the host:port/password

output alert_fwsam: 192.168.1.7:6783/secrets

The barnyard snortsam plugin uses a sid-block.map file to define what sids will be blocked, how they will be blocked and for how long they will be blocked. The format is quite simple "sid: where[option],duration;" and to test we will put the file at /usr/local/etc/snort/sid-block.map with the following entry

9999999: src[conn], 15 seconds;

I chose sid 9999999 so that I could create a custom rule in my local.rules to test my configuration.

alert icmp any any -> 1.2.3.4 any (msg:"test"; sid:9999999;)

Assuming you were able to add that rule, we are now at the point to fire things up and give it a good old fashioned roll (all in debugging verbose mode of course)!

Restart your snort so that it sees the new SID if you have not done so... -HUP FTW!@!!
Start snortsam (must be as root right now to have access to ipf)

[jj@Azazel /usr/home/jj]$ sudo snortsam-debug

Start barnyard with the new config file (even if you have a previosly running barnyard from the previous security appliance article... this will run at the same time, we have specified a new waldo file and pid file). Note that the following is ALL ONE LINE... no line breaks or crs! Note that this uses the snort.alert and not the snort.log just like the syslog facility.

[jj@Azazel /usr/home/jj]$ sudo /usr/local/bin/barnyard -c /usr/local/etc/barnyard-snortsam.conf -g /usr/local/etc/snort/gen-msg.map -s /usr/local/etc/snort/sid-msg.map -d /var/log/snort/ -f snort.alert -w /var/log/snort/barnyard-snortsam.waldo -p /usr/local/etc/snort/classification.config -X /var/barnyard-snortsam.pid -vvv

After starting barnyard you should see the following debug output from your snortsam-debug:

Debug: Connection from: 192.168.1.7.
Debug: Received Packet: CHECKIN
Debug: Snort SeqNo: cbb9
Debug: Mgmt SeqNo : 7000
Debug: Status : 1
Debug: Version : 14

Now that everything is up and running we can test. The best way to test all aspects is to point a separate system at the IP of this box (default router/gateway) or on my system as evident by the above config "192.168.1.7" and ping 1.2.3.4 with that separate system. The ipfw options that we previously set in the kernel will allow this host to simply route the traffic to the proper destination. You should see debug output from your snortsam-debug as such:

Blocking host 192.168.1.43 in connection 192.168.1.43->1.2.3.4:0 (icmp) for 60 seconds (Sig_ID: 9999999).
Debug: [ipf][28201600] Plugin Blocking...
Debug: [ipf][28201600] command /bin/echo "@1 block in log level local7.info quick on bge0 proto 1 from 192.168.1.43/32 to 1.2.3.4/32"|/sbin/ipf -f -

We can see from the output that it is blocking the source address of 192.168.1.43 and proto 1 (ICMP) only. This means that this host can still browse the internet and do everything (other than send icmp to 1.2.3.4 for 60 seconds), this is a function of the [conn] option in the sid-block.map file.

Wonderful, we now have a functioning version of snortsam running off of the snort output and not snort directly. This means that we can upgrade / change our snort instance itself and not have to re-patch and mess with that... (this of course assumes that the version you use can output unified so that your patched version of barnyard can read it). The final step in this process is to add the sids that you want to block to the sid-msg.map file. I have modified the create-sidmap.pl file to create a sid-block.map compatible output by reading all of the .rules files in a directory and dumping "sid: src[conn], 30min;" output. This output blocks the service by source that the alert was generated from for 30 minutes. The file can be obtained at http://www.redsphereglobal.com/data/tools/security/patches/create-sidblock.pl.gz. Usage is simple and as follows (again, note that it's one line):

[root@Azazel /home/jj]# ./create-sidblock.pl /usr/local/etc/snort/rules/ > /usr/local/etc/snort/sid-block.map
[root@Azazel /home/jj]# tail -n 3 /usr/local/etc/snort/sid-block.map
2500000: src[conn],30min;
2510000: src[conn],30min;
9999999: src[conn],30min;

I suggest that you not put ALL sids in this file, but rather take a subset from rules files that you know are bad news. To do this simply copy the .rules files into a directory of your choice and run the script against that directory (note that the sid-block.map must always live in /usr/local/etc/snort at this time). Other suggestions include daemonizing your barnyard instance (-D) rather than -vvv. The rest you can figure out.

The next part of this series will cover adding content filtration and a transparent squid instance into the mix on this box.

Cheers,
JJC

Cisco Acquires Sguil!

In many of my past writings I have mentioned using Sguil and have been an avid user of the solution. On that front, I would like to extend my congratulations to the core members of the team for their great success! It will be exciting to see it running on IOS!

Cisco Announces Agreement to Acquire Sguil™ Open Source Security Monitoring Project

Acquisition Furthers Cisco’s Vision for Integrated Security Products

SAN JOSE, Calif., and LONGMONT, Color., April 1st, 2008 – Cisco and the Sguil™ project today announced an agreement for Cisco to acquire the Sguil™ project, a leading Open Source network security solution. With hundreds of installations world-wide, Sguil™ is the de facto reference implementation for the Network Security Monitoring (NSM) model. Sguil™-based NSM will enable Cisco’s customer base to more efficiently collect and analyze security-related information as it traverses their enterprise networks. This acquisition will help Cisco to cement its reputation as a leader in the Open Source movement while at the same time furthering its long-held vision of integrating security into the network infrastructure.

Under terms of the transaction, Cisco has acquired the Sguil™ project and related trademarks, as well as the copyrights held by the five principal members of the Sguil™ team, including project founder Robert "Bamm" Visscher. Cisco will assume control of the open source Sguil™ project including the Sguil.net domain, web site and web site content and the Sguil™ Sourceforge project page. In addition, the Sguil™ team will remain dedicated to the project as Cisco employees, continuing their management of the project on a day-to-day basis.

To date, Sguil™ has been developed primarily in the Tcl scripting language, support for which is already present inside many of Cisco’s routers and switches. The new product, to be known as “Cisco Embedded Monitoring Solution (CEMS)”, will be made available first in Cisco’s carrier-grade products in 3Q08, with support being phased into the rest of the Cisco product line by 4Q09. Linksys-branded device will follow thereafter, though the exact deployment schedule has yet to be announced.

“We’re extremely pleased to announce this deal,” said Cisco’s Chief Security Product Manager Cletus F. Simmons. “For some time, our customers have told us that our existing security monitoring products did not extend far enough into their network infrastructure layer. Not only was it sometimes difficult to intercept and monitor the traffic, but there were often political problems at the customer site with deploying our Intrusion Detection Systems, as management had heard several years ago that they ere ‘dead’. Now, with Sguil™ integrated into all their network devices, they’ll have no choice!”

Although the financial details of the agreement have not been announced, Sguil™ developer Robert Visscher will become the new VP of Cisco Rapid Analysis Products for Security. “This deal means a lot to the Sguil™ project and to me personally,” Visscher explains. “Previously, we had to be content with simply being the best technical solution to enable intrusion analysts to collect and analyze large amounts of data in an extraordinarily efficient manner. But now, we’ll have the additional advantage of the world’s largest manufacturer of networking gear shoving it down their customers’ throats! We will no longer have to concern ourselves with mere technical excellence. Instead, I can worry more about which tropical island to visit next, and which flavor daiquiri to order. You know, the important things.”

About Cisco Systems

Cisco, (NASDAQ: CSCO), is the worldwide leader in networking that transforms how people connect, communicate and collaborate. Information about Cisco can be found at http://www.cisco.com. For ongoing news, please go to http://newsroom.cisco.com.

About Sguil™

Sguil™ is the leading Network Security Monitoring (NSM) framework. It is built for network security analysts by network security analysts. Sguil’s main component is an intuitive GUI that provides access to a wide variety of security related information, including real-time IDS alerts, network session database and full packet captures. Sguil™ was written by Robert “Bamm” Visscher, who was apparently too cheap to buy a book on Java or C.

Again, congrats to the team... if you get a chance, please stop in at #snort-gui on freenode and say hi / congratulate the team.

Cheers,
JJC

HeX 1.0.3 LiveUSB Final (Bug Fixes)

I just finished the bugfix version of the HeX 1.0.3 Live (CNY Release) image.

You can get it (in torrent form) from the Security Torrent Depot at http://www.redsphereglobal.com:88/torrent.html?info_hash=77f31dbc8d641500530760e62f17d1a08e433b96 or you can get it from the below direct download site.

USA Site
MD5 (HeX-i386-1.0.3-final-usb.img.gz) = 5fb1498b3437fada0b38602324d8f5e0

Usage instructions are simple:

dd if=/path/to/HeX-i386-1.0.3-final-usb.img of=/path/to/usbstick/device bs=1M

Look for the new HeX 2.0 to be out soon, all based on FreeBSD 7.0R!

Note that some usb sticks will be smaller than others (even if it's "2G") and that even if you write it and dd produces an error saying that not enough space is available... this is OK and your HeX LiveUSB will still work fine.

Cheers,
JJC

Security Torrents

To fill the need to host and download multiple large security related torrents, I have put a tracker online at http://www.redsphereglobal.com:88. You will primarily find items on this site in the following categories:

Toolkits
Anything that I or various other contributing members find useful, relevant or fun with respect to security. Current items that will go into this category are the various HeX (all) releases and InProtect LiveUSB releases.

Distros
Any custom distributions that have been designed to fit security needs and/or perform specific tasks.

Packet-Captures
Any large packet captures or trace files that are obviously not going to fit on the www.openpacket.org site. There is one up there now, it is the malicious traffic that Richard Bejlich captured at the 2007 Shmoocon. This torrent was created and added by giovani...so a shout out goes to him!

Having said all of that, we will (as with all trackers) need seeders. So if you have a little extra bandwidth and/or want to contribute in any way please let us know!

Cheers,
JJC

InProtect LiveUSB 0.80.3 Beta!

Though the InProtect project has not made a large number of public postings lately (beta releases and the like...) we have been quite busy. We will soon be releasing a tarball of the latest 0.80.3RC1. That is not, however, the purpose of this article but rather I am releasing a liveUSB image that is an entirely self-contained and functioning installation of InProtect on a FreeBSD 6.3-Current system.

I came up with the idea to create the InProtect LiveUSB when someone requested that I build one for another project that I am an active member of (HeX). Unfortunately it has taken me several months to get the time put together to actually build this tool. Having said that, I am quite pleased with the outcome and functionality of the tool. Placing this tool onto a USB thumb drive gives the user extreme versatility from the perspective of security. Obviously the nature of a USB thumb drive is not terribly secure; we can put them in our pocket and have them fall out in a parking lot where anyone could conceivably pick it up and snag the data off of it and multiple other scenarios. I am more talking about the security of the location or client that may have a sensitive environment with sensitive data and the like. In this scenario the USB device could be taken in and left with the organization, post scan, that has such sensitive data. Again though, the primary purpose of this build is to allow for a solid demo of the InProtect system.

As I said earlier, the system was built using FreeBSD 6.3-Current, ontop of this I built fluxbox (and several applications such as firefox), mysql51, apache22, php5 and several perl modules that are InProtect dependencies. I manually configured all of the components to work with InProtect, the installer currently does not work on freebsd though I am in the process of building a port. In-short, and as stated earlier, this is a fully functional InProtect scanner with a few things that need to be completed by the end-user; Nessus 3.0.x install and jpgraph for php5 install.

The Nessus and jpgraph items are not included in this image due to their licensing restrictions (not GPL). It is for this reason they must be manually installed.

First you will need to download the InProtect LiveUSB 0.80.3 image here:

http://www.redsphereglobal.com/data/tools/security/live/inprotect-i386-0.80.3-beta.usb.img.gz
MD5 (inprotect-i386-0.80.3-beta.usb.img.gz) = 605a5b20d754ea7e6305922695f301ba
SHA256 (inprotect-i386-0.80.3-beta.usb.img.gz) = 1d562d17db0ef4e3afefcca18fd40932b7faecdddd673910c3ad11a4aab4434b

After obtaining the image and gunzipping it you will want to use dd to write it to a 2G or larger USB thumb drive. NOTE that you want to write it to the device itself and NOT to a specific partition on the device. Also, if you didn't figure it out... this will overwrite anything that you may currently have on your thumb drive.

dd if=/path/to/foo/inprotect-i386-0.80.3-beta.usb.img of=/dev/da0 bs=1M

Your output file path may be different than /dev/da0 (this is mine on a freebsd boxen). The key is that you are writing directly to the device address and NOT to a partition, that will NOT work. Assuming that you have a thumb drive and computer capable of USB2.0 this process should take around 10 minutes to write all of the data.

At this point you should be able to boot from your new shiny LiveUSB thumbdrive. The initial login details are simple (these ARE case sensitive so pay attention!):

Username: InProtect
Password: inprotect

Once logged in type startx to get into fluxbox. From here, if you are not familiar suggest playing around just a little bit. A few tips, this isn't windoze, you access the main menuwith fluxbox, I by right clicking anywhere on the desktop. The image to the right shows the menu of the InProtect LiveUSB. The highlighted option will take you to the Nessus and jpgraph installation instructions.

Even before you install Nessus or jpgraph you will be able to login to the local instance of InProtect by selecting the InProtect menu option as displayed below. Once you have selected the InProtect menu item, you will be able to use admin / admin for the login and password to access the local instance of InProtect.

Note that until you install Nessus you will not be able to run any scans.

In this image I have already created a default scan zone and default scanner so that once Nessus is installed and the Nessus user created, as noted in the instructions contained on the image, the system is fully functional and scans can be immediately created and executed.

As always please feel free to contact me or leave any comments, criticisms, suggestions or otherwise that you might have.

Cheers,
JJC

HeX 1.0.3 LiveUSB (CNY Release)

After much adeau, here it is! Instructions for usage are quite simple, dd it to your usb thumb drive (the drive, not a partition or it will NOT work). This image includes all of the same features as our mainline HeX 1.0.3 release but is on USB not CD, the filesystem is therefore also writable. You will need a minimum of a 2G Thumb Drive or Memory Stick to write this. I say "Memory Stick" because I have heard rumor of some people using SD rather than USB Thumb Drives to use this tool.

So for example on my freebsd system I would dd as follows:

dd if=/path/to/foo/hex-i386-1.0.3.usb.img of=/dev/da0 bs=1M

command is simple... if is the Input File, output is the Output File (in this case it is the da0 device) and bs=1M is setting the block size to 1mb - this helps to speed up the write process.

Downloads:
USA Site (521MB)
USA MD5 Verification
USA SHA256 Verification

Malaysia Mirrors to be populated soon, I'll post them when they are.

Cheers,
JJC

HeX 1.0.3, the CNY Release

I am pleased to announce the release of HeX 1.0.3, release info is below. Thanks to the entire development team for their dedication and hard work. This release has been dubbed the CNY, or Chinese New Year release.

With the recent release of FreeBSD 7.0 RC2, we anticipate an actual 7.0 release in the near future. When the Release version of 7.0 becomes available we will begin working on the new HeX 2.0 project.

Get HeX 1.0.3 Here:
US Mirrors:
https://secure.redsphereglobal.com/data/tools/security/live/hex-i386-1.0.3.iso
https://secure.redsphereglobal.com/data/tools/security/live/hex-i386-1.0.3.iso.md5
https://secure.redsphereglobal.com/data/tools/security/live/hex-i386-1.0.3.iso.sha256

Malaysia Mirrors:
http://bsd.ipv6.la/hex-i386-1.0.3.iso
http://bsd.ipv6.la/hex-i386-1.0.3.iso.md5
http://bsd.ipv6.la/hex-i386-1.0.3.iso.sha256

Fixed:
- pkg_info works after installation
- ping works without sudo
- procfs is correctly mounted on /proc at boot

Upgraded:
1. NSM Console 0.6-DEVEL
Features:
- 'dump' command added, you can now dump packet payloads into a binary
file for later analysis
- Significant speedups in the harimau module and 'checkip' command if
wget is installed
- tcpxtract configuration file changed to extract more types of files
- Added foremost module
- Added clamscan module (Thanks JohnQPublic)
- Argus and tcptrace have reverse dns turned off by default now, it
was causing the module to hang for extremely large pcap files. Can be
switched on by changed the module options
- rot13 encoding and decoding added :)
Bugfixes:
- alias command
- urlescape (en|de)coding
- file existence check
- many other things
All the other enhancements, bugfixes and additions since the 0.2
release (there have been many!)

New Application Packages:
- xplot
- uni2ascii
- vnc
- vsftpd
- samplicator
- sflowtool
- pmacct
- ming
- ploticus
- tcpick
- bvi
- elinks
- feh
- tftpgrab
- arpwatch

Misc:
- New wallpapers with different color schemes

The LiveUSB image will be out shortly, it is undergoing a quick regression test currently.

Cheers,
JJC

HeX and NSM-Console Writeup in ISSA Journal

Russ McRee has written a nice piece about the HeX Live project and the included NSM-Console in his 'toolsmith' section of the ISSA Journal. This 3.5 page writeup has clearly captured our intent behind HeX and the NSM-Console created by Mathew Lee Hinman.

If you are not an ISSA subscriber, you can access the writeup at Russ McRee's column or here in the form of pdf.

I would like to thank the community for their continued support and feedback on this project.

Cheers,
JJC

How do I know if my Snort implementation is working?

How do I test Snort? How do I know if Snort is sniffing packets? How do I know if Snort is running properly? How do I generate a test alert with Snort? Recently, and over the years, I have regularly seen people join the #snort channel on freenode and post these very questions to the snort mailing lists. Perhaps this little article will index properly in the search engines and end their questions, this is of course assuming that they know how to use a search engine ;-).

There are really several ways of testing snort, some much more complex than others. Probably the most simple way is to define a custom rule that you can easily produce the traffic to trigger the alert. This can be done by creating a simple rule that looks for traffic of a certain type, to a certain address or many other ways but for the purposes of this article we will be looking for traffic to a certain address (as this tends to be the most easily produced). We begin by creating a custom rule either in a new rules file or by adding the rule into an existing rules file. To simplify this you can download the rule from the url below:

https://secure.redsphereglobal.com/data/tools/security/snort/rules/snort-test.rules

Once you have downloaded this rule file and added it to your snort.conf so that Snort has loaded it, simply generate traffic from the monitored network to one or more of the following hosts: 121.175.169.102,193.71.199.6,200.123.165.130. This traffic can be of almost any type. I will typically browse via browser or telnet to a standard IRC port (at the time that I wrote this, these hosts were on the known C&C list) such as 6666, 6667 ....

Once this is done you will see the alerts being generated by snort (assuming that everything is configured properly).

As a second method, you can attempt to generate traffic that an existing snort rule can detect and alert on. To do this, I suggest using a tool such as Metasploit to generate actual attack traffic. You will want to test it against a host that you own, I certainly am not advocating attacking someones network with Metasploit from your network, this host should either be intended to be a test host, and/ or be immune to the attack. A simple example would be to enable the web-iis.rules from snort.org and launch an attack against one of your patched webservers from metasploit in an attempt to exploit MS01-23 using the Metasploit Framework Exploit. This will in-turn generate the WEB-IIS ISAPI .printer access alert to fire.

Either of those two methods should allow you to test your Snort installation, there are some other tcpreplay type tools that you can generate traffic from some signatures with, but by and large they are not effective tests.

Regards,
JJC

HeX Virtual Appliance Image: 1.0.2R

While I have not yet had time to create images for multiple Virtualization technologies, I did finish the image for VMware. Please obtain it at the below URL.

This image is 825M in size and will decompress to a 3G VM.

https://secure.redsphereglobal.com/data/tools/security/live/HeX_1.0.2_VMware.tar.gz
https://secure.redsphereglobal.com/data/tools/security/live/HeX_1.0.2_VMware.tar.gz.md5
https://secure.redsphereglobal.com/data/tools/security/live/HeX_1.0.2_VMware.tar.gz.sha256

Enjoy,
JJC

HeX 1.0.2 LiveUSB Update

Unfortunately, the previous HeX 1.0.2 LiveUSB image was not the proper release, thanks to those that pointed this out. This has since been remediated, the original links are still valid, I will re-post here for your downloading pleasure.

For additional information on the project, please read my earlier post at: http://global-security.blogspot.com/2008/01/hex-102r-liveusb-release.html

https://secure.redsphereglobal.com/data/tools/security/live/HeX-i386-1.0.2.img.gz
https://secure.redsphereglobal.com/data/tools/security/live/HeX-i386-1.0.2.img.gz.md5
https://secure.redsphereglobal.com/data/tools/security/live/HeX-i386-1.0.2.img.gz.sha256

Cheers,
JJC

HeX 1.0.2R LiveUSB Release

As I have been away on holiday, I have been unable to release the next iteration of the HeX LiveUSB tool. Let this post serve to remediate that (albeit a bit late). Without further adeau, the download is located at the following URLs:

https://secure.redsphereglobal.com/data/tools/security/live/HeX-i386-1.0.2.img.gz
https://secure.redsphereglobal.com/data/tools/security/live/HeX-i386-1.0.2.img.gz.md5
https://secure.redsphereglobal.com/data/tools/security/live/HeX-i386-1.0.2.img.gz.sha256

For those that are not familiar with the HeX project, please read further at rawpacket.org. The LiveUSB project is a subset of the overall HeX project and adds a bit of functionality to suit your portable packet monkeying needs. Essentially it gives you a slightly larger (and writable) filesystem to do with what you please; i.e. update signatures, modify configurations, store data and the like.

To use the LiveUSB; simply download decompress and dd onto your device (example on fbsd: dd if=/path/to/HeX-i386-1.0.2.img of=/dev/da0 bs=1M). Note that for speed purposes it is important to increase your default block size in fbsd, the value of 1M takes about 200 seconds for my system to write the entire 2G image.

This release contains the NSM Console as described below.

Matthew(Dakrone) is the main developer of NSM Console, here’s the short description about it -

NSM Console (Network Security Monitoring Console) is a framework for performing analysis on packet capture files. It implements a modular structure to allow for an analyst to quickly write modules of their own without any programming language experience which means you can quickly integrate all the other NSM based tools to it. Using these modules a large amount of pcap analysis can be performed quickly using a set of global (as well as per-module) options. NSM Console also aims to be simple to run and easy to understand without lots of learning time.

If you want more information about what it is (and what it does), check out this introductory post -

http://thnetos.wordpress.com/2007/11/27/nsm-console-a-framework-for-running-things/

You can access NSM Console by clicking the menu -> NSM-Tools -> NSM Console

There are also several bug fixes in this release, as well as new nifty wallpapers (for the holiday season hah).

http://www.rawpacket.org/projects/hex/artwork

1. unicornscan run time error
2. svn run time error
3. lsof run time error
4. firefox startup issue
5. pidgin and liferea dbus issue
6. CDROM-Mount.sh syntax error
7. script command issue
8. ping setuid issue

Other known major or minor issues in the Base System are fixed, thanks to chfl4gs_.

Cheers,
JJC

InProtect Update...

And a few operational notes....

We are working hard to get out the next RC for your scanning pleasure. In the meantime, please continue the use and bug reporting, it's been great thus far!

Now, as to a big bug and how to properly handle it. In previous versions of InProtect you were able to control the number of scans with the max_scans value in the Nessus Servers configuration dialogue. Unfortunately with the modification of the nessus_run.pl script to streamline the scanning process, the max_scans variable does not properly control the actual scans being processed by the scanner. A simple example is as follows;

Lets say you schedule a scan with 60 hosts (IP Addresses) to be scanned and have limited in the Nessus Scanner Max_Scans setting a maximum of 10 concurrent scans on said server. When this scheduled scan starts to run it will start out with 10 scans, once those begin to complete it will immediately say that it's running 20 scans then 30 and so on.

To remediate this issue, you need to do a couple of things...first lets go ahead and kill our sched.pl process so that we can clean up the database (if you still show multiple scans running and none are actually running "ps -auxxx | grep nessus"). Once this is complete, go ahead and look in your Inprotect database under the nessus_scan table for any record with a value of 'R' in the status field ( select * from nessus_scan where status='R';". If you find that you do have records with 'R' as their status, you need to set them as 'C' "UPDATE`inprotect`.`nessus_scan` SET `status` = 'C' WHERE `status` = 'R' ; ", you will also need to reset the current_scans value in the nessus_servers table "UPDATE `inprotect`.`nessus_servers` SET `current_scans` = '0';". After completing these steps you can now start your sched.pl up again. As another note, you may want to set all of the status values to 'C' just to clean up that table, once you restart sched.pl it will clean out all of the 'C' status scans and set their main schedule back to a scheduled status.

Now that we have cleaned up the remnants of the aforementioned bug, lets go ahead and talk about the current workaround. This workaround is fairly straightforward and consists of two simple modifications to your scan profile and your nessus server settings. First, let's get into the InProtect GUI and select Settings -> Nessus Servers -> Edit, at this point we will be modifying the value for Max number of hosts to scan and setting it to an extremely high number such as 10000 or more.


The next part of this workaround is to define the maximum hosts that will be scanned in the actual scan profile. This will tell the nessusd server itself how many scans that it is allowed to run at the same time. Select Settings -> Nessus Scan Profiles -> Edit your existing default profile -> Preferences, under the serverprefs section are the options max_checks and max_hosts. The max_checks value defines the number of test to be run concurrently against a single hosts and the max_hosts defines the maximum concurrent number of hosts that the nessusd server will scan. As you can see by the below image, I have set my default values to 4 checks and 10 hosts.


Regards,
JJC

managing snort rulesets cont...

I need to amend my previous posting about the usage of Oinkmaster to automate and manage your Snort rules. I had added in the simple script a command that updates the sid-msg.map in a fairly unclean way. There is, infact, included within the /contrib of Oinkmaster a nifty little script called create-sidmap.pl. This script reads all of the rules from the rules path that you specify and generates sid-msg.map output that can be redirected into a clean sid-msg.map file.

The location in my original posting that should be changed is highlighted here:

secure2# vi /usr/local/bin/autooinkall.sh
#! /bin/sh
#
# simple script to run oinkmaster and obtain bleeding threat updates
# in addition to the regular snort.org updates
#
/usr/local/bin/oinkmaster -o /usr/local/etc/snort/rules/
/usr/local/bin/oinkmaster -C /usr/local/etc/oinkmaster-bleeding.conf -o /usr/local/etc/snort/rules/
cat /usr/local/etc/snort/rules/bleeding-sid-msg.map >> /usr/local/etc/snort/rules/sid-msg.map
/bin/kill -HUP `cat /var/run/snort_em1.pid`
/bin/kill -HUP `cat /var/run/by.pid`

This should be changed to /path/to/your/create-sidmap.pl /path/to/rules/ > /usr/local/etc/snort/rules/sid-msg.map so that the whole thing looks like the following:

secure2# vi /usr/local/bin/autooinkall.sh
#! /bin/sh
#
# simple script to run oinkmaster and obtain bleeding threat updates
# in addition to the regular snort.org updates
#
/usr/local/bin/oinkmaster -o /usr/local/etc/snort/rules/
/usr/local/bin/oinkmaster -C /usr/local/etc/oinkmaster-bleeding.conf -o /usr/local/etc/snort/rules/
/usr/lobal/bin/create-sidmap.pl /usr/local/etc/snort/rules > /usr/local/etc/snort/rules/sid-msg.map
/bin/kill -HUP `cat /var/run/snort_em1.pid`
/bin/kill -HUP `cat /var/run/by.pid`

Regards,
JJC

HeX 1.0.1R LiveUSB Image

After receiving numerous requests to create a HeX Live USB Key Image, I have completed it. This image includes all of the standard tools that you will find on HeX and is writable; so you can update things (signatures etc), make changes and so on.

To use this tool, simply download it from the below location, decompress it and use dd to place it onto your USB Key. If you are not familiar with the dd syntax it's quite simple really; dd if=/path/to/extracted/hex-i386-1.0.1.usb.img of=/dev/da0 (your USB device). Note, that you should not dd this to a mounted partition, it will not work. You need to dd onto a USB Key that you don't mind losing the data on, because this will overwrite everything on that key. You can create a small partition after the dd (this of course assumes that you know how to do this, leaving the existing partition in-place) and have that to write data to etc...

This image does require a minimum 2G key (actually uses 1.75G), and has no minimum memory requirements (other than standard fbsd and X requirements).

https://secure.redsphereglobal.com/data/tools/security/live/hex-i386-1.0.1.usb.img.gz
http://secure.redsphereglobal.com:8080/data/tools/security/live/hex-i386-1.0.1.usb.img.gz
MD5 (hex-i386-1.0.1.usb.img.gz) = cd7489ba0a2a1fe824d286c72eee6842
SHA256 (hex-i386-1.0.1.usb.img.gz) = ffbb428145e0184d3848e45afee0d10ba41a4d9177688db10befc943dd4058f5

Please test this out and let me know how it works for you, or let the entire team at rawpacket.org know.

Regards,
JJC