I've tried some experiments with collecting at high-speed off gigabit cards.
Created a test system that generated traffic at varied rates from 10 mbs to 1
gbs. Though the cards themselves reported very little loss (Endace products)
the ability to write the data to disk started failing around 250 mbs. Though I
intend to continue testing in the near future I first see that I need to become
a little smarter on storage and which is the quickest for write-speed.
Understand also that the tests were not done under the most scientific
conditions - still repeated testing showed our breakdown was primarily the
ability to write the data to disk and not the gigabit card's ability to process
the data.
The next question would be, if we COULD write to disk as fast as we collected
the data, at which point do other factors such as the OS, the CPU, the memory,
or the aplication itself become a factor?
The testing I am doing is for a Windows-based application and, as such, I have
not yet looked at the same conditions under *nix. I have some interest in that
area though so may see what I can find out there as well.
I know I have seen other folks discuss these same concerns here in the past. I
seem to recall someone else having found the ability to write to disk a severe
limitation. Mind you that most ID/IPS system do not try to write every packet
to disk - they usually only write the packet or session that that triggered an
event. So, for IDS/IPS sensors any speed limitations tend to deal with how
much "analysis" must be done on the traffic passing by/through the sensor. The
more rules in effect - the less they are able to process. Which is a big
reason why "tuning" of sensors to your network is so important. Normalizing
the sensor to your particular network configuration, addressing scheme, etc.
will reduce a LOT of the false positives. Disabling rules that do not apply to
your network also improves the sensor's time to spend analyzing the traffic you
DO determine is valid.
OK folks ... Flame on! <grin>
Hank
------------------------------------------------------------------------
Test Your IDS
Is your IDS deployed correctly?
Find out quickly and easily by testing it
with real-world attacks from CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
------------------------------------------------------------------------
