I agree - vulnerability assessment tools are only part of the story, and need
to be set correctly to imitate real world attacks, such as timed scans, newer
BIND attacks, port 80 attacks, etc.
It is equally important to know what the tool is actually doing - for example,
does the tool just check the settings for passwords to determine limitation of
attempts? If so, it does not check the several different types of login which
do not check the limiting file...
Also, are you testing the right perspective? from inside or outside the
firewall, the switched section, the IDS/IPS portion, etc? It's amazing how many
network budgets don't allow sufficient sensors to handle even appropriate, much
less complete, placement...
I remember checking a network with a tool which did ping sweeps, and finding
out it was so saturated with traffic that the ping sweeps brought it down in
the middle of business transactions - definitely a vulnerability, but not
possible from outside the firewall. Was my test correct?
Warm Regards,
Dave Druitt
--
CSO
InfoSec Group
703-626-6516
"using words to describe magic is like using a screwdriver to cut roast beef"
-Tom Robbins
"there is a big difference betwen kneeling down and bending over" -Bob Dylan
-------------- Original message from thaywood@karalon.com: --------------
Leea,
Your post raises an interesting topic, how often do users perform an
assessment
of their key security defenses to prove that they perform operationally as
described in the marketing materials, my bet is not that often in reality.
I have worked in the security market space for the last 15 years and during
that
time have seen many end users want to but not really know how to test their
security defenses. You spend a lot of money on these systems, then many times
users put their faith that the product is working as advertised without realy
being able to prove it or having the necessary tools to help.
One regular post to this list is "can I use a vulnerability scanner to test
my
IDS/IPS", the answer is generally no as they are not designed for that
purpose.
There are a number of things that you should really look at when testing an
IDS
or IPS system and one of the most important things is just how useable is it?
If the worst happens and some kind of attack is picked up does the management
console become unusable due to the scale and volume of alerts? (I've seen
many
deployments where a slight burst in activity can make the management system
become a monster and un useable)
How easy is it to spot if a sensor has gone off line? (I've seen many
occasions
when acording to the management console the sensor is working fine and active
but in reality somehow it has "gone to sleep" and is not picking up anything.
There are a number of resources out there to help you
http://www.karalon.com/products.htm
The Tolly Group also published a whitepaper on IPS testing and benchmarking
you
may find intresting.
http://www.tolly.com/ts/2006/TollyEdge/IPS-Wired/TollyWP206115TollyEdgeIPS-Wired
-May2006.pdf
Regards
Tony
Tony Haywood
CTO
www.karalon.com
Audit, Test, Prove & Validate
------------------------------------------------------------------------
Test Your IDS
Is your IDS deployed correctly?
Find out quickly and easily by testing it
with real-world attacks from CORE IMPACT.
Go to
http://www.coresecurity.com/index.php5?module=Form&action=impact&campaign=intro_
sfw
to learn more.
------------------------------------------------------------------------
------------------------------------------------------------------------
Test Your IDS
Is your IDS deployed correctly?
Find out quickly and easily by testing it
with real-world attacks from CORE IMPACT.
Go to
http://www.coresecurity.com/index.php5?module=Form&action=impact&campaign=intro_sfw
to learn more.
------------------------------------------------------------------------
