I will say the same thing to you I sad to Dave Aitel - you need to read our
methodology more carefully.
The exploits to which you refer by name are our BASELINE exploits - they are
chosen to ensure that EVERYONE has the signature in their database - there are
about 7 of those ONLY - they are used to do some of the basic evasion stuff.
Yes, they are old - yes they are simple - that doesn't matter because they are
not part of the DETECTION test.
If you see an explanatory reference such as (i.e. change GOBBLES to GOOBLES")
then it is just that - an explanation of the TYPE of evasion we are doing there
- it is NOT the extent of the testing. Why would we tell everyone (including
the vendors who will be in future tests) EXACTLY what we are going to do in our
evasion tests? We just provide EXAMPLES. UNDERSTAND the methodology before
making stupid claims.
You don't KNOW what is in our attack library because that is the one part of
the methodology we do not detail. People who get too hung up on this part of
the test don't understand about the realities of IPS devices. You are focussing
ONLY on the detection and nothing else when you dismiss the rest of our test
suite so blithely. As it happens, we have some VERY recent stuff in our library
and it is updated/changed for every edition of the report
We DO, however, understand that IPS is not IDS - I am not sure you do....
Come up with better tools than fragroute and Whisker/Nikto for testing those
PARTICULAR evasion techniques and we will gladly use them. Once again you seem
to imply that because we say we use "Whisker evasion techniques" in our
methodology that we are "outdated" because we should be using Nikto. Once
again, you do not understand what we are testing.
We do NOT use Nikto/Whisker as a Web scanner - we have better means to test
whether IPS can block stuff like that. We say quite clearly that we use the
Whisker EVASION TECHNIQUES (on exploits of our own, not those in the standard
test databases you get with these tools). Nikto is based on libwhisker - the
same techniques apply. Once again you have focussed on a very small point ("Oh
my GOD! They don't use Nikto!") without attempting to understand what exactly
it is we are testing.
FYI We STILL find devices that cannot do TCP seg reassembly properly - and in
the latest test, one device even failed two of the Whisker test cases. Just
because those tools have been around for a while does not make them irrelevant
- if we did NOT test against such commonly available tools, we would be doing
our readers - and the vendors - a disservice. FYI - we do far more than JUST
fragroute/whisker.... Contrary to what some other misinformed people have
claimed, we even... Horror of horrors.... Do MSRPC evasion testing.... Yes,
really!
I have no problems with people making constructive criticism, but you and Dave
Aitel are quite simply playing stupid public guessing games and getting it
wrong every time
You need to stop making ridiculous claims and get your facts straight before
posting in public forums. I am always happy to answer questions about our
methodologies.
Bob Walder
The NSS Group
On 12/1/05 7:02 pm, "Julius Detritus" <julius.detritus@ifrance.com> wrote:
About NSS tests:
"They're not open tests."
The NSS test methodologies are published in full.
You don't have the details of the tests (not even the "baseline"
signatures).
"They're outdated."
The first IPS test was a year ago and the NSS methodology was brand new.
You're right that it's mostly the same this year, save for some new
exploits, but I would not consider it outdated. I don't know of a more
recent or more comprehensive set of tests for a network IPS.
They are outdated. The most recent exploit tested must be two years old...
They are copy and paste from IDS tests which are far older.
And the whole methodology is not appropriate. IPS are not IDS.
For IDS "false alarms" generated by out of session packets (like the one
snot would raise on snort) are not acceptable as it would confuse
administrators in their research for effective attacks.
In the case of IPS it is different. OK, it was not a real attack but who
cares. The purpose of IPS is to block. Who cares if it blocked attacks out
of session? It was not legitimate anyway.
But to understand that, you need to understand IPS, and to be used to
security operations (devices management, post-mortem audits, forensics
analysis and the like...)
"They largely test for things you don't care about, such as pushing
packets down a wire..."
My experience shows that organizations DO care about the things that NSS
tests for: signature coverage, baseline performance, performance under
load, latency, application response times, anti-evasion capabilities,
stateful operation, management and configuration. I already mentioned my
view about "pushing packets down the wire."
Do you really care about the phf exploit? Or maybe the old sshutupteo from
gobbles? Are you talking about organizations or museums?
Bob Walder from NSS can chime in here, but my understanding is that the NSS
signature coverage tests include many RPC-related exploits and their
variants, run both "in the clear" and with various evasion techniques,
including modified exploit code and RPC fragmentation.
Anti-evasion is Whisker (not nikto, I said whisker) and fragroute 1.2...
Modified exploits are common ones with strings changed (GOBBLES to GOBBLED).
"No scientific test should be non-repeatable"
We've been able to repeat the majority of the NSS tests consistently in our
lab.
So your exploit database must be very old
My 0.02$
Julius
_____________________________________________________________________
Envie de discuter gratuitement avec vos amis ?
Téléchargez Yahoo! Messenger http://yahoo.ifrance.com
--------------------------------------------------------------------------
Test Your IDS
Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
Test Your IDS
Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
--------------------------------------------------------------------------
