In-Reply-To: <0501121745.3807e9@b0505.idoo.com>
The NSS test methodologies are published in full.
You don't have the details of the tests (not even the "baseline"
signatures).
The most important part of a test, in my point of view,
It is methodology.
You can see if this test is reasonable and suitable for known enviroments.
As for the test suit,
Maybe you can contact NSS to obtain a copy ?
They are outdated. The most recent exploit tested must be two years old...
They are copy and paste from IDS tests which are far older.
I do not understand your point, you claim they do not open the test suit.
And in the meantime you are flamming NSS for 'outdated tests'.
It is not logical.
And the whole methodology is not appropriate. IPS are not IDS.
For IDS "false alarms" generated by out of session packets (like the one
snot would raise on snort) are not acceptable as it would confuse
administrators in their research for effective attacks.
In the case of IPS it is different. OK, it was not a real attack but who
cares. The purpose of IPS is to block. Who cares if it blocked attacks out
of session? It was not legitimate anyway.
I think MOST network administrator cares.
If the vendors of IPS doing in your way, they are all crashed right now.
If it is IDS, it is not so painful to handle false alarm.
They are alarms only and will not cause side-effect in your network.
But, imagine a device in the network block legitimate traffic just because
it looks like an attack.
The network administrators will definitely suffer from this kind of
'technology'.
Do you really care about the phf exploit? Or maybe the old sshutupteo from
gobbles? Are you talking about organizations or museums?
I do not know how new or how old exploits they use in the test.
But again, if the methodology is correct, it doesn't matter if it applies
old signature to test.
If they can create most of the scenarios that attackers would apply in an
attack, and prove the device can work in the condition.
It is vendor's responsibility to maintain the latest pattern/signatures.
Anti-evasion is Whisker (not nikto, I said whisker) and fragroute 1.2...
Modified exploits are common ones with strings changed (GOBBLES to GOBBLED).
So your exploit database must be very old
It would be better if you can propose a more comprehensive methodology
rather than just flame others.
From my point of view, NSS test has its reputation in security technology
evaluation.
I would not blame them for the test fees.
All of the tests like ICSA/OSEC they are doing the same way.
Immunity can create their own test for free and with latest exploit DBs.
If they are as good as they claimed, I do not see any reason why vendors
would not join.
Regards,
.Jarvett.
Senior Consultant
BroadWeb Co.
--------------------------------------------------------------------------
Test Your IDS
Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
--------------------------------------------------------------------------
