I am looking forward to the results of a public
quantifiable test using SiteGenerator and a consistent
set of criteria.
The general problem with benchmarking though is that when you have
nothing formal it seems like a really good idea, but as soon as you have
a formalised standard test set, all the top vendors will simply
re-engineered to find 100% of all the benchmark test cases by default,
and there will still be nothing on which to select between them.
Additionally, many vendors will end up building a specific rule to test
for the exact circumstances of the test case, rather than analysing for
the generalised principal (such as checking for only a script tag,
rather than looking for unescaped data in the wrong place). The
benchmark could end up providing a false sense of security.
I quite like academic approach taken by the fine Dr; select your test
set independently (to suite your own needs), run your tests empirically,
publish all detail. The only down side to the report that provoked this
thread is that it also included an unnamed application, so the results
are not independently repeatable/verifiable.
Martin...
----------------------------------------------------------------------
CONFIDENTIALITY: This e-mail and any files transmitted with it are
confidential and intended solely for the use of the recipient(s) only.
Any review, retransmission, dissemination or other use of, or taking
any action in reliance upon this information by persons or entities
other than the intended recipient(s) is prohibited. If you have
received this e-mail in error please notify the sender immediately
and destroy the material whether stored on a computer or otherwise.
----------------------------------------------------------------------
DISCLAIMER: Any views or opinions presented within this e-mail are
solely those of the author and do not necessarily represent those
of Corsaire Limited, unless otherwise specifically stated.
----------------------------------------------------------------------
Corsaire Limited, 3 Tannery House, Tannery Lane, Send, Surrey, GU23 7EF
Telephone: +44(0)1483-226000 Email:info@corsaire.com
-------------------------------------------------------------------------
Sponsored by: Watchfire
Watchfire named worldwide market share leader in web application security
assessment by leading market research firm. Watchfire's AppScan is the
industry's first and leading web application security testing suite, and
the only solution to provide comprehensive remediation tasks at every
level of the application. See for yourself.
Download a Free Trial of AppScan 6.0 today!
https://www.watchfire.com/securearea/appscansix.aspx?id=701300000007t9c
--------------------------------------------------------------------------
