I just got sent this thread (not subscribed to the list) so hope this is in
time to be of interest.
Hacme Bank is now in Rev 2 (re-write including web services and new
sections), the version on the web site is Rev 1. It will be posted soon.
That said its not a good benchmarking tool for testing these tools, nor is
WebGoat. That is not to say there are some gapping flaws the tools typically
don't find but Hacme Bank and WebGoat weren't designed for this.
SiteGenerator however will be and is being specifically developed for
exactly that purpose. It will be complete for Beta by the end of October.
Here are some highlights from the current design (subject to change before
release)
1. Tool will build (user configured via GUI) ASP.NET sites of varying
complexity i.e. varying amount of;
a. Size (number of pages and number of links)
b. Page complexity
i. Client-side script
ii. Flash
iii. Applets
iv. Active-X
v. AJAX
c. Vulnerability density (number of vulns per page)
d. Type of vulnerability
i. Authentication
ii. Authorization
iii. Data Validation
1. SQL Injection
2. XSS
3. Path Traversal
iv. Exception and error handling
v. Configuration management
vi. User Management
vii. etc
e. Complexity of vulns (i.e. easy SQL injection ' OR 1=1-- and hard
ones)
The idea is that a user can configure the tool to generate a site that is of
15,000 pages, 30,000 links, with 10% of pages having JavaScript direction,
an average of 5 form elements per page (10% hidden form elements) and a
range of 1 to 100 elements or there will be an average of 4 vulnerabilities
per page with 40% data validation, 10% site design (no logout button etc),
30% authorization, 10% configuration management etc
It will also have a component (HTTP Module) that captures all signatures
attacking the site so you will be able to compare the attempted attacks; the
ones that were successful and the time it took to find types of issues. From
this you can compute the number of false positives and false negatives. We
won't publish any results of tools themselves but the tool is designed so
people can do that against an environment that is like their own and not
some canned site (I cant belive anyone would but based on results from a
canned site built by a vendor of the product but .I guess some do.) I am
sure people will share results in public.
____________________________________________________________________________
_____________________________
How about Foundstone's HacmeBank
(www.foundstone.com/resources/proddesc/hacmebank.htm) ? Anyone tried
this application for benchmarking ?
I'll be getting on this benchmarking task myself soon. If you can wait a
week or two, mostly I'll send you the review myself :)
