You have done lots in commentary over the years sir.
Its not about providing more issues, its about providing representative issues.
Its about a criteria that is truly representative of what the common web
security issues are. I agree these things are not *TOTALLY* useless (although I
would never buy one personally). Its about finding a way to get the facts out
there about what this set of technology can actually do and holding that up
against the big picture of web security issues for all to make a fair unbiased
judgement of their usefullness.
Imperva published a press release saying they cover the OWASP Top Ten a while
back if I am not mistaken. Thats just rubbish. Do they decrypt or flag bad
cryptography used in a cookie? No way. Do these products protect from a case
where object affinity results in one user getting another account balance? No
way. Its a data stream tech and only a small subset of issues can be detected
and protected in the data stream.
The challenge is about building a benchmark that is representative of a real
world site and the real world issues it faces. Those issues are not always via
a front-end web app. Its not about front-impact testing only.
This is saying if you are going to hold your products up as examples of what
you think the industry should accept, then hold them up against a real set of
criteria and not a convoluted set of things you know you will shine against.
You don't buy a car for safety because it has dent resistent side panels !
And on that note now for some relevant humor as I read this again today and
laughed hard..
If Cars were built like applications (from Denis Verdon)
70% of all cars would be built without following the original designs and
blueprints. The other 30% would not have designs.
Car design would assume that safety is a function of road design and that all
drivers were considerate, sober and expert drivers.
Cars would have no airbags, mirrors, seat belts, doors, roll-bars, side-impact
bars, or locks, because no-one had asked for them. But they would all have at
least six cup holders.
Not all the components would be bolted together securely and many of them would
not be built to tolerate even the slightest abuse.
Safety tests would assume frontal impact only. Cars would not be roll tested,
or tested for stability in emergency maneuvers, brake effectiveness, side
impact and resistance to theft.
Many safety features originally included might be removed before the car was
completed, because they might adversely impact performance.
70% of all cars would be subject to monthly recalls to add major components
left out of the initial production. The other 30% wouldnât be recalled,
because no-one would sue anyway.
The after-market for safety devices would include such useful products as
training wheels, screen doors, elastic seatbelts and devices that would
restrict the carâs top speed to 3mph, if found to be unsafe (which would be
always).
Useful safety could be found, but could only be custom retro-fitted, would take
six months to fit and would cost more than the car itself.
A DOT inspection would consist of counting the wheels and making
recommendations on wheel quantity.
Your only warning indicator would be large quantities of smoke and flame in the
cab.
You could only get insurance from one provider, it would be extremely
expensive, require a duplicate DOT inspection, and you might still never be
able to claim against the policy.
---- Michael Silk <michaels@phg.com.au> wrote:
Hi,
Firstly let me say I would consider myself a member of owasp
(although I haven't really done much :)) but I'm not taking sides here.
In this letter, you raise the point of " ... Building secure
software requires deep changes in our development culture, including
people, processes, and technology ... ". Valid point, I thought.
In the next paragraph, however - and seemingly the main point of
this article - you go on to say that your "solution" is to create a tool
that just covers more vulnerabilities then that provided by the ASC
(btw: are there any links to what they suggested?) and that this tool
should be used by these application firewalls (which you previously
suggested a 'not very' useful). This leaves me confused ... What was the
point here ? To say that they aren't covering enough technical
vulnerabilities (sql injection, etc) or that they are ignoring the most
important factor - a sound security design.
To me, it seems you are attempting to provide them with, to use
your words, the "... elusive silver bullet" that you claim they
shouldn't be searching for. If, however, your submission succeeds and
these corporations use the OWASP Testing Application it would be more
powerful then it otherwise would've been: they can get a big shiny stamp
from OWASP saying "We Passed!".
Don't get me wrong, I'm not against having OWASP, or whoever,
providing a comprehensive application that mimics common vulnerabilities
- but I'm just not sure what the point of this letter was and I am
wondering whether OWASP really wants to provide the companies with these
stamps of approval.
------------------
The issue, however, seems to be that of companies rating
themselves. Other industries have covered this issue ... Enegry
ratings, car safety, etc ... Perhaps there could be some discussion of
how to formalise these rating for the application security community ...
In Australia something like this would be enforced by the ACCC
(Australian Competition and Consumer Commission), I'm sure America would
have a similar organisation .. Perhaps proposals could be made ?
-- Michael
-----Original Message-----
From: The OWASP Project [mailto:owasp@owasp.org]
Sent: Tuesday, 16 November 2004 2:34 PM
To: webappsec@securityfocus.com
Subject: An Open Letter (and Challenge) to the Application Security
Consortium
An Open Letter (and Challenge) to the Application Security Consortium
Since its inception in late 2000 the Open Web Application Security
Project (OWASP) has provided free and open tools and documentation to
educate people about the increasing threat of insecure web applications
and web services. As a not-for-profit charitable foundation, one of our
community responsibilities is to ensure that fair and balanced
information is available to companies and consumers. Our work has become
recommended reading by the Federal Trade Commission, VISA, the Defense
Information Systems Agency and many other commercial and government
entities.
The newly unveiled Application Security Consortium recently announced a
"Web Application Security Challenge" to other vendors at the Computer
Security Institute (CSI) show in Washington, D.C. This group of security
product vendors proposes to create a new minimum criteria and then rate
their own products against it.
The OWASP community is deeply concerned that this criteria will mislead
consumers and result in a false sense of security. In the interest of
fairness, we believe the Application Security Consortium should disclose
what security issues their products do not address.
As a group with a wide range of international members from leading
financial services organizations, pharmaceutical companies,
manufacturing companies, services providers, and technology vendors, we
are constantly reminded about the diverse range of vulnerabilities that
are present in web applications and web services. The very small
selection of vulnerabilities you are proposing to become a testing
criteria are far from representative of what our members see in the real
world and therefore do not represent a fair or suitable test criteria.
In fact, it seems quite a coincidence that the issues you have chosen
seem to closely mirror the issues that your technology category is
typically able to detect, while ignoring very common vulnerabilities
that cause serious problems for companies.
Robert Graham, Chief Scientist at Internet Security Systems, recently
commented on application firewalls in an interview for CNET news. When
asked the question "How important do you think application firewalls
will become in the future?" his answer was "Not very."
"Let me give you an example of something that happened with me. Not long
ago, I ordered a plasma screen online, which was to be shipped by a
local company in Atlanta. And the company gave me a six-digit shipping
number. Accidentally, I typed in an incremental of my shipping number
(on the online tracking Web site). Now, a six-digit number is a small
number, so of course I got someone else's user account information. And
the reason that happened was due to the way they've set up their user
IDs, by incrementing from a six-digit number. So here's the irony: Their
system may be so cryptographically secure that (the) chances of an
encrypted shipping number being cracked is lower than a meteor hitting
the earth and wiping out civilization. Still, I could get at the next ID
easily. There is no application firewall that can solve this problem.
With applications that people are running on the Web, no amount of
additive things can cure fundamental problems that are already there in
the first place."
This story echoes some of the fundamental beliefs and wisdom shared by
the collective members of OWASP. Our experience shows that the problems
we face with insecure software cannot be fixed with technology alone.
Building secure software requires deep changes in our development
culture, including people, processes, and technology.
We challenge the members of the Application Security Consortium to
accept a fair evaluation of their products. OWASP will work with its
members (your customers) to create an open set of criteria that is
representative of the web application and web services issues found in
the real world. OWASP will then build a web application that contains
each of these issues. The criteria and web application will be submitted
to an independent testing company to evaluate your products. You can
submit your products to be tested against the criteria (without having
prior access to the code) on the basis that the results are able to be
published freely and will unabridged.
We believe that this kind of marketing stunt is irresponsible and
severely distracts awareness from the real issues surrounding web
application and web services security. Corporations need to understand
that they must build better software and not seek an elusive silver
bullet.
We urge the Consortium not to go forward with their criteria, but to
take OWASP up on our offer to produce a meaningful standard and test
environment that are open and free for all.
Contact: owasp@owasp.org
Website: www.owasp.org
**********************************************************************
This email message and accompanying data may contain information that is
confidential and/or subject to legal privilege. If you are not the intended
recipient, you are notified that any use, dissemination, distribution or
copying of this message or data is prohibited. If you have received this
email message in error, please notify us immediately and erase all copies of
this message and attachments.
This email is for your convenience only, you should not rely on any
information contained herein for contractual or legal purposes. You should
only rely on information and/or instructions in writing and on company
letterhead signed by authorised persons.
**********************************************************************
