Warning: long responses below :-)
----- Mensaje original -----
De: rzaluski <rzaluski@ivolution.ca>
A Penetration Test entirely depends on the scope and depth of the
operation and what has been agreed upon. Sometimes finding a
vulnerable system is
enough. It all depends on the level of intrusion of the
Penetration Test.
I agree. Flexibility is a key to sell PenTests. Clients do have problems (or
not idea at all) defining the scope and vendors should help them defining the
scope by making an adequate balance between cost and benefits. The benefit
should also be very clear to the client, and this is the key to a successful
Pentest.
My point is if you are Pen Testing a client's mission critical
productionsystems do you really want to bring it / them down? It
is entirely possible
to do so costing a company potentially large amounts of money in the
process. This is especially if the testing is taking place with
anything that processes monetary transactions.
I also concur. My comment regarding manual tests wasn’t meant to imply that the
consultant gives added value by going beyond scope. You can bring a system down
with automated tools as well in any case. The important thing here is not
whether DoSing a system is in the scope or not, but rather whether the
consultant knows what he is doing and is complying with the defined scope. It
is within the scope that the consultant employs its abilities to make better
tests.
There won’t ever be enough time to throughfully test everything within an
organization, therefore automated tools are used to save time with some tasks
(which should be also saving money for your client), but even configuring the
tools correctly is not a simple task. You just can’t run a tool with all tests
available turned on, it will be a waste of time and get you huge amounts of
false positives that you will need to filter out. In the end, this costs money
to the client and to the PenTest provider.
Making a good job is simply not good enough, results have to be of value and
understandable for the client. Even if people who make automated tools make
good efforts to make comprehensible reports, there is always a need to explain
things. Filling that gap is also a responsibility of the consultant because he
is able to identify certain situations and needs of their clients.
Summarizing some key points of what a good PenTest should have from my point
of view (also including comments from other people in the forum):
* PenTest consultant should be flexible and understand the needs of their
clients ( a methodology is a guideline, but in this business vendors do not
necessarily need to go step by step by the book)
* Have a well defined scope and comply with it (no more and no less).
* Have capable consultants able to perform manual checks if/when necessary
(basically having people that know what they do).
* Choose and adequate set of tests (manual and automated) that will yield the
best balance of cost/benefit (example: if the client provides detailed
information of their infrastructure to improve results and it is documented in
the scope, then there is simply no need to include tests for
plataforms/applications that are not part of that infrastructure. I emphasize
this point because I’ve seen it done many times and it makes the consultant
look bad to the eyes of some clients).
* Have a methodology (a PenTest consultant can’t just be cluelessly poking
around; there is money paying for the job and even a black box test can be
planed and documented).
* Make results understandable for the client (whether on the executive summary
or on the technical report, it is the responsibility of the consultant to
explain every finding and recommendation to the client until it is clear).
* Findings and recommendations should consider the clients infrastructure and
business process; consultant should make use of all information available to
him to make proper annotations. There is a big difference between technical
impact of a vulnerability and the actual impact for business, since resources
have diverse degrees of importance for an organization (automated tools can
measure the first reasonably well, but measuring the impact for a business
process is a job for a capable human being).
Now, if I interpreted correctly Adriel’s concern: clients are still distrust
PenTest consultants. He discussed a specific points that cause this problem:
the grade of use of automated tools and manual testing, and confusion derived
from PenTest definition (what is included and how should it performed). There
are many more causes that make selling and performing a PenTest a difficult
task, and also from the clients point of view there are a lot of problems while
buying a PenTest (Is the price of this proposal right? Is this vendor really
good at it?).
Methodologies such as OSSTMM make things easier for consultants, but there is
really few information and tips available for clients (not all clients have an
Infosec department with experienced personnel that is able to discern between
bad and good proposals/consultants). So, maybe providing the clients with such
information will make things easier for both (probably extending OSSTMM with
some questions that an organization might ask themselves when evaluating a
vendor). Here are a few I can think of:
While evaluating a PenTest proposal, a client should ask himself:
* What certifications do the consultants posses(I know, I know… it doesn’t
guarantee capability, but at least it attest that the guys making PenTest had a
minimum knowledge of these arts.)
* What methodology does the vendor employs? If proprietary is he disclosing it
for evaluation?
* What kind of tools will the vendor employ?
* What kind of manual test will the vendor perform?
* Is the scope reasonable from the business point of view? Are the proposed
targets part of the critical business processes? Can it be adjusted to business
needs?
* Can someone from the company be with the consultants during PenTest to see
how the job is done?
And some more difficult questions:
* Is the price right? (you can’t check it out on the Internet like the price of
a certain brand of firewall or a Kg. of Tomato).
* Where have these guys worked before? (sometimes, with permission, vendors can
provide some references) Has this vendor/consultant screwed up badly elsewhere?
(vendor won’t be telling, of course…).
Enough of it. I’ll stop here or moderator will ban me for DoSing the list with
lengthy posts, written in bad English.
Best regards,
Omar Herrera
