To add a little to this debate.
First, there are two types of Audit, internal and external. An audit,
consisting of an evaluation of an organisation's systems processes and
controls, is performed against the set standard or documented process.
Audits are designed to provide an independent assessment through a
qualified independent assessment of representations about the system or
process. An audit may also provide a gap analysis of the operating
effectiveness of the internal controls.
An audit differs from an inspection in that an audit makes
representations about likely future results. An inspection evaluates
past results. Or an audit to be valid it must be conducted according to
accepted principles. In this, the audit team and individual auditors
must be certified and qualified for the engagement. Numerous "audits"
are provided without certification, these however are qualified reviews.
A penetration test is an attempt to bypass controls and gain access to a
one system. The goal of the penetration test is to prove the that the
system may be compromised. A penetration test does not assess the
relative control strength nor the system or processes deployed, rather,
it is a "red teaming" styled exercise designed to prove illicit access.
The real strength of a penetration test is marketing the need to improve
controls to internal management. A penetration test is of limited value
in the greater scheme of a systems information security due to the
restricted nature of the test and the lack of inclusion of many key
controls.
A vulnerability assessment is an assessment and gap analysis of a site's
or a system's control strengths. A vulnerability assessment is a risk
based process. The process involves the identification and
classification of the primary vulnerabilities which may impact the
system. Often, methodologies such as fault tree analysis end cause
consequence analysis are employed in this review.
Both vulnerability assessments and penetration tests may be conducted as
a white box or black box analysis. A black box analysis is instigated
with little or no knowledge of the system being tested. A white box
analysis is conducted for knowledge of the system.
A vulnerability assessment is a critical component of any threat risk
assessment. Following the vulnerability assessment and impact analysis
is conducted and used in conjunction with a threat report to provide for
an estimation of the organisation's risk to selected attack vectors.
External audits are conducted (or at least should be) by independent
parties no rights or ability to alter or update the system. Internal
audits involve a feedback process where the auditor may not only audit
the system but also potentially provide advice in a limited fashion. And
external auditor is precluded from advising their client. They are
limited to reporting any control gaps and leading the client to a source
of accepted principles.
The common perception that running an automated scanner such as Nessus
or one of its commercial cohorts is in itself a vulnerability or
penetration test is false.
Most of the so-called penetration tests that are provided are no more
than a system scan using tools. A penetration test it correctly provided
will attempt the use of various methodologies to bypass controls. In
some instances this may involve the creation of new or novel
scripts/programs.
The issue is not that many people commonly use the words interchangeably
but that so-called professionals fail to differentiate the terms. Of
particular concern is the use of audit and the designation auditor. This
is as these terms are often restricted in code. This is that most
jurisdictions have statutory requirements surrounding their use and
application.
Information security systems provide many of the functions that
construct a control system. Of particular concern are controls that
limit access to accounting and financial records. This includes records
held by systems that provide an e-commerce transaction path. In many
jurisdictions it is an offence to sign off an audit report when you are
not a certified auditor. Traditionally the path around this has been not
to call the process of testing the system and audit, but rather to call
it an agreed procedures review.
An agreed procedures review or simply a review is an analysis of
controls performed against an agreed process.
Some example's of an audit include SAS 70 (part 1 or 2) audits, ISO
9001,17799:2/27001 certification audits, HIPPA audits. There are many
different types of audits and many standards that an audit may be
applied against.
There are various processes and procedures used to provide vulnerability
assessments and threat risk analysis. Standards such as AS/NZS 4360:2006
I commonly mandated by government organisations.
Penetration testing, if done correctly, may provide some value in its
free-form approach. When correctly implemented, a penetration test adds
a level of uncertainty to the testing. The benefit of this uncertainty
is that it might uncover potential flaws in the system or controls that
had not been taken into account when designing the control system. To be
of value, a penetration test must needs to do more than scan a system.
It needs to do something novel and unexpected.
There is little similarity between a penetration test, vulnerability
assessment, risk assessment or audit. The lack of understanding of these
differences impedes the implementation of effective security controls.
Bus to finish, 'Stylewar' is correct in stating that "an audit must
follow a rigorous program...". Christine's appraisal of a vulnerability
assessment would more correctly be termed as a controls assessment. A
controls assessment may also be known as a security controls review.
As for the need to develop a structured taxonomy (naming system), there
is already one in existence. None of these terms or services is new. All
these services have been provided for as long as computers have been
used by business and government. They were definitely employed as far
back as the 70s.
Regards,
Craig
-----Original Message-----
From: StyleWar [mailto:stylewar@cox.net]
Sent: Wednesday, 9 August 2006 3:19 AM
To: 'Christine Kronberg'; 'Arkem Paul'
Cc: pen-test@securityfocus.com
Subject: RE: Vulnerability Assessment vs. PenTest
Point of fact that an audit must follow a rigorous program, and has a
set of
documentation and traceability requirements with it that an 'assessment'
does not. They are 'approximate' in the hands of a well disciplined
assessment team - but I would stop a hair short of calling them equal..
-
StyleWar
"Ancora Imparo"
-----Original Message-----
From: Christine Kronberg [mailto:seeker@shalla.de]
Sent: Sunday, August 06, 2006 11:54 AM
To: Arkem Paul
Cc: pen-test@securityfocus.com
Subject: Re: Vulnerability Assessment vs. PenTest
On Sun, 6 Aug 2006, Arkem Paul wrote:
A Vulnerability Assessment should be a comprehensive look
from policy
and procedures to implementation of security in the network
and should
include such things as patch management, virus protection, user
education, SOE hardening, infrastructure configuration, etc.
So basicly an assessment is equal to an audit? The
description above
is what I usually expect from someone doing an audit.
A vulnerability assessment I tend to understand in terms
of investigating
a specific application (in far more detail than a
penetration test).
There are a couple of term mixed every now and again (like someone
else just stated: funny that we professionals don't come
up with _one_
definition):
Audit
Security Scan
Security Assessment
Vulnerability Assessment
Penetration Test
Did I miss one?
Cheers,
Christine Kronberg.
Liability limited by a scheme approved under Professional Standards Legislation
in respect of matters arising within those States and Territories of Australia
where such legislation exists.
DISCLAIMER
The information contained in this email and any attachments is confidential. If
you are not the intended recipient, you must not use or disclose the
information. If you have received this email in error, please inform us
promptly by reply email or by telephoning +61 2 9286 5555. Please delete the
email and destroy any printed copy.
Any views expressed in this message are those of the individual sender. You may
not rely on this message as advice unless it has been electronically signed by
a Partner of BDO or it is subsequently confirmed by letter or fax signed by a
Partner of BDO.
BDO accepts no liability for any damage caused by this email or its attachments
due to viruses, interference, interception, corruption or unauthorised access.
------------------------------------------------------------------------------
This List Sponsored by: Cenzic
Concerned about Web Application Security?
Why not go with the #1 solution - Cenzic, the only one to win the Analyst's
Choice Award from eWeek. As attacks through web applications continue to rise,
you need to proactively protect your applications from hackers. Cenzic has the
most comprehensive solutions to meet your application security penetration
testing and vulnerability management needs. You have an option to go with a
managed service (Cenzic ClickToSecure) or an enterprise software
(Cenzic Hailstorm). Download FREE whitepaper on how a managed service can
help you: http://www.cenzic.com/news_events/wpappsec.php
And, now for a limited time we can do a FREE audit for you to confirm your
results from other product. Contact us at request@cenzic.com for details.
------------------------------------------------------------------------------
