Excellent point. There are no objective benchmarks, or even useful rules
of thumb. However, these metrics do provide a baseline and then an
indication of whether you're getting better or worse over time. For most
measurements, the target goals are strictly theoretical. For example, you
could strive to approach zero delay between announcement and patching of
vulnerabilities, achieve 100% blocking of viruses/spam before reaching the
desktops, write a policy that addresses every part of ISO17799, etc.,
etc..
These goals will essentially never be reached, but they help in developing
a systematic approach to mitigating risk. Dashboards are not solutions,
merely relative gauges. As such, their importance should be valued
accordingly.
- Rich
Fred Cohen <fred.cohen@all.net>
02/24/2006 01:50 AM
To
"Steve R. Smith" <steve_smith1999@sbcglobal.net>
cc
horizons nouveaux <horizonsnouveaux@hotmail.com>,
loganalysis@securityfocus.com, security-management@securityfocus.com
Subject
Re: Security Dashboard
On Feb 22, 2006, at 9:19 AM, Steve R. Smith wrote:
Some of the typical manually generated security
metrics that I've seen are things like:
Patch cycle times vs. unpatched machines (from SUS or
automated patch tools, manual patch application and
monitoring, etc)
But in what sense does this help actually answer any important or
useful questions? Say I measure it and I find that the patch cycle
time for a computer is 5 days and the number of unpatched machines is
20% of all machines. What value does this have for me. Should it be
more? less? at what cost?
Phishing, SPAM content blocked at SMTP gateways
What portion of what spam content - and by whose definition? You mean
some spam blocked at some gateways? I don't even know what to count
here.
Helpdesk calls- virus cases, password resets, static
token requests
Suppose the helpdesk calls is 500 / week and virus cases is 17 /
password rests are 4,589 . static token requests are 987. What does
that mean exactly. Am I doing well or poorly? How should I change
what I am doing - toward what goal at what price?
Vulnerabilities over time (Qualysguard, Foundscan,
IP360, Nessus, consulting reports, etc)
I have reports of 47 vulnerabilities every time I scan with Nessus -
what does it mean?
Security policy compliance (ISO17799)
What am I measuring here?
Typically, this information gets sent up the chain to
various IT Executives such as CIOs, CTOs, CISOs, Audit
Directors, and/or corporate governing bodies.
I send the report with the numbers above up to a CEO. What do they
do? How do they interpret it?
I have seen lots od dashboards - and so far I didn't find even one
that did much to help me manage risks better. What am I missing?
FC
Regards,
Steve
--
-- This communication is confidential to the parties it is intended
to serve --
Security Posture securityposture.com tel/fax
University of New Haven unhca.com 925-454-0171
Fred Cohen & Associates all.net 572 Leona Drive
ASP Press asp-press.com Livermore, CA 94550
