Fred Cohen wrote, On 2/24/2006 12:50 AM:
On Feb 22, 2006, at 9:19 AM, Steve R. Smith wrote:
Some of the typical manually generated security
metrics that I've seen are things like:
Patch cycle times vs. unpatched machines (from SUS or
automated patch tools, manual patch application and
monitoring, etc)
But in what sense does this help actually answer any important or useful
questions? Say I measure it and I find that the patch cycle time for a
computer is 5 days and the number of unpatched machines is 20% of all
machines. What value does this have for me. Should it be more? less? at
what cost?
This metric doesn't necessarily compare against a baseline, but in some
ways it helps us calculate risk. Especially whne a new threat emerges
for which there is a patch. We have some idea fo how long we'll
specifically be vulnerable, and what the costs might be. This metric
isn't useful in isolation, but combined with a look at the
threats/risks, we get a sense of our exposure.
Helpdesk calls- virus cases, password resets, static
token requests
Suppose the helpdesk calls is 500 / week and virus cases is 17 /
password rests are 4,589 . static token requests are 987. What does that
mean exactly. Am I doing well or poorly? How should I change what I am
doing - toward what goal at what price?
In an of itself this is purely an operational metric, but it does
potentially track the impact that given security decisions are having on
the rest of the organization. Increase or decrease the conditions on
passwords such as length, frequency of change, etc. and you're certainly
going to see movement in these numbers, which can be considered
direct/indirect costs of the security policy. Not measures of security
itself, but certainly of the costs.
Security policy compliance (ISO17799)
What am I measuring here?
Well, potentially regulatory compliance. Being out of compliance, and
how far can certainly be an issue.
I have seen lots od dashboards - and so far I didn't find even one that
did much to help me manage risks better. What am I missing?
Fred - do you think it at all appropriate to actually capture metrics
about the security program? In any process we can simply measure the
results (risks) but we might also want to understand how cost effective
our security program is.
For a few of the metrics measured above they aren't useful in isolation
and/or without some guidance as to acceptable, but that doesn't make
them worthless. In some cases specific security measures aren't going
to be directly to risk reduction in a purely security sense. That said,
an organization that doesn't patch its machines and has an incident is
going to much worse liability wise than one that does. So, if we
believe our goal should be to have n% of machines patched, then tracking
how we're doing is probably a good idea.
The metrics themselves don't result in more security, but they certainly
help tell us how we're doing against our own measures, which admittedly
are still just ballpark ideas about how to reduce risk.
Do you have any other approaches you think work better?
--
Andy Steingruebl | e-mail: asteingruebl@cccis.com
Information Security Architect| phone: (312) 229-2409
CCC Information Services | post: 444 Merchandise Mart
| Chicago, IL 60654-1005
