Information systems slide IT security metrics

After completing this workshop, you will be able to: • Identify why metrics are important for IT security • Identify the relationship among GISRA, NIST SP 800-26, and IT Security Metrics

Introduction

IT Security Metrics Training

Audience: Federal IT security personnel with GISRA reporting

responsibilities

Goal: To train Federal IT security personnel how to develop metrics that they

can use immediately to assist with GISRA reporting

Duration: 3 hours

After completing this workshop, you will be able to:

• Identify why metrics are important for IT security

• Identify the relationship among GISRA, NIST SP 800-26, and IT Security Metrics

• Describe IT Security Metrics

• Describe metrics development process

• Apply metrics development process by completing a Metrics Form for one of the OMB GISRA reporting requirements for FY02

• Identify metrics-related Roles and Responsibilities

• Describe how to implement a Metrics Program

Metrics Development

In this section, you will:

• Learn the definition and characteristics of IT Security Metrics

• Identify the difference between Performance Goals and IT Security Metrics

• Learn the seven-step IT Security Metrics Development Process

• Discover the types of information and insights that can be gained from IT

Security Metrics

• Complete three examples of IT Security Metrics

What are IT Security Metrics?

IT Security Metrics are tools that facilitate decision making and

accountability through collection, analysis, and reporting of relevant

• Provide relevant performance trends over time

• Useful in tracking performance and directing resources

Why Measure IT Security?

• Measure successes and failures of

past and current security investments

• Justify future investments

• Improve accountability to stakeholders

• Ensure appropriate level of mission support

• Determine IT security program

• Government Information Security Reform Act (GISRA)

• Ensure best value from security

• Build confidence in leadership

• Demonstrate improvement to stakeholders

• Play key role in initiating improvement actions based on performance trends

• Satisfy regulatory requirements

IT Security Metrics should support IT security goals and

objectives

IT Security Performance Goals identify desired results of system security

program implementation.

IT Security Performance Objectives enable accomplishment of goals by:

• Identifying strategic practices, defined by security policies, procedures, and

controls

• Directing consistent implementation of policies and procedures across the

organization

IT Security Metrics monitor accomplishment of goals and objectives by:

• Quantifying the level of implementation of security control objectives and

techniques for a system and the effectiveness and efficiency of the controls

• Using analysis of collected IT Security Metrics to determine adequacy of security activities and make appropriate business decisions

Exercise: Performance Goal or IT Security Metric?

Program Officials understand the risk to systems under

their control and determine the acceptable level of risk.

Percentage of system security plans that are

updated annually.

Duties are separated to ensure least privilege

and individuals accountability.

Percentage of systems with automated virus updating.

Data integrity and validation controls are used to

provide assurance that the information has not been

altered and the system functions as intended.

IT Security Metric

Metrics development is a seven step process

The focus of the metrics program depends on IT security program maturity

Stakeholders and Interests

• Anyone within an organization is an IT security stakeholder, though some functions have a greater stake than others:

– CIO

– Program Manager/System Owner

– Security Program Manager

– Resource Manager

– Training/Human Resources Personnel

• Each stakeholder needs a set of metrics that provides a view of the organization’s

IT security performance within their needs, for a total of no more than 10-20

metrics per stakeholder

• Many IT Security Metrics can be created to measure each aspect of the

organization’s IT security Selecting the most critical elements of the organization’s

IT security program during metrics prioritization will make the program manageable and successful

IT security performance goals and objectives are expressed in the form of high level policies and requirements in many laws, regulations, policies, and

guidance that describe the dimensions of an effective IT security program:

• Clinger Cohen Act

• Presidential Decision Directives 63

• Government Information Security

Reform Act (GISRA)

• OMB Circular A-130, Appendix III

• Critical Elements within

NIST Special Publication 800-26

• Federal Information Security Compliance

Audit Manual (FISCAM)

IT Security Performance Goals and Objectives

IT Security Policies, Guidance, and Procedures

Some Federal guidance and agency-specific policies and procedures provide more detailed information specific to the agency:

• NIST SP 800-12, 800-14

• Agency-specific policy and guidance

• Subordinate questions within

NIST Special Publication 800-26

System Security Program Implementation

System Security implementation includes:

• Processes and procedures in place

• Existing capabilities

• Areas for improvement

• Existing metrics

• Existing data sources that can be used to derive metrics data

These may be documented in the following sources:

• System Security Plans

• OMB Plan of Actions and Milestones (POA&M) reports

• Latest GAO and IG findings

• Tracking of security-related activities

• Risk assessments and penetration testing results

Metrics can describe three aspects of IT security program

operations and management

an organization’s processes are self-regenerating andmeasurement data gathering is transparent

Security Program Effectiveness and Efficiency:

As an organization’s process maturity increases and performance data becomes more readily available,

metrics will focus on program efficiency and

It is important to record the specifics of each metric for the purposes of data analysis and possible metric reuse

Defines the metric by describing the quantitative measurement(s) provided

Describes the calculation to be performed that results in a numeric expression of a metric

Lists the location of the data to be used in calculating the metric

Provides information about the meaning of the metric and its performance trend

Proposes possible causes of trends, identified through measurement, and points at possible solutions to correct observed shortcomings

Metrics can help identify causes of poor performance,

Inefficient planning processes that influence the metrics (including communication processes necessary to direct organizational actions)

Awareness and Commitment

Policies and Procedures

Architectures

Inefficient Processes

Employees sign employee agreements stating that they have read and understood rules of behavior

How does NIST SP 800-26 relate to metrics?

13.1.1 Have employees received a copy of the Rules of Behavior?

13.1 Have employees received adequate training to fulfill their security responsibilities?

13.1.5 Have employees received a copy of

or have easy access to agency security procedures and policies?

13.1.3 Is there a mandatory annual refresher training?

Example 1: Security Awareness, Training, and Education

Percentage employees who underwent initial IT security awareness training

To determine the number of new employees who underwent required IT security awareness training, including receiving a copy of Rules of Behavior and security policies and procedures

Semi-Annually

If response to 1-3 was “Yes” % = (# who took training) / (Total # of new employees)

Yes No

1 Is security awareness training required for new employees?

Critical Element 13.1 Have employees received adequate training to fulfill their security responsibilities? Subordinate Questions 13.1.1 Have employees received a copy of Rules of Behavior?

13.1.5 Have employees received a copy of or have easy access to agency security procedures and

policies?

Security awareness training is effective when it includes specific information on rules of behavior and information on security policies and procedures This metric validates the content of training and determines the percentage of employees who took the training High numbers close to 100% are highly desirable Having employees who do not understand the security implications of their actions

Example 2: Incident Response Capability (Efficiency)

Example 3: Hardware and System Software Maintenance

Percentage of systems with latest patches installed

To quantify the level of risk exposure caused by the lack of current security patch implementation

Monthly(# components with up-to-date patches)/(Total # of components)Regular Vulnerability Scanning

1 Is regular vulnerability scanning conducted?

Critical Element 10.3 Are systems managed to reduce vulnerabilities?

Subordinate Question 10.3.2 Are systems periodically reviewed for known vulnerabilities and software

patches promptly installed?

Insufficient staff Superseding other priorities

This metric monitors compliance with applicable patches and provides useful information about the level of risk exposure at a system level The goal in this case is 100% The

Implementation

Evidence

2 How many components are scanned every time? Fill in the blank

3 How many of those had up-to-date patches during the last scanning cycle?

Fill in the blank

Yes No

4 If your answer to question 1 was “no”, what was the reason?

Insufficient funding

Metrics Development Criteria: What is a Good Metric?

 Based on IT security performance goals and objectives: NIST SP 800-26 Critical Elements

and Subordinate Questions are used to derive performance goals and objectives

 Quantifiable: Metrics should yield quantitative rather than qualitative information to increase

the objectivity and validity of data

 Obtainable/Feasible to measure: Metrics data should be available or easily collected through

interviewing or by accessing data repositories If a metric requires significant modification of

agency processes or implementing a new tool, data collection may not be feasible at this time

 Repeatable: Measurements should be able to be repeated in a standard way at

predetermined intervals to identify trends or identify if positive changes have occurred as a

result of corrective actions

 Provide relevant performance trends over time: Repeated measurements reveal change in a

timely manner

 Useful in tracking performance and directing resources: Metrics should be useful to

stakeholders and should yield information that is important in financial decision making

Breakout Session

Breakout Session

Goal: To complete a Metric Form for one of the metrics that is required for GISRA reporting for FY

2002 This includes identifying the NIST SP 800-26 Critical Element and Subordinate Question that map to the specific GISRA question from OMB guidance.

Duration: 30 minutes

Method:

• Read the metric your Breakout Group is assigned

• Select the NIST SP 800-26 Critical Element that includes your metric

• Select the Subordinate Question within the Critical Element that maps to your metric

Remember, a single metric can use more than one Subordinate Question

• Complete the Metric Form’s sections, giving particular attention to what

implementation evidence may exist that corresponds to your Subordinate

Question

Follow up: Each Group will have five minutes to brief their Form

to the other groups This brief should include:

Metrics Program Implementation

In this section, you will:

• Receive an introduction to the IT Security Metrics-related roles and

responsibilities

• Learn the steps involved in IT Security Metrics program implementation by

learning the process and following an example through the process

Multiple success factors can influence quality and

sophistication of IT Security Metrics (slide 1 of 2)

Ensure that IT Security Metrics Program is manageable:

• Use no more than 10-20 metrics at a time, based on current priorities

• Phase old metrics out and phase new metrics in when performance targets

are reached or when requirements change

Ensure acceptable quality of data:

• Data collection methods and data repositories should be standardized

• Events must be reported in a standard manner throughout the organization and the results of such reports need to be stored in the data repository

Multiple success factors can influence quality and

sophistication of IT Security Metrics (slide 2 of 2)

Obtain organizational acceptance:

• Metrics need to be validated with organization’s stakeholders within

headquarters and in the field

• Metrics should be vetted through appropriate approval channels

Ensure that metrics are useful and relevant:

• Useful data should be collected

• Not all data are useful

Metrics-related roles and responsibilities are dispersed

throughout an organization

Responsibility for

Organizational Acceptance of Metrics Program

Responsibility for

Metrics Data Collection and

Each organization will implement a metrics program specific

to its needs

• Tailor to organization and business processes

• Identify IT Security Metrics-related

stakeholder roles and responsibilities

• Lay out required infrastructure changes, such as

creation of web-based data collection tools and

of new data repositories

• Identify required modifications of the

current data sources

• Define data reporting formats

IT Security Metrics data collection must be as transparent and non-intrusive as possible.

Output from standard security activities can be used to

quantify IT security performance

IT Security Metrics Program Implementation Process

• Identify stakeholders

• Determine goals /

objectives

• Review existing metrics

• Develop new metrics

• Identify data collection

methods and tools

• Collect metrics

• Analyze collected data

• Conduct gap analysis

- Identify gaps between actual

and desired performance

• Identify reasons for undesired results

• Identify areas requiringimprovement

• Determine range of corrective actions

• Select most appropriate corrective actions

• Prioritize corrective actions based on overall risk mitigation goals

• Develop cost model

- Project cost for eachcorrective action

• Perform sensitivity analysis

• Develop business case

• Prepare budget submission

• Budget allocated

• Available resources prioritized

Process Implementation Example

a password cracker that isrun regularly

Employees should be required

to take annual IT security refresher training as part of their annual review process

Since annual refresher training has ceased, the number of weak passwords has increased by 50%

A budget submission detailingmetrics findings related to annual IT security refresher training was submitted, and

Annual refresher training,

• Discussed why Metrics are important for IT security

• Obtained understanding of the relationship between GISRA, NIST SP 800-26,

and IT Security Metrics

• Described IT Security Metrics

• Described the Metrics Development Process

• Created metrics to be implemented at a system level through applying metrics

development process

• Discussed metrics-related Roles and Responsibilities

• Described how to implement a Metrics Program