Final Report of the Customer Satisfaction Work Group

Final Report of the Customer Satisfaction Work Group of the Workforce Information Council Section I – Executive Summary The Customer Satisfaction Work Group Work Group of the Workforc

Final Report of the Customer Satisfaction Work Group

of the Workforce Information Council

Section I – Executive Summary

The Customer Satisfaction Work Group (Work Group) of the Workforce Information Council (WIC) was appointed to explore the issue of customer satisfaction as

it relates to labor market information (LMI) providers and users of the information The Work Group was composed of eleven members, two support personnel and a consultant They met formally three times between October 2002 and June 2003, and gave quarterly updates to the WIC

The Work Group utilized existing documents and gathered other reports to guide its work Many of the concepts employed are private sector marketing concepts The group verbalized the process of developing labor market information products and services as being a process of data collection, data analysis and distribution of information products and services The process defined in this manner seems void of customer satisfaction measurement and needs assessment Therefore, the Work Group concentrated on incorporating the needs and satisfaction of customers into the LMI creation process

Two models were constructed to demonstrate how customer satisfaction and needs assessment can guide the creation of meaningful labor market information and move toward a demand-driven workforce information system One model charts a state scenario and the other extends the model to federal products The result is a dynamic, customer-driven labor market and workforce information system which is responsive to customer needs and wants in a changing environment The process embraces not only the concept of assessing customer satisfaction, but also of working (through evaluation teams and the WIC) toward continuous improvement of the system This is the objective

of the requirement to “consult” with customers as set forth in Section 309 of the Workforce Investment Act of 1998

As part of infusing customer satisfaction into the labor market information development process, some critical components have been identified They include:

• The establishment of evaluation teams in each state, charged with turning customer satisfaction assessment results into improved products and services, and with sharing customer satisfaction processes and improvements with other states and federal agencies;

• Nine measurable attributes which the Work Group believes will lead to more meaningful customer assessment activities and provide for easier information sharing;

• A methodology that state and federal agencies can employ to “think through” the customer satisfaction process The methodology is displayed in a handbook developed by the Work Group which will be made available to assist federal and

state agencies as they work to develop protocols and choose methods for assessment;

• Strategies for documenting demand for labor market information products which can provide guidance to funding agencies, in addition to encouraging broader use

of available information

This document has five sections including this executive summary Section II outlines the Work Group’s approach to work Section III discusses and models the relationships between customer satisfaction and labor market information Section IV reviews pilot studies, and Section V of this document responds specifically to the charter given to the Work Group by the WIC

The response in Section V notes findings by the group and then provides seven actionable recommendations These recommendations include establishing or identifying

a customer satisfaction focal point for sustaining the effort and providing on-going customer satisfaction technical assistance, offering training to agency representatives on implementing customer satisfaction programs, exploring customer satisfaction as a deliverable in the BLS cooperative agreement and continuing it as a deliverable in the ETA core products grant, and expanding the efforts of the Work Group into a demand-driven marketing effort for workforce information

There are also attachments under various tabs Tab III is a customer satisfaction handbook Tab IV reviews pilot study results Tab V is the draft document “Workforce Information Customer Satisfaction Assessment: A Primer for State and Local Planning” developed by Mediacall, Inc and the Heldrich Center Tab VI documents BLS customer satisfaction activities Tab VII is the project work plan which set the course for the Work Group Tab VIII is feedback information from federal agencies on their customer feedback programs Tab IX is a tally of state customer satisfaction activities, and Tab X

is the first progress report of the Work Group

Section II – Approach to Work

Introduction

The Customer Satisfaction Work Group (Work Group) of the Workforce Information Council (WIC) was appointed to explore the issue of customer satisfaction as it relates to labor market information providers and users of the information The WIC defined the scope of the Work Group by providing a charter with five specific outcomes:

1 What are the broad parameters for a customer satisfaction measurement system?

2 What customers and products/services should the system address?

3 How should the activities related to customer satisfaction measurement be

communicated to state LMI offices and state employment statistics agencies? How can their support for the effort be encouraged?

4 What technical assistance should be provided to states in implementing customer satisfaction measurement within their states?

5 How should customer satisfaction measurement be incorporated into federal funding agreements for LMI activities?

The Work Group was composed of eleven members, two support personnel and a consultant, including:

• Olaf Bjorklund, Employment and Training Administration, Federal Co-chair

• James McFadden, Chief, Office of Occupational and Labor Market Information, Delaware Department of Labor, State Co-chair

• Garry Breedlove, Florida Agency for Workforce Innovation

• Tom Gallagher, Manager, Research and Planning, Wyoming Department of Employment

• Rod Fortran, New York Department of Labor, Division of Research and Statistics

• Yolanda Harris, Marketing Communication Specialist, Illinois Department of Employment Security

• Betty McGrath, Economist, North Carolina Employment Security Commission

• Peter Phelan, Center For Workforce Information and Analysis, Pennsylvania Department of Labor and Industry

• Mary Ann Regan, Director, Center for Workforce Information and Analysis, Pennsylvania Department of Labor and Industry

• Vivian Shapiro, Assistant Commissioner, New Jersey Department of Labor

• Stanley Stephenson, San Francisco Regional Commissioner, Bureau of Labor Statistics

Support Staff

• Dixie Sommers, Center on Education and Training for Employment, Ohio State University

• Nancy Pyon, Center on Education and Training for Employment, Ohio State University

Consultant

• Don Norton, Vice President, Mediacall Marketing and Advertising, Inc

The Work Group began meeting quarterly with the consultant in October 2002 Other meetings were held in January and May 2003 The WIC was apprised of status of the project at meetings in December 2002, March and June 2003 A presentation was made

on the Work Group efforts at the BLS LMI conference in May 2003

Several documents were used to guide the effort Some of the documents were developed by staff specifically for the project and others were collected to build in

previous efforts These documents include:

1 A draft “Customer Satisfaction Primer,” funded by the ETA and written

by Mediacall and the Heldrich Center for Workforce Development at Rutgers This document also notes what actions states are implementing,

as well as private sector customer satisfaction practices;

2 A report from the Bureau of Labor Statistics (BLS) on the type of customer satisfaction activities it has undertaken;

3 A “Project Work Plan” developed by the Work Group;

4 Examples of “Customer Feedback Information” collected from federal agencies; and

5 Tally of Customer Satisfaction Activities currently being undertaken by states

Copies of these documents are attached as appendices to this document Similarly, the group developed a report at the mid-point of the process to document resources It is also attached as “Report 1.”

Approach to Work

The Work Group began to approach customer satisfaction with concepts set forth

in the Primer, noted above The group set forth working definitions These include:

o Customer satisfaction: meeting or exceeding the expectations of the customer It is recognized that there may be many attributes to satisfying the customers

o Customer expectations: attributes that are in the mind of the customers, and consist of all parameters of what they, the customers, need, want and

think they need and want

o Customer feedback: information gathered from the customers to assess their level of satisfaction

There were many discussions of “marketing” as it relates to labor market information The group embraced the concept of marketing as a process In the private sector, the process involves conducting research (listening to the customer to ascertain needs and wants), developing products to meet those needs, promoting products, and then re-evaluating product effectiveness Products are changed through innovation or discontinued, as noted below

The group verbalized the process of developing labor market information products and services as being a process of data collection, data analysis and distribution

of information products and services The process has not historically recognized the customer as part of the development process, as noted below

LMI Product Development Model

Section 309 of the Workforce Investment Act of 1998 clearly mandates that customer satisfaction data be gathered in an effort to continuously improve the LMI system The Work Group has been keenly aware of this mandate, and worked toward developing a model which could incorporate private sector concepts into the labor market information creation process

In that regard, the group was less concerned with a process that developed a

single number for measuring satisfaction Rather, the group focused on ways to collect

information regarding how labor market information products can be improved to better

meet the needs of users The resultant model could be seen as follows:

Integrating Customer Satisfaction into LMI Product Development

Based on this fundamental model of incorporating the needs and satisfaction of customers into the LMI creation process, the work group then set about establishing models which could allow this process to thrive in the complex state and federal workforce information development system

In addition to seeking and responding to customer satisfaction input, the Work Group identified the lack of documentation regarding the usefulness of and demand for workforce and labor market information Therefore, the Work Group’s effort focused on developing a process for assessing customer satisfaction with and documenting demand for labor market information

Section III – Customer Satisfaction and Labor Market Information

Modeling the Process

In addition to framing the approach to work with the models noted in Section II, the Work Group established two models to further define the process that states and federal agencies should use in assessing customer satisfaction The first model is for states and is outlined below:

State Product Customer Needs and Satisfaction

The following is a verbal description of this process:

1 The process notes the three fundamentals of creating labor market and workforce information: data collection, data analysis and information distribution

2 Information distribution encompasses both products and services These products and services are the items that are evaluated by the customers

3 Results of the assessment of customer satisfaction are given to an evaluation team at the state level Work Group research determined that these teams already exist in many states as the entity which develops and distributes current information products

4 The evaluation team then makes changes at any one of the three levels of creating labor market information, as it is able depending on funding and other considerations

5 Furthermore, the evaluation team shares information with a customer satisfaction focal point – a central collecting point yet to be determined Training on customer satisfaction best practices and techniques, as well as lessons learned on document preparation, then should be offered to states, perhaps through the Labor Market Information Training Institute

6 Finally, the state evaluation shares information with representatives at the federal level to facilitate improvement or changes to the workforce system

This same process should be used for developing new programs, products and services The second model is noted below The right side of the model below is the same as the state model above, but the model is completed by adding customer satisfaction measurement at the national level and the relationship between federal and state entities

System-wide Customer Satisfaction for Workforce Information

The following is a verbal description of this process:

1 Federal activities are in blue, state activities in green, and those entities which are state and federal cooperative have both colors

2 The process notes the three fundamentals of creating labor market and workforce information as the central activity of both federal and state entities: data collection, data analysis and information distribution

3 The process described above is mirrored at the federal level for national or federal product customers Results are reported to the WIC, which may choose to work through an on-going Customer Satisfaction Work Group For products which have consortia or task forces, satisfaction results will be sent to that entity for change recommendations

4 The entity for making change may either be the Bureau of Labor Statistics or the Employment and Training Administration, which, like the state model, will effect change at any of the three levels of data collection, data analysis or information distribution

5 Federal entities then communicate the customer satisfaction results to the state focal point to facilitate possible improvements in state products and assessment practices

The end result of this entire process is a dynamic, customer-driven labor market and workforce information system that is responsive to customer needs and wants in a changing environment The process embraces not only the concept of assessing customer satisfaction, but also of working (through evaluation teams, federal agencies and the WIC) toward continuous improvement of the system

Components of Assessment

Part of the charter from the WIC was to determine how to encourage support for customer satisfaction assessment from the states Work Group members concluded that the process of assessment would have to be broken into easy-to-follow components to be embraced by states In addition, the group assumed that there would not be one mandatory survey imposed upon the states Therefore, one particular survey instrument

or assessment mechanism would not be recommended, but rather several assessment options would be offered

To meet the objective of encouraging states to participate, the Work Group is providing a handbook entitled “Customer Satisfaction Made Easy for Labor Market Information Professionals.” The information discussed in this section of the report has been discussed by the group and formulated into the development of the handbook

Who to Assess:

The Work Group consulted Section 309 of the Workforce Investment Act of 1998 which clearly indicates the customer groups need to be assessed They include:

• Businesses make decisions about product and financial markets, business location, and

employee recruitment, compensation, and training

• Individuals, including young people and adults, make choices about careers, education,

training, and job search

• Elected Officials and Policy Makers, including Workforce Investment Boards, make decisions

about law, policy, budgets, and regulations

• Program Planners determine what workforce and economic development services to provide,

and evaluate program performance

• Education and Training Providers, including teachers and curriculum specialists, design,

deliver, and evaluate programs that develop students’ knowledge and skills

• Intermediaries, such as parents, counselors, teachers, mentors, placement workers, and

case workers, assist others in choosing education and training opportunities, and in finding employment, and

• Researchers and the Media study how the labor market works, and conduct policy research

The Work Group recommends that agencies place the customer into one of these groups This will facilitate information sharing between states and federal organizations

in terms of how customer groups utilize core products

In addition, the Work Group believes it is important that products be assessed at a customer-familiar level As an illustration, a candy customer sees a chocolate bar The chocolate bar manufacturer sees that same bar as sugar, cocoa, milk and other ingredients In LMI terms, the customer sees a brochure that lists projected job opportunities The LMI shop sees that same brochure as projections data, skills sets and wage records State and federal agencies should ask the customer about the satisfaction level in terms familiar to the customer, but report a level of satisfaction in terms of each core data product

State Evaluation Team:

The models for customer satisfaction noted above feature a state evaluation team This team has three purposes:

o To oversee the customer satisfaction effort at the state level, and incorporate results of that effort into continuous improvement of state products;

o To share information gathered in the customer satisfaction effort, both in terms of process and conclusions, with other states;

o To share information gathered in the customer satisfaction effort with representatives and organizations at the federal level to encourage systemic change

States may want to consider having a publications specialist, an LMI projections professional and an information analyst on the team to represent some of the individuals involved in product production It should be noted that the composition of these teams will vary by state During research developed by the group it was discovered that many states already have such teams that also function as the teams that develop LMI products

Attributes:

A review of “customer satisfaction” surveys at the state and federal levels found that most surveys actually are an attempt to measure customer service The Work Group believes that customer service is one aspect of the total customer satisfaction picture These service-oriented questions include “Was our staff helpful?” “Could you find the