customer satisfaction made easy

A plan for actively listening to customers of Labor Market Information Products and Services A resource handbook from the Customer

Customer Satisfaction Made Easy

Customer Satisfaction Made Easy

A plan for actively listening

to customers of

Labor Market Information Products and Services

A resource handbook from the

Customer Satisfaction Work Group of the Workforce Information Council

Copyright 2003

Introduction – Why Measure Customer Satisfaction? 1How to Measure Customer Satisfaction 2-3Step 1 – Pick a Product or Customer Group 4-6Step 2 – Set the Stage for Assessment 7-31

What do you want to find out? 7-10

Sample Problem Definition Statement 8-10

How you can best find the answers? 10-31

Quantitative Approach 13-14Guide for Using Focus Groups 16-18Guide for Using A Mail Survey 19-20Guide for Using A Telephone Survey 21-23Guide for Using a Personal Interview 24-26Guide for Using an Internet Survey 27-29

I n t r o d u c t i o n –

Why measure customer satisfaction?

Because it’s required! The Workforce Investment Act of 1998

information disseminated through the statewide employment statisticssystem, in order to continuously improve the system.” The two keyparts of this statement are:

• Consult with customers

• Continuously improve the system

It’s also a requirement of the One Stop LMI grant from the Employmentand Training Administration!

of increased accountability in government, every activity needs to provethat taxpayers’ money is used efficiently and effectively To ensure thatLMI products retain their value to customers, we need to continuouslyask customers if our products are meeting their

needs We also need to create new products that are

responsive to customer demands Asking our

cus-tomers about our products provides proof that LMI

products are a good investment

limited funding, we have to make sure that every

dol-lar we spend on developing and disseminating

infor-mation products (whether its for printing or updating

a website) is being spent on a product people will

use The BEST way to make sure our products are

being used is to ask the people who use them

Customer satisfaction assessment is a great opportunity for states to make positive changes in the future

of the labor market information system!

1

How to measure

customer satisfaction?

This handbook will help states make customer

satisfaction assessment as easy as 1, 2, 3! In order to

make it this easy, there are some things you should do

and should know.

Form an evaluation team. The team will oversee the collection of customer satisfaction feedback and then perform three responsibilities.

1 Make changes to products and services based on that feedback;

2 Share satisfaction processes and results with other states;

3 Share satisfaction processes and results with federal

representatives to make systemic changes.

Consider having a publications specialist, an analyst and a projections professional on your team This team may already exist as the group who deter- mines your LMI product content

Customer satisfaction assessment does not

necessarily have to be a statistically rigorous exercise.

The process outlined in this handbook is based on solid descriptive research methodology, but it is different from the types of research normally conducted in “LMI shops.” While you may have to meet strict standards for response rate based on a stratified random sample to meet the requirements of the Bureau of Labor Statistics, customer satisfaction assessment is based on a

“dialogue” with your customers This dialogue may not be conducted the way you would conduct a BLS survey.

You should document the demand for your

products and services

In addition to finding out if customers are satisfied with your products, find out how many are actually using them, how they are using them and where those users are located We have a special section at the end which addresses this issue.

2

Create Customer Lists

Many of the processes outlined in this handbook will require you to have

a list of your customers You may want to consider creating these lists if you do not have them already These are lists of customers who receive or use a partic- ular document or LMI product you produce

You have many tools in the toolbox of customer

satisfaction research

You may want to conduct a survey, you may want to have a focus group

or conduct interviews with your main customers of a particular product.

Remember, when you ask just one customer their opinion of a product, you have started to listen to the customer When you change a document based on what a customer said, you are engaged in the process of continuous improve- ment This leads to the final point:

You can’t do it all at once!

This handbook will walk you

through conducting customer

satisfac-tion research At first, it may seem a

daunting task to find out what all your

customers think about all of your

prod-ucts But, you have to start somewhere

and you can’t do it all at once Remain

calm and dig in!

3

Pick a Product or Customer Group

State LMI providers should be producing ments for particular customer groups According to theWorkforce Investment Act, states should be serving thefollowing customer groups:

docu-• Businessesmake decisions about product and financial markets, ness location, and employee recruitment, compensation, and training.

busi-• Individuals, including young people and adults, make choices about careers, education, training, and job search.

• Elected Officials and Policy Makers, including Workforce Investment

Boards, make decisions about law, policy, budgets, and regulations.

• Program Planners determine what workforce and economic ment services to provide, and evaluate program performance.

develop-• Education and Training Providers,including teachers and curriculum cialists, design, deliver, and evaluate programs that develop students’ knowledge and skills.

spe-• Intermediaries, such as parents, counselors, teachers, mentors,

placement workers, and case workers, assist others in choosing cation and training opportunities, and in finding employment, and

edu-• Researchers study how the labor market works, and conduct policy research The mass media may be included in this category.

Take a look at the LMI products

and services your state produces

Which customer groups use each of the

products? Create a grid, like the

sam-ple that follows, which shows the

cus-tomer groups and the products you

pro-vide for them

Step

1

4

At each intersection on the grid,

you should be assessing satisfaction.

Once you identify which customers are using what products and services, you can measure customer satisfaction by starting with either of two paths:

• start with a product, or

• start with a customer.

As noted on the previous grid, if your state is about to re-publish a Career Handbook, you may want to ask people in the three customer groups that use the document if they are satisfied with it In that case, you would be starting with a product

You can be in tune with the changing needs of customers by ing needs assessment without discussing an existing product For example, each year employment statistics agencies must confer with local workforce investment boards to discuss information needs This is an opportunity to conduct a needs assessment Ask them about how they use data in their core business functions and if workforce information could help them do their work This may help you prioritize which information products and services you provide and customize those products for their specific needs.

conduct-6

Set the Stage for Assessment

There are two things you need to do in step two:

1 determine what you want to find

out, and

2 determine how best to find the

answers!

The rest of this step walks you through

this process and gives you examples

What Do You Want to Find Out

You should develop an Assessment Statement This merely helps you “think through” your descriptive research process It may seem obvious when you complete it, but it will help guide you This is what should be in and should

be answered in your statement:

• A general description what you want to find out and from whom.

• A basic question the research needs to answer

• What you want to determine in your research?

• Management objectives, such as:

✓ Why you are doing the research?

✓ What will be the uses of the information you learn?

✓ Why is it important that you do the research?

✓ What is the general setting for the research?

✓ Who are you going to assess?

• What data collection technique will you use and why?

• When will you do it?

• How much time/money can you spend doing it?

• Who will do it?

The following pages provide an example of a completed Assessment Statement.

DETERMINING THE ATTITUDES

Step

2

7

OF CAREER COUNSELORS CONCERNING

CAREER GUIDES AND POSTERS

A s s e s s m e n t S t a t e m e n t

General Description: To assess satisfaction with career guidance

mate-rials delivered by the State Department of Labor (DOL) and determine methods for improving those products for customers.

Basic Research Question: In what ways can career guidance materials be

improved to assist those who advise students on career goals?

Research Information Questions:

- To determine the level of satisfaction with existing career materials, specifically Guides to Career Choices and accompanying posters;

- To determine the most effective attributes of career materials which s h o u l d b e r e t a i n e d i n

Management’s Objectives:

Why is this research being done?

We are required to assess customer satisfaction with products as part of its funding grant from the U.S Dept of Labor, Employment and Training Administration (ETA), and needs cus- tomer satisfaction input to guide its continuous improvement efforts This assessment is in par- tial fulfillment of this obligation We also want

to create a more useful product.

8

SAMPLE

What are management’s intended uses of the information?

DOL management will document comments from customers and, as necessary and fiscally possible, make changes to career information materials.

How important is the research?

Limited resources to fund printing require DOL management to ensure that only the most useful information is published.

What is the general setting for the research?

Labor Market Information has been published by DOL for many years under cooperative agreements with the U.S Dept of Labor, Bureau of Labor Statistics and Employment and Training Administration Career guidance materials are part

of meeting these and other mandates (e.g., the Workforce Investment Act of 1998) to serve edu- cators DOL wants to ensure that this key target group is being effectively served.

Who is to be assessed?

The career guidance personnel in two vocational districts in the state will be invited to participate in focus groups.

Data Collection Technique: Because the group members can be identified by

name, a focus group will be held in each of two districts with approximately 7 to 12 persons in each group Most of the questions will be open- ended to allow for maximum response by the per- sonnel One group is from a suburb The other group is from the more rural area The two career guidance centers in the above-mentioned commu- nities will be contacted for a list of names of per- sonnel who use the materials These individuals will be invited to attend the focus group until a maximum of 12 people are confirmed in each loca- tion The focus groups will be held at the voca- tional offices in the communities noted.

SAMPLE

Researchers: The focus groups will be conducted by

an independent research company.

How You Can Best Find the Answers

Notice that in the above example a focus group was selected There are many options in addition to focus groups The Customer Satisfaction Work Group recommends one or a combination of these five:

10

A GUIDE FOR USING

A QUALITATIVE APPROACH

General Description and Purpose

This is an approach that is primarily designed to gather informationusing open-ended questions to allow for more in depth interviewing

Appropriateness

When you want more insights into a question or general opinions,this is a good method to use

Recommended LMI Products or Services

This can be for all products and services that you provide

Types of Questions

Normally open-ended questions are used (Such as “how do you usethese” or “how do you feel about this.”) However, do not hesitate toask closed-ended questions but not too many (Such as “how manystudents do you serve.”)

Customer Groups

All customer groups are appropriate

Selection of Customers

A nonrandom, subjective method may be better than a random,

objective method Select people who you feel will share feelings and ideas

Typical Response rate

Unless you are asking the open-ended question of a person, you mayfind that many people do not answer open-ended question on a

Method of Conducting the Exploratory Study

There are many methods available Focus groups are commonlyused However, you can do individual interviews This differs from

a focus group since there is not the dynamics of reaching group census Another technique is the “think aloud” method Commentcards can also provide insights

con-Buying a program to document and tabulate the open-ended

responses can be beneficial

Benefits of Using this Approach

You can obtain in-depth customer feedback and gain an ing of how information is actually used by the customer You havemuch flexibility in your approach The approach can be very infor-mal Using a complex sampling method is not needed You alsowill not normally need, nor want, a large sample size

understand-Cautions in Using this Approach

If you are doing a focus group or interviews, you cannot project theresults to a larger group of people If you are analyzing the results

of open-ended questions asked on a quantitative survey, you canproject the results to the larger group The problem becomesanalyzing the answers in some meaningful manner

12

A GUIDE FOR USING

A QUANTITATIVE APPROACH

General Description and Purpose

The quantitative approach is used when you want to “count” a certaincharacteristic and project the results to a defined population or cus-tomer group

Appropriateness

This method is most appropriate when you have a randomly selectedgroup of persons In addition, having a fairly large sample is very helpful

Recommended LMI Products or Services

All LMI products and services can be evaluated using a quantitativebasis

Types of Questions

Normally, closed-ended questions are used in a quantitative approach.But one can count the number of ideas generated from open-endedquestions Behavior, attitude or demographic questions can all beasked using this approach

13

more than 5.6% If you have a small number in a customer group such

as 200 on a mailing list, survey all persons

Typical Response rate

By surveying a large number of persons, we expect that not everyonewill respond Response rates of 10% to 50% can be expected depending

on the customer group surveyed

Method of Conducting the Quantitative Study

You can use any method: mail, telephone, Internet, self administered(hand out), computer-assisted, Web page analysis, or comment cards

Typical Costs

Quantitative methods normally involve more persons than a qualitativeand therefore cost more You may need to purchase special computerpackages for tabulating and analysis

Time Required

Normally, the approach will take longer

than a qualitative approach

Staffing Needs

People with different skills in developing

questions, collecting data, entering data,

analyzing data and writing reports will

be needed

Benefits of Using this Approach

You will be able to provide consistent

information from one time period to

another By taking a sample, you can project results to the entire tomer group

cus-Cautions in Using this Approach

This approach will normally take more time and cost more than a tative approach 14

quali-FREQUENTLY ASKED QUESTIONS ABOUT CUSTOMER ASSESSMENT METHODS

How likely are people to respond to different techniques?

(1 is the best and 5 the worst)

CHARACTERISTIC

Respondent Contact 90% 95% 30% 30% 95% Respondent

Highest quality

Focus Group MAIL T Focused Personal Interview Internet

15