Customer satisfaction report

EUROCONTROL A Centre of Excellence in ATM Training IANS Customer Satisfaction Report Report... Customer Satisfaction Report Introduction This is the 7th report that the Institute devot

EUROCONTROL

A Centre of Excellence in ATM Training

IANS

Customer Satisfaction

Report

Report

ii © Copyright 2006 EUROCONTROL

Copyright Notice

© 2006 European Organisation for the Safety of Air Navigation (EUROCONTROL) All rights reserved

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise

Any other use is subject to prior written consent by EUROCONTROL Request shall be addressed to:

Director of the Institute of Air Navigation Services, 12, rue Antoine de Saint-Exupéry, L-1432 Kirchberg/Luxembourg

The EUROCONTROL Institute of Air Navigation Services aims to provide the services that you want and to make your stay in the Institute as enjoyable as possible All Institute personnel are there to ensure that your stay at the Institute is successful However, if you do have a complaint (or a compliment) please tell us If you are not satisfied with the service we provide or you would like to propose an improvement then please fill out the form at http://www.eurocontrol.int/ians/complaint.html ,

or contact IANS.complaints@eurocontrol.int directly.

Table of Contents

INTRODUCTION 4

ATM TRAINING 5

ATC TRAINING 12

E-LEARNING 13

STAKEHOLDER FEEDBACK 14

COMPLAINTS 15

CONCLUSION 16

Customer Satisfaction Report

Introduction

This is the 7th report that the Institute devotes to the

assessment of the level of satisfaction of its

customers The study on which this report is based

has been conducted in the framework of the quality

improvement programme implemented at the

Institute since end of 1996

Measuring customer satisfaction on a regular basis

is one of the key elements of the quality strategy

implemented by the Institute since 1997

Our objective is to understand exactly how our

customers perceive the Institute and, based on their

level of satisfaction, to improve our services and

products

This report proposes a summary and an analysis of

the data collected in 2005

The customer segments

assessed in 2005

IANS applies a policy to measure the satisfaction of

its customers with all its products and services

These include:

• The ATM course participants

• The ATC course participants

• The E-Learning Students

• Training Sponsors

• Consultancy Bodies

• User Groups

Assessment Tools

The assessment is based on a set of assessment

tools and measurement methods During 2005

further work has been done to increase the integrity

of the data collection

Evaluation Forms

To measure the satisfaction of its customers with

the products and services IANS uses the following

evaluation forms:

• A Student Evaluation Form filled in by the students at the end of a course This evaluation form is used for both the ATM course participants and ATC course participants

• An E-Learning Evaluation Form filled in by a student upon completion of an E-Learning Module

• A generic Stakeholder Feedback Form filled in

by various stakeholders The form is available

at

http://www.eurocontrol.int/ians/feedback.html

Customer Visits

IANS visits a sample of its customers on a yearly basis The customer visits serve as a main source

of feedback on existing products and a source of information for improvements and new products

The customer visits include visits to Air Navigation Service Providers and civil and military State Authorities

Complaint Procedure and Complaint Form

IANS uses an anonymous form, available on its web-site for the collection of complaints related to its products, services and staff A reference to the complaint procedure and associated form is made

in all products and services The complaint form is available at

http://www.eurocontrol.int/ians/complaint.html

ATM Training

The Satisfaction of Students with the ATM Training Products is measured through the use of a Course Evaluation

Form

Measurement

Method

At the end of each

(classroom) course,

students are presented

with an evaluation form

containing a number of

criteria to be assessed

Each criterion can be

given a result between 1

and 4 meaning

unacceptable (1),

unsatisfactory (2), good

(3) an excellent (4)

The complete set of

criteria are grouped in

five main categories:

• Training Content

• Instructional Team

• Documentation

• Equipment

• Administration

The first three categories

are directly related to the

course The last two

categories are directly

related to the supporting

environment and

organisation

Figure 1 Student Evaluation Form

Customer Satisfaction Report

From Forms to

Results

We map the marks of the

students on a scale from

0 (all students mark

Unacceptable) to +100

(all students mark

Excellent) When all

students mark Good

then the result is 75 and

when students mark, on

average, between Good

and Excellent then the

result will be around 85

Likewise, if students fill

in 'all-Uncacceptable'

then the result will be 0

If students fill in

'all-Un-Satisfactory' then the

result will be 25

1

©Copyright 2003 EUROCONTROL

Measuring Satisfaction

1

2

Good 3

Excellent 4 Marks from Students

Result

All 3 All 2

All 1

0

All 4

Figure 2 How we measure satisfaction

IANS aims to meet our objective of being a Centre

of Excellence in ATM Training For us this means

we aim to achieve results on any criterion between

75 and 100 As an example: the overall average (all

courses, all criteria, 3090 forms) for IANS in 2005 is

87.5 – compared to 85 in 2004 The highest result

on a single criterion is 95 (Subject Knowledge of the

Instructor) The lowest result remains pre-course

information (76.5 – compared to 70 in 2004)

Results

The results presented below are extracted from the automated student satisfaction evaluation system Results can be presented per category, per training domain and per criterion We have selected the most significant results to be included in this report

0 500 1000 1500 2000 2500 3000 3500

Figure 3 Number of Evaluation Forms

Size of the Population

This graph shows the number of evaluation forms received and processed by IANS In

2005 the Institute continued to optimise the processing of the forms

This led to another significant increase of the number of processed forms in 2005

Composition of

the Population

Students indicate on the

evaluation form which

type of staff they are

This chart shows that a

majority of Students at

IANS in 2005 are

Operational and

Technical Staff

These results are

consistent with the

results of the previous

years

192

345

977

693

223

600

1298

824

0 200 400 600 800 1000 1200 1400

2004 2005

Figure 4 Composition of the population (number of students)

Customer Satisfaction Report

50 55 60 65 70 75 80 85 90 95 100

Figure 5 ATM Course Results

Satisfaction

This chart shows the satisfaction of the students with the ATM courses The result is obtained by averaging the results obtained on every criterion in the categories Training Content, Instructional Team and

Documentation

A result of 87 indicates that, on average, students mark criteria between Good and Excellent

The increase in satisfaction has been achieved through detailed analysis of the results of each course and introducing corrective action wherever required

50 55 60 65 70 75 80 85 90 95 100

Figure 6 ATM Course Results: per Category

Segmented View

This chart presents the results per measurement Category These are Training Content, Instructional Team and Documentation

IANS has improved the results on each of these three categories during the last four years IANS will continue to analyse the results and make improvements where possible and practicable

‘Best’ Results

Customers often praise

IANS for its staff and

excellent infrastructure

This informal feedback is

confirmed by the course

evaluation results The

criteria ‘Instructor

Subject Knowledge’ and

‘Classroom Equipment’

have consistently

received the highest

marks from the students

during the last three

years

IANS will continue to

invest in the competence

of its instructors by

continuation training and

competency assessment

and it will maintain the

building at its high

standard

50 60 70 80 90 100

Instructor Subject Knowledge

Classroom Equipment

Figure 7 Criteria with ‘Best’ Results

50 60 70 80 90 100

Pre Course

Information

Figure 8 Criteria with less positive Results

Areas for Improvement

IANS continuously looks for opportunities to improve Criteria with the less positive results are the Pre-Course Information and the Course Duration

In 2004 IANS has put in place an action plan to improve the Pre-Course information and significant progress has been made in this domain in 2004 and

2005

The optimum duration for each course will be reviewed on a course-by-course basis

In 2005 IANS changed the layout of the coffee area and installed new equipment in the kitchen

to improve the flow of students

Customer Satisfaction Report

Focus on Safety

IANS is heavily involved

in the development and

delivery of Safety

Courses to support

Safety Regulation and

Safety Management

In 2005 the number of

courses delivered in the

Safety domain increased

from 36 to 50 and the

number of participants

increased from 614 to

844

We are pleased to note

that the results in this

relatively new area for

IANS equal the average

obtained in other, more

mature, training

domains

It is a clear indication

that the rigorous

development approach

at IANS leads to

products that meet

customer expectation

50 55 60 65 70 75 80 85 90 95 100

Figure 9 Safety Domain Course Results

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Figure 10 Would recommend course to a colleague

Measuring Loyalty

In 2004 we introduced a new question on the evaluation form: ‘Would you recommend the course to a colleague’

In 2005, like in 2004, more than 90% of the students would recommend the course that they attended to a colleague