A handbook for project managers, developers, implementers, evaluators and donors working to counter trafficking in persons. ppt

Handbook on Performance Indicators for Counter-Trafficking Projects Chapter 3: Developing Performance Indicators for Counter-Trafficking Projects Initial Considerations Now that the log

Handbook on Performance Indicators for Counter-Trafficking Projects

A handbook for project managers, developers, implementers, evaluators and donors working to counter trafficking in persons.

The International Organization for Migration (IOM) is committed to the principle that humane and orderly migration benefits migrants and society As an intergovernmental body, IOM acts with its partners in the international community to: assist in meeting the operational challenges of migration; advance understanding of migration issues; encourage social and economic development through migration; and uphold the human dignity and well-being of migrants.

Publisher: International Organization for Migration

17 route des Morillons 1 1752 N Street NW, Suite 700

Tel: +41 22 717 91 11 +1 202 862 1826

Fax: +41 22 798 61 50 +1 202 862 1879

Internet: http://www.iom.int http://www.iom.int/unitedstates

ISBN 978-92-9068-440-4

©2008 International Organization for Migration (IOM)

Design: Erin Dridi

Cover art: Thomas Mahoney

All rights reserved No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise without the prior written permission of the publisher

I

This handbook was made possible through the funding and support of U.S Department of State’s Bureau

of Population, Refugees, and Migration (PRM) Technical input was provided by State Department staff for this handbook, which was also shared for comments with the U.S Departments of Health and Human Services (HHS) and Labor (DOL), the Office of Management and Budget (OMB) and the United States Agency for International Development (USAID) Principal drafters and contributors to this handbook included Rosilyne Borland, Sarah Craggs, Kristin Dadey, Richard Danziger, Timor El-Dardiry, Christophe Franzetti, Lidia Futter, Harvy Gadia, Ashley Garrett, Tatiana Ivanyuk, Frank Laczko, Fred Larsson, Par Liljert, Amy Mahoney, Agueda Marin, Jonathan Martens, Ruzayda Martens, Virginia Martinez, Chissey Mueller, Malathy Nagasayee, Anh Nguyen, Helen Nilsson, Monika Peruffo, Thien Trang Nguyen Phan, Lua Pottier, Anna Eva Radicetti, Olga Rakhmatullova, Jo Rispoli, Vivita Rozenbergs, Shpetim Spahija, Miwa Takahashi, Ani Zamogchian with the support of Jeff Labovitz, Richard Scott and Yorio Tanimura Special thanks to IOM staff who attended drafting sessions in Geneva and Kiev for your feedback and support throughout this project It could not have been done without you

ADS Automated Directives System

CT Counter-Trafficking

DOJ United States Department of Justice

EC European Commission

EU European Union

GPRA United States Government Performance and Results Act 1993

GAO United States Government Accountability Office

GTIP United States Department of State’s Office to Monitor and Combat Trafficking in Persons

ICMPD International Centre for Migration Policy Development

IFAD International Fund for Agricultural Development

IOM International Organization for Migration

LE Law Enforcement

M&E Monitoring and Evaluation

MLA Mutual Legal Assistance

MOU Memorandum of Understanding

MoV Means of Verification

NCC National Coordination Committee

List of Acronyms

II

NGO Non-Governmental Organization

NPA National Plan of Action

OECD Organization for Economic Cooperation and DevelopmentOMB United States Office of Management and BudgetOSCE Organization for Security and Cooperation in EuropePPP Prevention, Protection, Prosecution

PRM United States Department of State’s Bureau of Population, Refugees, and MigrationSOP Standard Operating Procedures

TIP Trafficking in Persons

UNODC United Nation Office for Drugs and Crime

UNDP United Nations Development Programme

UNHCHR United Nations High Commissioner for Human RightsUNHCR United Nations High Commissioner for RefugeesUNICEF United Nations International Children’s Emergency FundUSAID United States Agency for International DevelopmentVoT Victim of Trafficking

Table of Contents

List of Acronyms II

List of Terms IV

INTRODUCTION 1

What is the handbook? 1

Overview of chapters 2

CHAPTER : Trafficking in Persons 3

Responding to the challenge 4

Why are performance indicators important in counter-trafficking? 7

CHAPTER 2: Conceptual Framework for Developing Performance Indicators in Counter-Trafficking 11

The logical framework 12

CHAPTER 3: Developing Performance Indicators for Counter-Trafficking Projects 15

Initial considerations 15

How to use the matrix 17

The matrix 20

CHAPTER 4: Data Collection 44

Data sources 44

Data collection 45

Data quality 47

Limitations for data collection 48

Conclusion 49

Bibliography 50

Annex A 56

Annex B 69

III

List of Terms

Activity – An important task that is seen as an essential step to achieving a result These are the

actions proposed to achieve the result

Data – (plural of datum) Individual facts, statistics or items of information.

Evaluation – The systematic and objective assessment of an on-going or completed project,

program or policy, its design, implementation and results The aim is to determine the relevance and fulfillment of objectives, development efficiency, effectiveness, impact and sustainability

Indicators – Quantitative or qualitative benchmarks that provide a simple and reliable basis for

assessing achievement, change or performance They are a means of analyzing and monitoring the characteristics of operations, services and processes, and their implementation In addition, they can also be used to measure, monitor, evaluate and improve performance

Logical Framework – A methodology for conceptualizing projects and an analytic tool that allows

a project developer / manager to detail a project clearly and understandably

Means of Verification (MoV) – The evidence behind the indicators – i.e the documentation to

prove that the measure given by the indicator has been achieved on the overall objective, project purpose and results levels

Monitoring – A management tool that contributes to effective and efficient project

implementation It can be defined as a continuous function that uses systematic collection of data

on specified indicators to provide management and stakeholders with indications on progress and achievement of objectives, and to take action to improve performance

Overall Objective – A single statement of the broader aim of a project, i.e how the project can

contribute to a larger national or international development plan or action

Performance Indicator – Pre-determined measurements that track specific changes or results of a

project Performance indicators are directly linked to measuring progress toward project objectives and are often a combination of monitoring and evaluation

Personal Data – Includes material circumstances of data subjects, as well as supporting documents

that verify identification (see Annex B for more information)

Project Purpose – The objectives that will be directly achieved by the project They define the

primary reason for the project by directly addressing the problem

Result – Lists direct and measurable results expected from the project activities They should be

tangible, visible and measurable means of achieving the project purposes

IV

Stakeholders – These are agencies, organizations, groups or individuals who have a direct or indirect

interest in the counter-trafficking activities and responses, and who affect or are positively or negatively affected, by the implementation of activities Stakeholders could include governments, donor communities, implementation partners, businesses and project beneficiaries

Trafficking in Persons – The recruitment, transportation, transfer, harbouring or receipt of persons,

by means of the threat or use of force or other forms of coercion, of abduction, of fraud, of deception,

of the abuse of power or of a position of vulnerability or of the giving or receiving of payments or benefits to achieve the consent of a person having control over another person, for the purpose

of exploitation Exploitation shall include, at a minimum, the exploitation of the prostitution of others or other forms of sexual exploitation, forced labour or services, slavery or practices similar

to slavery, servitude or the removal of organs

Vulnerable Groups – This includes any group or sector of society that is at higher risk of being

trafficked due to issues such as having an irregular migration status; conflict or war; natural or manmade disasters; being refugees, internally displaced persons or stateless; discriminatory practices; violence in the home; and economic hardship, etc For the purposes of counter-trafficking, it is important to consider those persons more susceptible to trafficking in persons, including former victims, in project development and implementation

V

In 2004, a new initiative began between the International Organization for Migration (IOM) and the U.S Department of State’s Bureau of Population, Refugees, and Migration (PRM) to develop performance indicators for counter-trafficking (CT) projects As part of continued efforts to improve the quality and assess impact of its counter-trafficking work, IOM has worked closely with the U.S Government to develop performance indicators within the programming context of trafficking in persons This effort has been field and experience driven with the goal of developing monitoring and evaluating systems and establishing institutional guidelines for measuring the short, medium and long-term impacts of counter-trafficking work It is hoped that this handbook will be useful for government and non-governmental stakeholders and service providers for establishing performance indicators for counter-trafficking projects that can be compared across countries and regions, and ultimately assist in effective and successful programming Development of this handbook rests upon IOM’s worldwide experience of more than 13 years in the area of counter-trafficking, where it has implemented some 212 projects in 84 countries and assisted over 14,000 victims of trafficking Various definitions, logical frameworks, IOM project documents and monitoring and evaluation structures were compared and reviewed for this project This included those used by several governmental and inter-governmental agencies that have vast project experience in the area of counter-trafficking

VI

Handbook on Performance Indicators for Counter-Trafficking Projects

What Is the Handbook?

This handbook is a resource and guide for project managers, developers, implementers, evaluators and donors working in the field of counter-trafficking, who want to develop performance indicators for their counter-trafficking projects It is not intended to be a comprehensive set of guidelines for project development It is meant to be used by any organization or donor to help define and incorporate indicators that measure project performance within the framework of prevention, protection and prosecution

Having data is essential to make decisions in designing, re-designing, reviewing, revising or updating counter-trafficking projects Valid and reliable information can provide useful insight into the background and mechanisms of the human trafficking process Such information can also help identify gaps and needs in the provision of assistance, which can affect the formulation and implementation of targeted measures to improve conditions Data and monitoring can help shape a project by highlighting situations that may require corrective action, fill an identified gap or highlight something that is working well that should be replicated

Performance indicators help measure change occurring in the fight against human trafficking as a result

of targeted activities, recognizing that the overall goal of all counter-trafficking projects is to reduce and ultimately eliminate trafficking in persons The performance indicators included in this handbook can serve as practical guidelines to measure progress toward project goals In the process of monitoring and evaluation, an analysis of a general set of indicators can not only help determine a specific project’s short, medium and long-term impact, but can allow the testing and comparing of project results across regions

so that there is better measurement of real “best practices” which have proven impact It is hoped that the included performance indicators will help shape the process of standardizing some base performance indicators that will then allow for comparable counter-trafficking data for projects across regions

Introduction

A certain level of knowledge regarding trafficking in persons, project development and monitoring and evaluation is assumed for those who use this handbook For more information on these topics, please refer to some of the resources cited in the bibliography

note

Overview of Chapters

This handbook is divided into four chapters The first chapter presents the general framework for trafficking projects as stated in the United Nations Convention against Transnational Organized Crime’s supplementing Protocol to Prevent, Suppress and Punish Trafficking in Persons, Especially Women and Children (UN Protocol), and details why performance indicators are important in counter-trafficking The second chapter provides basic definitions and the logical framework used in this handbook for developing counter-trafficking performance indicators The focus of this chapter is on two of the four levels in a logical framework (project purpose and results), in order to highlight the advantages

counter-of using such indicators to evaluate the performance counter-of counter-trafficking projects The third chapter describes how performance indicators are developed and applied in project design and supervision, and discusses important issues related to the meaningful use of indicators It also includes the matrix that contains the performance indicators chosen for this handbook, an explanation on how to use the matrix in developing performance indicators and tips for adapting it to a project’s local context within the framework of prevention, protection and prosecution Chapter four covers data collection related to counter-trafficking projects, including limitations to consider

Introduction

Handbook on Performance Indicators for Counter-Trafficking Projects

Human trafficking is a rapidly expanding global phenomenon that impacts countries and communities throughout the world.1 Though governments and non-governmental organizations are increasingly responding to the problem of human trafficking, there is still much to be done There are lessons to

be learned from both the successes and challenges of counter-trafficking efforts globally Measuring performance by establishing indicators that are comparable across countries and regions is an important step in sharing lessons learned and for measuring the impact of counter-trafficking projects However, before discussing performance indicators by which to evaluate counter-trafficking projects, it is necessary

to understand the basic concepts related to trafficking in persons

In 2000, the international community developed a definition of trafficking in persons as a supplemental protocol to the United Nations Convention Against Transnational Organized Crime.2 Article 3 of the United Nations Protocol to Prevent, Suppress and Punish Trafficking in Persons, Especially Women and Children, was signed in December 2000 in Palermo, Sicily, Italy The text reads as follows:

(a) “Trafficking in persons” shall mean the recruitment, transportation, transfer, harbouring or receipt

of persons, by means of the threat or use of force or other forms of coercion, of abduction, of fraud,

of deception, of the abuse of power or of a position of vulnerability or of the giving or receiving of

payments or benefits to achieve the consent of a person having control over another person, for the

purpose of exploitation Exploitation shall include, at a minimum, the exploitation of the prostitution

of others or other forms of sexual exploitation, forced labour or services, slavery or practices similar

to slavery, servitude or the removal of organs;

(b) The consent of a victim of trafficking in persons to the intended exploitation set forth in

subpara-graph (a) of this article shall be irrelevant where any of the means set forth in subparasubpara-graph (a) have

been used;

(c) The recruitment, transportation, transfer, harbouring or receipt of a child for the purpose of

ex-ploitation shall be considered “trafficking in persons” even if this does not involve any of the means

set forth in subparagraph (a) of this article;

(d) “Child” shall mean any person under eighteen years of age

This chapter has been adapted from the IOM Counter-Trafficking Training Modules.

Official text of the Convention and the Protocols can be downloaded at http://www.unodc.org/unodc/en/crime_cicp_convention.html

1.

2.

Chapter 1: Trafficking in Persons

The UN Protocol came into force 25 December 2003 and currently has 117 countries as signatories and

118 countries as parties to the Protocol.3 The Protocol is an important international instrument that defines and standardizes counter-trafficking terminology.4

The Protocol requires States to:

Criminalize trafficking in persons

Provide assistance and protection to victims in countries of origin, transit and destination

Assist in the repatriation of victims

Manage migration to prevent and detect human trafficking (e.g border control, travel documents)Provide training, research and information to prevent and counter trafficking in persons

Abide by technical provisions (related to signature or ratification)

Responding to the challenge

Trafficking in persons is a global problem that affects hundreds of thousands of people every year.5 Responding to human trafficking is a daunting task for governments, policymakers and organizations that assist victims What can be done? Comprehensive counter-trafficking activities often focus on three broad objectives:

1 Prevention

2 Protection

3 Prosecution

Children as a special case

The UN Protocol recognizes the special situation of children in Article 3 sub-paragraph (c), which removes the need for “means” to be present to be considered trafficking in persons under the UN Protocol In other words, for children it is not necessary that there be “threat or use of force or other forms of coercion, of abduction, of fraud, of deception ” to be considered trafficking in persons For example, if a parent arranges to have their child transported into a situation where they are forced to work (e.g begging on the streets), this can be considered trafficking, even if the child is a willing participant and is returned after a period of time It is important to note, however, that national laws on child labour vary greatly, and may not agree with the standards set in the Protocol

As of April 2008 Updated numbers available at http://www.unodc.org/unodc/en/crime_cicp_signatures_html

The UN Protocol refers to international trafficking in persons However, the UN Legislative Guide for the Protocol does elaborate on internal trafficking

Exact figures for trafficking in persons are difficult to determine and estimates vary The U.S Government estimates that 600,000 – 800,000 people are trafficked annually worldwide According to UNICEF, 1 million children are trafficked every year The UNODC states that the annual profit in trafficking in persons is USD 8 billion, comparable to figures for drug trafficking.

Handbook on Performance Indicators for Counter-Trafficking Projects

The complexity of human trafficking requires a holistic, human rights-based response that is both broad enough to address the problem on multiple levels and specific enough to make sense in the local context Human traffickers look for existing migratory flows and potential migrant groups that can be exploited While trafficking often involves crossing national borders, there are also significant numbers

of victims that are trafficked within their own national borders Identifying the local context and the specific mechanisms of trafficking that are taking place within a country is essential to creating a strategic response

to provide alternatives to groups that could be vulnerable to trafficking in persons, for example A transit country could work on strengthening border control and providing training to officials to enable them

to identify sub-groups of trafficked persons within larger migrant groups moving through their country, legally or illegally Destination countries might look at exploitation taking place within their borders, and work to strengthen the labour rights of migrants, or could work to facilitate legal flows of migrant workers

to lessen the demand for irregular migrants Countries with internal trafficking issues might strengthen education or create employment programs in vulnerable communities that provide opportunities

In addition to activities designed to prevent trafficking in persons, protecting victims is an essential part

of any counter-trafficking response Protection includes providing immediate protection for potential

or identified victims of trafficking It also means keeping identified victims safe while meeting their immediate basic needs (e.g shelter, medical, psychological and psychosocial care, food and clothing) Following this, interim care and return and reintegration/integration options should be discussed This includes options for settlement in the destination area, third country or in another part of the victim’s home country as appropriate Keeping victims safe also requires a high level of confidentiality when managing cases – the identity of victims must be protected This is essential not only to protect victims and service providers from traffickers, but also to protect victims from potential stigma within their families or communities

Protection also means creating an environment (social, political and legal) that fosters the protection

of victims of trafficking This could mean creating special temporary or permanent visas that allow victims from other countries to remain legally in the country of destination (in some cases in exchange for cooperation with authorities) Appropriate legislation can be used not only to prosecute traffickers, but also to protect victims Access to the judicial system grants victims the opportunity for justice and compensation Emphasis must be given to the fact that victims of human trafficking are victims of crime and must be recognized and treated as such - not as criminals themselves

Protection could also involve helping victims return to their home country or community – safe voluntary return and reintegration/integration This includes providing an individualized, victim-centered response

to help reduce a person’s vulnerability to re-trafficking

Prosecuting traffickers is another important part of a comprehensive strategy to combat trafficking in

persons Strengthening legislation and policy to hold traffickers accountable for their crimes is necessary

in order to have an effect on traffickers In some countries, the assets of traffickers are confiscated and used to fund victim assistance projects International cooperation is extremely important, especially when the crime is transnational in nature

Finally, it is important to keep some crosscutting themes in mind when working in counter-trafficking, such as:

to human trafficking is a daunting task, it is a challenge that the global community must continue to address through increased, improved and sustainable counter-trafficking measures and enhanced cooperation This will be the focus of the indicators in the matrix (Chapter 3)

For the purposes of this handbook, the performance indicators related to prevention, protection and prosecution, or the “Three Ps,” will be looked at in the following substantive categories (for further information see Chapter 3):

6.

Chapter 1: Trafficking in Persons

Handbook on Performance Indicators for Counter-Trafficking Projects

Why Are Performance Indicators Important In Counter-Trafficking?

Obtaining reliable data on trafficking in persons is admittedly difficult This is for a variety of reasons, including the fact that human trafficking is a clandestine activity and many cases go unreported Cases that do surface may not receive priority by law enforcement or, if there are no specific or stringent laws

on trafficking, a country may report a case under another heading The capacity to collect and maintain data may be weak or there may be no organized response in a community/country There is also a general lack of information sharing between organizations, agencies and governments Even when information

is shared, often the methodology for obtaining data is unclear Regardless “[w]e need to move beyond stating that trafficking is a problem to assessing in more detail how well we are dealing with this problem

If our understanding of trafficking is to improve, we also need to find ways to generate much better data and indicators of the problem.” 7

Working to counter trafficking in persons is important and efforts have undoubtedly achieved the goal

of raising awareness and providing assistance for victims of trafficking and their families in a variety of countries and contexts However, currently, there is no comprehensive method of measuring performance

in the field of counter-trafficking While some reliable data can be gleaned from specific projects, there is

International Organization for Migration (IOM), Data and Research on Human Trafficking: A Global Survey, 2005, p14.

7.

Performance Indicator Matrix Structure

Policy and Legislation

Root Causes and Vulnerabilities Cooperation

Policy and Legislation Direct Assistance

Cooperation

Policy and Legislation

Criminal Justice System

Cooperation

little in the way of systematized data gathering and analysis on human trafficking This makes it difficult

to measure the impact of counter-trafficking efforts and to see if the response has been successful on a large scale To date, the global monitoring and evaluation of counter-trafficking projects has generally been non-standardized and output (not impact) focused Consequently, the questions remain: Has trafficking in persons reduced? Are we succeeding in our tactics? What is and is not working? How can

we adapt activities in one community/country to complement another? Has vulnerability been reduced?

Is reintegration generally successful? What are the medium and long-term impacts of activities on the community and individual levels?

The aim of this handbook is to provide the first steps in developing performance indicators for trafficking projects that not only allow for results and purposes of individual projects to be measured, but also to create a method of global comparability to help understand the overall impact By doing so, efforts can be adjusted as needed at local, national, regional and global levels through similar methods

counter-of measurement However, before exploring specific counter-trafficking performance indicators, it is important to discuss terminology

Indicators are quantitative or qualitative benchmarks that provide a simple and reliable basis for

assessing achievement, change or performance They are a means of analyzing and monitoring the characteristics of operations, services and processes, and their implementation In addition, they can also be used to measure, monitor, evaluate and improve performance.8

An indicator is generic; it could be anything such as the number of people trained or the number of posters printed However, performance indicators seek to measure specific changes or results of a project So, instead of measuring just what was printed or what was done, we are measuring a change that actually took place, which implies that the project had some sort of impact

Performance indicators are pre-determined measurements that track specific changes or results of a

project Performance indicators are directly linked to measuring progress toward project objectives and are often a combination of monitoring and evaluation

Adapted from the IOM Counter-Trafficking Training Modules.

8.

Chapter 1: Trafficking in Persons

We recognize that the term performance indicator can be used in other ways However, we are using the term as defined above in this handbook

important

Handbook on Performance Indicators for Counter-Trafficking Projects

During the lifetime of a project, performance indicators:

show how activities are progressing in achieving project purposes and results;

enable the identification of problems at an early stage; and

allow for corrective measures

Performance indicators are also essential upon completion of project activities, because they serve as a tool to demonstrate if and to what extent achievements/results correspond to set targets and purposes and in some cases, provide a detailed set of preliminary data for measuring impact

Performance indicators can also be used for advocacy purposes, such as promoting a service provided by the project For example, an indicator that reveals increased service delivery can support the argument that additional capacity and resources are needed to adequately meet the increased demand

At the same time, the quantitative and qualitative information used to verify the performance indicators helps in the identification of weaknesses and consequently in devising ways to improve project planning,

as well as the provision of services

Performance indicators are therefore important in that they enable institutions to gauge their own performance and contribute to accountability Independently of whether they are quantitative or qualitative, indicators help those involved in projects – implementers, beneficiaries, donors – to focus on specific evidence-based results

Performance indicators need to be specific, measurable, attainable, relevant and trackable (SMART) If too ambitious, for instance, they are not useful as it could become difficult to assess what is realistically achievable and attributable to the project.9 Thus, measuring performance can help to “tell a story” of a project’s progress and achievements, particularly for key stakeholders such as governments, the donor

It is important to keep in mind that performance indicators are envisioned prior to implementing

a project However, project activities may result in measurable changes that were not foreseen For example: an awareness raising advertisement is put in a magazine as part of a counter-trafficking information campaign As a result, the magazine polls its readers and the respondents comment

on the awareness campaign’s message This was not part of the campaign’s planed activities, but the response does provide information related to the impact of the campaign’s message This, therefore, qualifies as a performance indicator

note

of the project is clear If the target is not met, information as to why can help the project manager and the donor address the reasons for this and adjust the project or future projects as needed.

The ability to communicate the achievement of results, share lessons learned and compare project impact across regions is dependent on the ability to collect useful performance information

Chapter 1: Trafficking in Persons

Handbook on Performance Indicators for Counter-Trafficking Projects

Chapter 2: Conceptual Framework for Developing

Performance Indicators in Counter-Trafficking

Before introducing specific performance indicators, it is important to go over some basic definitions related to monitoring, evaluation and the logical framework used for this handbook

Monitoring is a management tool that contributes to effective and efficient project implementation It

can be defined as a continuous function that uses the systematic collection of data on specified indicators

to provide management and stakeholders with indications on progress and achievement of objectives, and to take action to improve performance.10 The monitoring of activities is meant to ensure that inputs through activities are transformed into results.11

During monitoring, findings should be used to modify implementation if such actions seem warranted in order to achieve the intended results Monitoring also allows for an identification of previously unknown trends and issues that have come up during the implementation of a project that could have substantial impact It thereby provides the basis for evaluation and review.12

While monitoring allows the assessment of project implementation, which adjusts to changes in the local context, evaluation allows the assessment of impact, which measures changes in behavior in order

to modify project activities and to measure success

Evaluation is the systematic and objective assessment of an on-going or completed project, program

or policy, its design, implementation and results The aim is to determine the relevance and fulfillment of objectives, development efficiency, effectiveness, impact and sustainability Evaluation also refers to the process of determining the worth or significance of an activity, policy or program.13 Evaluations assess the project’s progress vis-à-vis the project purposes and overall objective and looks at what the overall performance and success of the project has been.14

The main objectives of evaluation are to improve decision-making, resource allocation, accountability and the development of future projects Therefore, evaluation should measure and describe what the activity or service has accomplished and the impact it has had, compare the progress to the original intention, analyse the reasons for what happened or the changes that occurred and, based on the answers to these questions, recommend actions for decision-makers

IOM, Evaluation Guidelines, p10 op cit

IOM, Project Development Handbook, January 2005, p77.

PRM, Chapter 7, Monitoring, Monitoring and Evaluation Guidance, Monitoring and Evaluation Workshop 15 – 19 August 2005

IOM, Evaluation Guidelines, p4, op cit

IOM, Project Development Handbook, p77 op cit.

While monitoring focuses on project management directly (helping to answer questions such as: Are

we on schedule? Are we on budget?), evaluation is more concerned with the consequences of project activities (What are the results of the project? How were these achieved? What is the impact?) Evaluation thus enables judgments and improvements of project effectiveness and/or generation of knowledge to inform decisions about future projects Indicators are an important part of both the monitoring and the evaluation process

The Logical Framework

The logical framework (log frame) is a methodology for conceptualizing projects and an analytic tool that allows a project developer/manager to detail a project clearly and understandably

The definition of what a project should achieve qualitatively and quantitatively is fairly uniform among agencies working in counter-trafficking, although not all of them use the same language For instance, the terms objective, strategic objective, development objective, program goal, strategic goal and performance goal all refer to what is expected to be achieved through the implementation of a project Equally, the definition of what a project produces varies between agencies

The terminology selected for this handbook is in harmony with the key definitions previously mentioned, and is of common use and understanding by many project managers and implementers Hence, the key definitions and structure of the logical framework used in this handbook are: overall objective, project purpose, results and activities

We recognize that different agencies use varying logical frameworks in project development While many versions were reviewed for this handbook, the log frame chosen was that of IOM

It is important to include the data collection component when developing a project Know what you plan to do so you can plan for the associated time and related costs Also remember that the actual data collection process may be different than expected/planned

note

For the purposes of this handbook, performance indicators are defined as pre-determined measurements that track specific changes or results of a project Performance indicators are directly linked to measuring progress toward project objectives and are often a combination of monitoring and evaluation

remember

Chapter 2: Conceptual Framework for Developing Performance indicators in Counter-Trafficking

Handbook on Performance Indicators for Counter-Trafficking Projects

IOM Project Logical Framework Matrix Template

Project Components Indicators Means of Verification Assumptions

Treaty and ment records

Treaty and ment records

Overall objectives are difficult to measure quantifiably Measuring the project activities alone does not tell enough about a project’s performance Therefore, as seen above, this handbook will concentrate on the project purpose and results levels They show short- and medium-term progress towards achieving the overall objective This is an operational perspective, and the aim is to help us understand the value

of our work

Logical Framework Key Definitions15

Ibid.

15.

Chapter 2: Conceptual Framework for Developing Performance indicators in Counter-Trafficking

Overall Objective: A single statement of the broader

aim of a project, i.e how the project can contribute to

a larger national or international development plan or

action Because the project can only contribute to, but

not fully achieve the larger development objective, it

would normally employ verbs such as “to contribute

to,” “to enhance,” to strengthen,” “to support,” “to

reinforce,” etc

Project Purpose: The objectives that will be directly

achieved by the project They define the primary reason

for the project by directly addressing the problem In

formulating the project, it is useful to think in terms of

what should be achieved by the end of the project The

project purposes must directly ensure the sustainable

benefits for the target group They should not explain

the activities of the project, nor re-state the results.

Result: Lists direct and measurable results expected

from the project activities They should be tangible,

visible and measurable means of achieving the Project

Purposes The result will be clear about what type of

change is implied, what is expected to change: i.e

a situation, a condition, the level of knowledge, an

attitude or a behavior

Activity: An important task that is seen as an essential

step to achieving a result These are the actions proposed

to achieve the result They would normally employ

“direct action” verbs, e.g “to identify,” “to establish,” “to

recruit,” “to carry out,” “to train” etc Activities produce

measurable results that may capture a number of

discrete and more specific results

Activities (To do…)

Overall Objective (To contribute to)

Project Purposes (To achieve…)

Results (To produce…)

Handbook on Performance Indicators for Counter-Trafficking Projects

Chapter 3: Developing Performance Indicators for

Counter-Trafficking Projects

Initial Considerations

Now that the logical framework has been established – the overall objective, project purpose, results and activities have been identified – we want to determine how to measure our progress To do so, a matrix with various common performance indicators is provided as the framework (see page 20) As we begin

to look at the matrix and implement its use, remember to consider the following:

Bear in mind one should consult with stakeholders when setting performance indicators This way not only do they extend their support, but also are able to advise on weaknesses or external interfering factors and can help identify processes that may need to be adjusted Thus, consulting with donors can help ensure that the chosen indicators correspond to the donor’s expectations, while consulting with partners and beneficiaries will help make the indicators feasible Aligning performance indicators with similar counter-trafficking projects could help provide for data that is comparable across countries For example, a project funded for US10,000 would probably not have the same expectations and performance indicators that a US500,000 project would have

Stakeholders are often part of the data collection process for performance indicators, particularly

as most counter-trafficking projects do not operate completely independently, but often are focused on building the capacity of those stakeholders

remember

Adapted from the IOM Evaluation Guidelines 2006, p28.

16.

Vulnerable Groups

This includes any group or sector of society that is at higher risk of being trafficked due to issues such

as having an irregular migration status; conflict or war; natural or manmade disasters; being refugees, internally displaced persons or stateless; discriminatory practices; violence in the home; and economic hardship, etc For the purposes of counter-trafficking, it is important to consider those susceptible to trafficking in persons, including former victims, in project development and implementation

Cost Linked to Performance Measurement for Counter-Trafficking

Cost related to measuring performance can vary greatly depending on the method used and what

is measured There are expensive methods, such as long-term follow up for victims that have been reintegrated, i.e minimum 2 years of evaluation, and inexpensive methods such as focus groups, phone surveys, random sample polls and many other forms in between Performance measurement related to monitoring or shorter-term evaluation tends to be less expensive This is why it is important to consider the availability of financial and human resources, related cost of data collection, data analysis, reporting and storage methods from the beginning when developing a project

A common issue faced by counter-trafficking projects is project length versus expectations For example,

a one-year direct assistance project may have a performance indicator that is “the reduction of trafficking

in persons in x country/area.” However, a measurable reduction in human trafficking would not occur in one year The activity is not directly linked to the identified performance indicator

Funding and staff must be made available for performance measuring work If anticipated costs appear prohibitive, consider some of the following:

Modifying performance indicators to permit less expensive approaches to regular data collection Rather than sending a consultant to interview partners, have them create a questionnaire that could be administered via email, or through implementing partners, for example

Data collection based on desk review exercises may be preferred to sample surveys that imply meeting in person For example, if a reintegration project is under review, phone calls with partners or beneficiaries rather than individual interviews with each beneficiary may be considered

Modifying the approach/design of evaluative activities; consider rapid, low-cost alternatives such

as focus groups and community interviews, informal surveys and direct observation as appropriate

Modifying the purpose or expected result, since it is not possible otherwise to judge progress at a reasonable cost

Handbook on Performance Indicators for Counter-Trafficking Projects

Data Collection and Data Sources (see Chapter 4 for further information)

The development of performance indicators comes from measuring expected results against objectives Data has to be collected to be measured In this process, collected data is used as a means of verification

Means of Verification (MoV) are the evidence behind the indicators – i.e the documentation to prove

that the measure given by the indicator has been achieved on the project purpose and results levels (ex surveys, records, statistics, interviews, etc).17

Keeping Performance Indicators SMART

When developing performance indicators, it is important to keep them SMART To define SMART

indicators, these need to be:

Specific: in terms of magnitude and time

Measurable: as to specify the change resulting from the implementation of the objective and the

way to tell whether it has been reached

Attainable: sources of information on indicators have to be reliable and accessible

Relevant: as to reflect or measure the effect of the project, rather than the effect of external factors Trackable: indicators have to draw upon data that is readily available or that will be collected as part

of the project management

How to Use the Matrix

Developing performance indicators can be challenging, but it is important to remember that a project does not need many indicators to measure performance; three to five is optimum An important aspect

of doing so is to be concrete Decide what is to be measured, and how it will be measured, based on the local context Know where you can get data and how it will be obtained While performance indicators are dependent on each context and situation, in many cases there are common indicators The performance indicators provided in this handbook are meant to be, at a minimum, a starting point for anyone looking

to develop performance indicators for counter-trafficking projects The choice of performance indicator varies according to the project’s purpose and anticipated results Performance indicators can be set at different levels and relate to different purposes They do not have to be numerous, but should be specific, independent of each other and verifiable

As described in Chapter 1, counter-trafficking work often focuses on three broad objectives: prevention, protection and prosecution (Three Ps) The accompanying matrix includes some of the most common performance indicators that could occur in counter-trafficking projects Within each “P,” the matrix is divided into substantive areas of counter-trafficking work

For example, the first category in the matrix is Prevention Under Prevention, there are the substantive areas of Policy and Legislation, Root Causes and Vulnerabilities and Cooperation Under each of those substantive categories, there are results listed (ex A.1) with their potential indicators, targets and means

of verification See chart below:

Overview of Matrix Structure

When developing performance indicators, it is better to have fewer than too many Three to five is optimum

remember

Performance Indicator Matrix Structure

Policy and tion: A-A.1.3

Legisla-Root Causes and Vulnerabilities:

B-B.4.2 Cooperation: C-C.1.4

Policy and tion: D-D.1.2

Cooperation: I-I.1.3

Chapter 3: Developing Performance Indicators For Counter-Trafficking Projects

Handbook on Performance Indicators for Counter-Trafficking Projects

When using the matrix to develop performance indicators, consider the project, idea, document, and so on:Locate which of the “Three Ps” is the focus – prevention, protection or prosecution – and find that “P”

The goal of the matrix is to help establish or strengthen systems to monitor, analyse, review and report performance data

The matrix is not meant to serve as a comprehensive, definitive list of options, but as a starting point If a performance indicator does not fit your needs, then create a new one, keeping the previous chapters in mind, and remembering to keep indicators SMART

Prevention

A Policy and Legislation

Project Purpose1 Performance

Indicators Target

Means of Verification Data

Collection Data Source

A.1 To enhance

policy/legislation

to prevent TIP and promote safe migration

Policy/legislation instruments are

in place and meet international standards

government records

•

Results Performance Indicators Target

Means of Verification Data

Collection Data Source

A.1.1 Existence

of policy framework to combat TIP which incorporates prevention measures

Country has ratified the UN Protocol

ing law and/or related legislation passed/amended that includes prevention Regulations and/or guidelines estab- lished to enforce anti-trafficking law and/or related legislation National Plan of Action enacted which contains provisions on prevention

Treaty and government records

Treaty and government records

Treaty and government records

•

•

•

•

Keep in mind that a project

purpose or result may not fit your

context Each will need to be

adjusted to link with the specific

project and what is within its

power to achieve For example, an

organization that does not work

on legislation would not work to

enhance it and thus, it would not

be in a project’s activities The

corresponding project purpose in

the matrix would either need to

be adapted or a different purpose

would need to be used

1.

The Matrix

Penalties are enforced for non- compliance

X%2 or number of key stakeholders who are penalized for non-compli- ance3

coopera-on migrant labour between sending and receiving countries in place Established registration mecha- nisms for migrant labour in sending and receiving countries

•

•

Government records

Government representative

•

•

Prevention

B Root Causes and Vulnerabilities 6

Project Purpose Performance Indicators Target

Means of Verification Data

Collection Data Source

knowledge of

X target group about TIP and safe migration7

survey, focus groups

• • X target group

Percentage can only be used if the denominator is known (total number of key stakeholders) Otherwise the indicator should

be a number.

Depending on the context, the numerical value of the performance indicator may increase or decrease.

The percentage increase/ decrease for targets should be based on the baseline number and analysis of what you think your realistic change would be based on your project ‘s activities Thus, X needs to be replaced with your target number.

Depending on context, the numerical value of the performance indicator may increase or decrease.

When creating indicators, be sure to clarify how each specific performance indicator relates

to both root causes and /or vulnerabilities

For example: Pre information campaign survey results in 6 out of 10 questions right (60% knowledge) This is the baseline The target could then be a 25% increase in knowledge (7.5 out of

Results Performance Indicators Target

Means of Verification Data

Collection Data Source

B.1.1 Information

campaign message is received and used

by X target group

X% target group who can iden- tify/describe the campaign message

X% target group who took action

as a result of the campaign message

or contacted a provider of/for information or services

•

•

X% increase of get group who can identify/describe the campaign message X% increase of target group who took action

tar-as a result of the campaign message

or contacted a provider of/for information or services

•

•

Sample survey, focus groups

Hotline statistics (“stats”) review8

•

•

X target group

Hotline call records

•

•

B.1.2 Enhanced

capacity of media to report accurately on TIP.

X number of media coverage on TIP issues

X number of curate9 reports

of TIP article coverage Review of media orga- nizational policies

ing curriculum in educational system

Counter-traffick-is institutionalized Educators’ level

of knowledge on TIP and their role

in preventing trafficking11

X number in referrals of at risk individuals

•

•

•

Analysis of curriculum

Focus groups, sample surveys, etc

Records review of referral stats

•

•

•

Educational body

Educators

tional body, implementing partners, governmental bodies

Educa-•

•

•

To identify change, compare pre

and post data.

Accurate means that the

definition of TIP is applied

correctly.

Remember to adapt to the

local context; choose the

education level on which the

project activities will focus (e.x

elementary, university)

Level of knowledge can be

measured in different ways, such

as number of correct answers on a

test (expressed as a percent).

This is assuming that social

programs exist (State or Civil).

Number of referrals

of at risk als

individu-•

•

X% increase of target group members, after X months, who have conducted some TIP prevention related activity on their own X% increase in referrals of at risk individuals

•

•

Reports review and interviews

Records review of referral stats

•

•

Target group

Target groups, partners

informa-on trafficking and safe migration

of information channel staff who have received targeted training

on trafficking and safe migration

•

•

•

Relevant stakeholders

Informational channels management

Training organizations

13.

Project Purpose Performance Indicators Target

Means of Verification Data

Collection Data Source

B.2 To increase social and economic op- portunities for vulnerable per- sons in source communities

X% of project beneficiaries em- ployed, enrolled and/or participat- ing in micro-credit schemes

Number of nesses participat- ing in employ- ment programs for vulnerable groups

busi-Number of higher education institu- tions providing tuition for vulner- able groups

X% of able groups using public services (health, legal, social)

or participating

in micro-credit schemes X% increase in number of busi- nesses participat- ing in employment programs for vulnerable groups X% increase in number of higher education institu- tions providing tuition for vulner- able groups X% increase of vulnerable groups using public services (health, legal, social)

or reports review

Records review, interviews

Records review, interviews

Stats/records review

Businesses

Educational institutions

Public services institutions

Collection Data Source

B.2.1 Increased

access to education for vulnerable groups14

X% of target group enrolled in formal

or non-formal education

X% of target group regularly attend- ing/completing formal / non-formal education15

•

•

X% increase of target group enrolled in formal

or non-formal education X% increase of target group regularly attend- ing/completing formal / non-formal education

•

•

Records review

Records review, interviews

•

•

Ministry of Education

Educational institutions and educators

•

•

Implies all levels of education.

Includes vocational training

programs and life skills for

orphans (social).

14.

15.

The Matrix

X% of target group participating

in micro-credit schemes

Number of nesses participat- ing in employment programs for vulnerable groups

busi-•

•

•

X% increase of project beneficia- ries employed

X% increase in target group participating

in micro-credit schemes X% increase in number of busi- nesses participat- ing in employment programs for vulnerable groups

•

•

•

Surveys, interviews, reports review Surveys, interviews, reports review

Records review, interviews

•

•

•

Beneficiaries, implementing partners

Beneficiaries, implementing partners

in place to reduce discrimination17

Level of positive attitudes of target groups toward X vulnerable groups

X% of vulnerable groups using pub- lic services (health, legal, social)

X% of vulnerable groups employed

X% of vulnerable groups enrolled in and/or completing formal or informal education

or informal tion

Stats/records review

Stats/records review

Stats/records review

Government records

Target groups

Public services institutions

Government records, labour records Government records, school records

Some results in the matrix will

be too ambitious or too limited, depending on the local context and implementing organization’s role Do what makes sense where you are and within the project’s activities

For example: quotas, bathrooms for girls in schools, alternative insurance schemes for irregular migrants, services for undocumented migrants 16.

17.

Project Purpose Performance Indicators Target

Means of Verification Data

Collection Data Source

B.3 To reduce TIP

through side measures

demand-Level of edge of target consumer group

knowl-on products and services produced

by exploitative and trafficked labour

Number of companies that are levied fines for using trafficked/ex- ploited labourers

•

•

X% increase in level of knowl- edge of target consumer group

on products and services produced

by exploitative and trafficked labour X% increase

in number of companies that are levied fines for using trafficked/

exploited labourers

•

•

Sample survey

Records review

•

•

Target (consumer) group

Controlling body

•

•

Results Performance Indicators Target

Means of Verification Data

Collection Data Source

B.3.1 Consumers

are informed about products and services produced by exploitative and trafficked labour

Level of knowledge

of target sumer group about products and services produced

con-by exploitative and trafficked labour

Number of target group who are reached as a result

of the campaign

Developed guidelines for consumers to dis- tinguish between goods/services that have been produced through exploitation versus fair labour methods

•

•

•

X% increase in level of knowledge

of target sumer group about products and services produced

con-by exploitative and trafficked labour

% of reached target group who took action as a result

of the campaign message or con- tacted a provider

of information or services Yes/True

•

•

•

Sample survey

Sample vey, review

sur-of service provider records18

Target group, service provider

Relevant stakeholders

•

•

•

I.e number of calls from target

group to the hotline before and

after the campaign.

18.

The Matrix

Number of victims identified and referred to direct assistance networks by labour inspectors, labour unions and govern- ment officials

Number of companies that are levied fines for using trafficked/

X% increase

in number of victims identified and referred to direct assistance networks by labour inspectors, labour unions and govern- ment officials X% increase

in number of companies that are levied fines for using trafficked/ex- ploited labourers

•

•

•

Review of training stats, pre and post surveys

Stats/records review

Records review

•

•

•

Training organiza- tions, labour organizations

Relevant stakeholders (referring and/or receiv- ing parties)

Controlling body

or company cies on exploitation Number of compa- nies that publicly support fair labour practices.19

poli-Number of victims identified and referred to direct assistance networks by target group

X% increase victims identified and referred to direct assistance networks by target group

•

•

•

Review of policies

Monitoring

of related actions

Stats/records review

•

•

•

Company records

Companies

Relevant stakeholders (referring and/or receiv- ing parties)

disciplin-•

•

•

Yes/True Yes/True

X% increase in victims identified and referred to direct assistance networks by target group

•

•

•

Review of policies Review of policies Stats/records review

•

•

•

Company records Company records Relevant stakeholders (referring and/or receiv- ing parties)

19.

Project Purpose Performance Indicators Target

Means of Verification Data

Collection Data Source

B.4 To improve

data/information

on TIP (routes, scale, root cause, social and economic impacts, demographics, trends)

Prevention ties, strategies and policy (changes) are influenced by research informa- tion and recom- mendations Best practices are identified and disseminated

Monitoring, interviews

•

•

Project documents, research reports

Project documents, relevant stakeholders

•

•

Results Performance Indicators Target

Means of Verification Data

Collection Data Source

B.4.1 Information

on different aspects of TIP is systematically collected and processed

Mechanism for data collection among diverse stakeholders established Annual reports shared

Interviews, surveys

•

•

Relevant stakeholders

Relevant stakeholders

•

•

B.4.2 Increased

information and deeper understanding

of X aspect of TIP and impact of CT measures

Targeted research conducted

Best practices are identified and disseminated

•

•

Relevant stakeholders

Relevant stakeholders

•

•

The Matrix

Collection Data Source

coopera-Roles and sibilities are clearly defined within the cooperation mechanism21

Document reviews, interviews

•

•

Relevant stakeholders

Relevant stakeholders

•

•

Results Performance Indicators Target

Means of Verification Data

Collection Data Source

Number of nesses participat- ing in employment programs for vulnerable groups

busi-•

•

X% increase in financial and in- kind contributions

by private sector for prevention activities X% increase in number of busi- nesses participat- ing in employment programs for vulnerable groups

•

•

Surveys, interviews

Records review, interviews

•

•

Businesses, recipients

•

•

Yes/True

X% increase in budget allocation for civil society prevention efforts

•

•

Document review, interviews

Budget review

•

•

Relevant stakeholders

to coordinate prevention efforts Level/number of duplicated efforts23

•

•

Yes/True

X% decrease in level/number of duplicated efforts

•

•

Direct observation

Reports review

•

•

Government

Government documents

•

•

X% reduction in people trafficked

is the ideal target for this action, but not easily measurable Thus, other indirect indicators must be used.

In some cases, a Memorandum of Understanding (MOU) or Standard Operating Procedures (SOP) could

be formalized.

For example, a Plan of Action.

An example of this includes one joint hotline as opposed to multiple hotlines.

20.

21.

22.

23.

C.1.4

Coordinated efforts between source, transit and destination areas

Joint agreements

to promote regular migration flows24 established/im- proved Number of coordi- nated information and awareness raising activities transnationally/be- tween areas

•

•

Yes/True

X% increase in number of coordi- nated information and awareness raising activities transnationally/be- tween areas

•

•

Desk review

Observation, document review, interviews

•

•

Government documents

Relevant stakeholders

in source, transit and destination areas

•

•

Protection

D Policy and Legislation

Project Purpose Performance Indicators Target

Means of Verification Data

Collection Data Source

D.1 To promote,

establish and implement policies and legislation that facilitates victim protection and assistance

State policies established to provide appropri- ate and compre- hensive direct assistance to VoTs

documents

•

Results Performance Indicators Target

Means of Verification Data

Collection Data Source

D.1.1

Comprehensive anti-trafficking and/or related legislation enacted and implemented that entitles victims to comprehensive protection

Country has ratified the UN Protocol Anti-traffick- ing law and/or related protection legislation passed/

amended that mandates free and comprehensive services to VoTs and is consistent with international standards (ex med- ical, psychosocial, voluntary return, reintegration, victim compensa- tion)

State budget allocated towards victim assistance

•

•

•

Yes/True Yes/True

Budget review

•

•

•

Government documents Government documents

State budget

•

•

•

For example: a bilateral agreement

for facilitated labour migration.

24.

The Matrix

or strengthened that entitles victims

to comprehensive protection Budget amounts allocated towards victim assistance

Roles and sibilities of relevant stakeholders are identified

•

•

•

Document review, observation

Budget review

Document review

Collection Data Source

an educational system, etc.25

re/integrated victims employed, re/inserted in educational system, etc.

interviews, direct observation

• Victims, direct

assistance providers

•

Results Performance Indicators Target

Means of Verification Data

Collection Data Source

Number of victims removed from de- tention facilities27SOPs/guidelines established for identification

•

•

•

X% increase in number of victims identified and re- ferred for services X% change of victims removed from detention facilities Yes/True

•

•

•

Stats/records review

Stats/records review

Document review, interviews

•

•

•

Relevant stakeholders

Government, civil society groups

Relevant stakeholders

as 0-6 months as the emergency phase; 6 months – 2 years as the interim phase; and 2 years + as the long term, and adjust indicators accordingly.

Please keep in mind that there are also other factors that affect this indicator, such as the overall prevalence of trafficking.

Number of victims removed from detention is based on context For example, some countries rarely place VoTs in detention facilities, thus the change in number might

be low or zero

25.

26.

27.

E.1.2 Victims

received appropriate and tailored medical and psychosocial care

Establishment

of standardized medical and psychosocial needs assessment tools X% of victims receiving voluntary medical and psychosocial care linked to the needs identified in an assessment

•

•

Yes/True

X% increase of victims receiving voluntary medical and psychosocial care linked to the needs identified an the assessment28

•

•

Document review, interviews

Case review

•

•

Direct assistance providers

Direct assistance providers documents

•

•

E.1.3 Victims

provided with appropriate legal assistance/

temporary resident permit

Number of porary residence permits issued29

tem-X% of victims receiving compen- sation

X% of identified victims who have filed criminal complaints X% of identified victims who have received legal counseling

X% increase of identified victims who have filed criminal complaints X% increase of identified victims who have received legal counseling

Records review, interviews

Stats/records review

X% of victims whose return home was consistent with interna- tional protection standards (security assessment, family tracing completed)

Protection dards/mechanisms established

stan-•

•

X% increase of victims whose return home was consistent with international protection stan- dards30 (security assessment, family tracing completed) Yes/True31

•

•

Interview, case review

Document review, interviews

•

•

Victims, direct assistance providers

Direct assistance providers

Consider the context If nothing

exists, create it.