LV Thạc sỹ_Research on monitoring and evaluation of international development projects

INGOs IN VIETNAM AND THEIR MONITORING AND EVALUATION WORK 3.1 INGOs and their international development projects in Vietnam...19 3.1.1 History of INGOs in Vietnam...19 3.1.2 Current s

This thesis concludes my studies for a master in management degree in the Master of BusinessAdministration course at the CFVG (Centre Franco-Vietnamien de formation à la Gestion)) Thethesis was carried out at East meet West Foundation of Vietnam, Hanoi during the period of June

to December

First of all, I would like to thank all professors, lectures, and tutors of the CFVG MBA Programfor giving me helpful lectures and excellent support and guidance during the study and to helpcomplete the course with good results

Furthermore, I would like to acknowledge the supervision of, Prof… at CFVG He has providedvaluable guidance for me since I started to work on the thesis

I also express my gratitude to leaders and colleagues at East Meets West Foundation, who spenttheir time to share with me the helpful information and comments Their help has contributedmuch to finish this thesis

Thank you!

Hanoi, December 2009

…

TABLE OF CONTENT

1 INTRODUCTION TO THE THESIS Page

1.1 Rationale of the topic 5

1.2 Scope of the research 6

1.3 Objective of the research 6

1.4 Methodology 6

1.5 Limitation 7

1.6 Structure of the research 7

2 LITERATURE REVIEW- INTERNATIONAL DEVELOPMENT PROJECTS (IDPS) AND THEIR MONITORING AND EVALUATION 2.1 International Development Projects ……… 9

2.1.1 Definition and Basics of International Development Projects 9

2.1.2 Characteristics of International Development Projects 10

2.1.3 Life cycle of International Development Projects 11

2.2 Monitoring and evaluation of international development projects 13

2.2.1 Definition of monitoring and evaluation 13

2.2.2 Role of monitoring and evaluation to the project management of IDP 14 2.2.3 Designing of monitoring and evaluation process for IDPs 15

3 INGOs IN VIETNAM AND THEIR MONITORING AND EVALUATION WORK 3.1 INGOs and their international development projects in Vietnam 19

3.1.1 History of INGOs in Vietnam 19

3.1.2 Current situation of the INGOs in Vietnam and their projects 21

3.2 Monitoring and evaluation of INGOs in Vietnam 24

4 MONITORING AND EVALUATION OF THE EAST MEETS WEST FOUNDATION’S SUPPORT NETWORK FOR PEOPLE WITH DISABILITIES

4.1 General background of the program 27

4.1.1 Disability and Agent Orange issue in Vietnam and QuangNgai 27

4.1.2 Program goals, objectives and activities 28

4.2 Monitoring and Evaluation of the Support Network for the People with Disabilities 30

4.2.1 Measurable objectives and indicators 33

4.2.2 M&E overall process, methodology and tools 34

4.2.3 M&E personnel 34

4.2.4 Coordination with local partner 35

4.2.5 Mid-term evaluation 36

5 EVALUATION AND RECOMMENDATIONS FOR THE MONITORING AND EVALUATION OF THE “SUPORT NETWORK FOR PEOPLE WITH DISABILITIES” 5.1 Evaluation 40

5.1.1 Strengths 40

5.1.2 Shortcomings, difficulties and challenges 42

5.2 Recommendations 41

CONCLUSION 43

REFERENCES 44

ANNEXES 46

EXECUTIVE SUMMARY

Currently there are about 50 international non-governmental organizations (INGOs) working inVietnam, providing support to the disadvantaged people in the country Their aid has beenrealized though many development programs carried out in different regions and focused oneducation, health, social issues and economic development Along with the financial andhumanitarian support, the NGOs have also brought into Vietnam the knowledge of projectmanagement of international development projects One of the most important techniques used inthe international development projects is monitoring and evaluation (M&E)

East Meets West Foundation, where the author is working with, has been present in Vietnam formore than 21 years and is currently the largest non-governmental organization in the country.Among its innovative programs and projects in health, clean water and sanitation, education andcommunity infrastructure is a well-established and managed program to support the disabled

people and the Agent Orange survivors, the Support Network for People With Disabilities

(SN-PWD) It is a 5-year program which was started in 2006 in Quang Ngai province

This research is focusing on understanding how monitoring and evaluation is important andcarried out in international development projects by INGOs and later focusing on analyzing theM&E process, methodology and techniques of East Meets West Foundation’s Support Networkfor the People with Disabilities program, In the last part, the research concentrates in giving outsome recommendations for a more effective M&E of the project

CHAPTER 1 GENERAL INTRODUCTION TO THE THESIS1.1 Rationale of the topic

Over the last decade there has been a dramatic growth in the number of NGOs involved indevelopment aid, in both developed and developing countries The total amount of public fundsbeing spent through NGOs has grown dramatically and the proportion of development aid goingthrough NGOs, relative to bilateral or multilateral agencies, has also increased

Associated with this growth has been a growing concern about identifying the achievements ofNGOs This has been evident in the burgeoning literature on the monitoring and evaluation ofNGO activities Internationally, a lot of INGOs are implementing different types of monitoringand evaluation methods and processes

In Vietnam, currently there are about 500 international non-governmental organizations (INGOs)working to provide support to the disadvantaged people in the country Their aid has been realizedthough many development programs carried out in different regions and focused on education,health, social issues and economic development According to incomplete statistics of thePeople’s Aid Coordinating Committee (PACCOM), INGOs provided an estimate of 2,700 grants(project and non-project), with a total disbursement of over US$216 million, compared to about

175 million USD in the year 2005, or an increase of 23 %

The M&E concept has just been brought to Vietnam through the large INGOs, where the M&Esystems are in place However, in general picture, the M&E is at the new stage, where NGOspractitioners are applying the internationally successful project management models, and themonitoring and evaluation process in particular, in Vietnam context

East Meets West Foundation has been working in Vietnam for more than 21 years and is currentlythe largest non-governmental organization in the country EMW implements innovative programsand projects in health, clean water and sanitation, education and community infrastructure thathave made a lasting impact on millions of children and families living in poverty Thus far, EMWhas invested over US$75 million in Vietnam

Among the largest programs that EMW has been carrying out, disabled people and AO survivors

are of the organization’s top interests The East Meets West established the Support Network for People With Disabilities (SN-PWD) in 2006 with a grant from The Ford Foundation The SN-

PWD is an initiative to help create channels of medical and educational services for people withdisabilities (PWDs) of various kinds in Quang Ngai province The main goal is to improve thequality of life for people with disabilities by th e creation of long-term, sustainable channels bywhich these people can access health care and other services.

Working at East Meets West Foundation, the author finds it so exciting and useful to have a study

of this project’s monitoring and evaluation, as it is a really important part of the projectmanagement and the following research has been formulated:

“MONITORING AND EVALUATION OF INTERNATIONAL DEVELOPMENT PROJECTS –

A CASE OF THE EAST MEETS WEST FOUNDATION’S SUPPORT NETWORK FORPEOPLE WITH DISABILITIES”

1.2 Scope of the research

Project management in general and monitoring and evaluation embraces a wide range of differenttypes of projects that could not be fully captured within this research The monitoring andevaluation itself can be applied in many different types of projects

The research will address some theoretical issues of the monitoring and evaluation at developmentprojects and then focusing on a case study of monitoring and evaluation of one of the largestprogram of the East Meets West Foundation: “Support Network for the People with Disabilities”

1.3 Objectives of the research

The primary aim of the research is to look into how East Meets West, as an INGO, has employedmonitoring and evaluation theories and tools to develop and manage their program to support thedisabled, which started in Quang Ngai province since 2006 Subsequently, it also wishes tosuggest for some recommendations on how to improve the monitoring and evaluation work.There are specific objectives to be met in the research:

- Carry out a literature review on monitoring and evaluation for NGOs’ internationaldevelopment programs/projects

- Provide some successful exercises in monitoring and evaluation of some world-class known NGOs

well Study the monitoring and evaluation at East Meets West Foundation’s Support Networkfor the People with Disabilities program

- Give some recommendations for effective monitoring and evaluation for the program

1.4 Methodology of the research

1.4.1 Data collection method

In order to have a panorama view on the monitoring and evaluation of the Support Network forthe People with Disabilities, both primary and secondary data were generated from the followingsources:

Primary Data Sources

- In-depth interviews and discussions were conducted with the following key informants:

 EMW’s Country Director, who is responsible for and overseeing all the programs

- Qualitative research was conducted with program managers and managers from 20 INGOs

in Vietnam to understand how M&E is implementing in INGOs in Vietnam

Secondary Data Sources

- Program quarterly reports and mid-term reviews

- Books, journals

- On-line, web-based information postings

1.4.2 Data Analysis Method

The study is developed in a descriptive and analytical framework Furthermore, in order to makethe analysis fore deeply, some other complementary tools are being used such as statistic andcomparative methods

It is noteworthy that the research requires its author to apply her knowledge obtained during thestudy course at CFVG Hanoi in regards of project management This is a good chance for theauthor to reflect the theories in the real management context

1.5 Limitations

The author claims to finish the research at her best effort However, as the author’s knowledge inthe field is limited, the research hence may be performed with certain shortcomings Still, due totime constraint and restricted access to documents of East Meets West Foundation (as aconfidentiality principle), the presentation will be impacted to some extent Every and all ideas todevelop the research are highly appreciated by the author

1.6 Structure of the research

The research study is divided into four chapters, including:

Chapter 1: General introduction to the research: the chapter provides the rationale of the topic,

the objective and scope of this research as well as the methodologies applied

Chapter 2: Literature review – international development projects (IDPs) and their monitoring

and evaluation: the chapter starts from raising the definition and other theoretical issues regardingproject, project management, international development projects and monitoring and evaluation

Chapter 3: INGOs in Vietnam and their monitoring and evaluation work: the chapter features the

history, development and current issues of the INGOs in Vietnam as well as the current situation

of the monitoring and evaluation of these organizations’ programs

Chapter 3: Monitoring and evaluation of the East Meets West Foundation’s Support Network for

People with Disabilities: focusing on how the program’s monitoring and evaluation is planned,implemented and contributed to the effectiveness of the program as well as the decision makingprocess of the organization

Chapter 4: Evaluation and Recommendations : From the description in chapter 3, chapter 4

analyses the strengths, weaknesses and challenges of the monitoring and evaluation of the SupportNetwork for the People with Disabilties, then provide with some recommendations to help thisimprove and overcome the challenges and difficulties

CHAPTER 2 LITERATURE REVIEW:

INTERNATIONAL DEVELOPMENT PROJECTS AND

THEIR MONITORING AND EVALUATION 2.1 International Development Projects

2.1.1 Definition and Basics of International Development Projects

2.1.1.1 Project and project management

Before going into particular definition of international development projects (IDPs), it isnecessary to run quickly through what a project is and how project management is defined anddivided into steps and processes

The definition of a project pointed out in the Project Management Guidebook by Method 123Ltd., a project is “a unique endeavor to produce a set of deliverables within clearly specified time,cost and quality constraints” As such, they defined project management as the skills, tools andmanagement processes required to undertake a project successfully

The Project Lifecycle, which consists of four phases:

Fig 1: Project Life Cycle (Source: Method 123 Ltd 2003, Project Management Guidebook)

Project initiation: In this phase, a business problem (or opportunity) is identified and a business

case which provides various solution options is defined A “Project Charter” is completed, whichoutlines the objectives, scope and structure of the new project

Project planning: One the scope of the project has been defined in the Project Charter, the project

enters the detailed planning phase This involves the creation of a project plan, a resources plan,financial plan, quality plan, risk plan, acceptance and communications plan At this point theproject has been planned in detail and is ready to be executed

Project execution: This phase involves the execution of each activity and tasks listed in the

Project Plan While the activities and tasks are being executed, a series of management processesare undertaken to monitor and control the deliverables being output by the project

Project completion: Project completion involves releasing the final deliverables to the customer,

handing over project documentation, terminating supplier contracts, releasing project resourcesand communicating the closure of the project to all stakeholders The last remaining step is toundertake a Post Implementation Review to quantify the overall success of the project and list anylesions learnt for future projects

As seen from the above introduction about the project, the monitoring and evaluation work fallsinto the third and fourth part of the project But its planning and actions should be considered andplanned from the beginning of the cycle

2.1.1.2 Definition of international development projects (IDPs)

According to Robert Younker in “The Nature of International Development Projects”, 2003,World Bank, international development projects (IDPs) are medium to large size projects and/orprograms in all sectors of developing countries financed by the following types of institutions:

- Multilateral Development Banks such as the World Bank and regional development banks(ADB, AfDB, IADB, CDB, etc.)

- United Nations Associated Agencies (UNDP, FAO, ILO, WHO, UNIDO, etc.)

- Bilateral and multilateral government agencies (such as USAID, European Union or CDA)

- Non-governmental organizations (NGOs) (such as CARE, Catholic Relief Services,Oxfam or Save the Children)

- Government agencies in developing countries

Thus, by definition, IDPs involve a number of different actors including donor agencies, (oftenmore than one), government organizations at several levels, consultants, trainers, evaluators,researchers, and local beneficiaries including local organizations

2.1.2 Characteristics of International Development Projects

In order to have more comprehensive definition of IDPs and to understand how different IDPs arefrom normal projects, it is also necessary to give out its characteristics:

 The objectives of IDP are for economic and social development, often involving povertyreduction and the usual profit motive is often missing The financing agency often hasmotives and objectives of its own;

 The financing can be both via a loan or an outright grant;

 By definition, all IDPs are in developing countries and at least partially, externally financed;

 The management of IDPs requires dealing with entire Project Life Cycle from Identification

to Operation and Ex-ost Evaluation;

 All projects can have a variety of interested parties but IDPs usually have a large array ofstakeholders whose views must be considered;

 The role of project sponsor is sometimes unclear It can appear that the real sponsor is theexternal agency rather than a domestic agency;

 Developing country environments are difficult environments There is often a lack ofinfrastructure All resources are in short supply especially human resources like trainedaccountants Local citizens may have different concept of time and different value structuresand cultures The external forces may be seen in a negative or a positive way The countrieshave their own systems, each donor may have its own systems and all key differences;

 The financing agency often leads the project identification in line with its own objectives;

 It is difficult to involve the local beneficiary stakeholders in project discussions because ofliteracy, volume, distance and communication problems;

 Government in developing country are continually short of resources and usually havedifficulty meeting the resource requirement they promised at the time of project planning andapproval;

 Corruption is often an endemic problem and require s monitoring systems to insuretransparency;

 The local government environment is often a “one-entrepreneurial” one and yet projectsuccess often requires an entrepreneurial project manager and project champion

2.1.3 Life cycle of International Development Projects

IDPs, like all projects, also go through a project life cycle Munns and Bjeirmi also identifiedgroups of parties involved in each stage, with different activities, different outputs All can besummarized as follows:

(Source: Van Thi Ngoc Ha 2005)

Key activities End Products Key players

Conception stage: the idea for the project is born within the client organization and its feasibility is determined

- Identify the potential target groups and prioritiesfor support

- Assess needs

- Develop and evaluate project alternatives

- Need assessment report and project concept paper

- Funding agency

- Consultants-Implementing agencies

Planning stage: the method to achieve the original idea is planned and designed

- Develop the project scope and LOGFRAME

- Estimate resourcesrequired

- Mobilize support and commitment

- Plan for project schedule and organization set up

- Negotiate for final approval

- Project documents (Project scope, LOGFRAME, budget, organizationset up, schedule, risk management plan)

Project agreement

- Funding agency

- Consultants-Implementing agencies

Implementing stage: the plans are convert into physical reality

- Kick off the project

- Carry out the project activities as planned

- Control the projectbudget

- Monitor the progress and solve problems

- Manage relationships with stakeholders

- Resources mobilized

- Activities carried out

Outputs produced

- Project management team

- Subcontractors, suppliers, partners

- Target beneficiaries

parties will coordinate for project financial settlements project completion reports At the end of this stage, project team is dissolved

- Final test the project outputs

- Complete the project final report

- Settle all financial transactions with subcontractors, suppliers, consultantetc…

- Hand over the project output and asset

- Dissolve or transform the project team

- Project completionreport

- Project outputs andassets transferred

- Dissolution or transformation of the project team into

an ongoing operation

- Funding agency

- Government

- Consultants

2.2 Monitoring and evaluation of international development projects

2.2.1 Definition of monitoring and evaluation

The OCED (2002) defines monitoring and evaluation as below:

Monitoring is a continuous function that uses the systematic collection of data on specified

indicators to provide management and the main stakeholders of an ongoing developmentintervention with indications of the extent of progress and achievement of objectives andprogress in the use of allocated funds

Evaluation is the systematic and objective assessment of an on going or completed project,

program, or policy, including its to determine the relevance and fulfillment of objectives,development efficiency, effectiveness, impact, and sustainability An evaluation shouldprovide information that is credible and useful, enabling the incorporation of lessons learnedinto the decision-making process of both recipients and donors

In juxtaposing these two definitions, it is immediately evident that they are distinct yetcomplementary Monitoring gives information on where a policy, program, or project is at anygiven time (and over time) relative to respective targets and outcomes It is descriptive inintent Evaluation gives evidence of why targets and outcomes are or are not being achieved Itseeks to address issues of causality Of particular emphasis here is the expansion of thetraditional M&E function to focus explicitly on outcomes and impacts

Evaluation is a complement to monitoring in that when a monitoring system sends signals thatthe efforts are going off track (for example, that the target population is not making use of theservices, that costs are accelerating, that there is real resistance to adopting an innovation, and

so forth), then good evaluative information can help clarify the realities and trends noted withthe monitoring system For example, “If annual performance information is presented by itself(in isolation) without the context and benefit of program evaluation, there is a danger ofprogram managers, legislators and others drawing incorrect conclusions regarding the cause

of improvements or declines in certain measures Simply looking at trend data usually cannottell us how effective our government program interventions were” (Channah Sorah 2003) Westress the need for good evaluative information throughout the life cycle of an initiative—notjust at the end—to try and determine causality

CHAPTER 3 INTERNATIONAL NGOS IN VIETNAM AND THEIR MONITORING AND EVALUATION

3.1 INGOs and their international development projects in Vietnam

3.1.1 History of INGOs in Vietnam

INGOs have been providing support to Vietnam for many decades Right from the early1970s, some sixty-three INGOs worked in Vietnam Source Populaire Francais may be thefirst INGOs to have worked in Vietnam while other com to work in Vietnam after the end ofthe war against the French in 1954 Earlier international humanitarian activities had beensupported through mission of the Roman Catholic Church Most of these first INGOs toVietnam had religious (Christian) affiliations

Before 1965, the North saw support from Socialist countries through various FriendshipAssociations After 1965, popular support for Vietnamese people suffering from the war grew

in other Western Countries and some INGOs started to ship goods to Vietnam In the end of

1974, there were totally 60 of which 32 were America ones However, some organizationsoperated under the political pressure of America Government

Some INGOs realized the nonsense war caused by America Government an withdrew theiroperation in Vietnam such as Voluntary Service organization (IVS), Save Children Fund UK(SCF/UK) and some other medical organizations also stopped its operation in Vietnam,However, in 1975, most INGOs closed their offices and foreign staff left as the SocialistRepublic of Vietnam assumed responsibility for their activities

From 1975 to 1979, some INGOs (such as AFSC-Quaker Service…) moved their offices toThailand and/or Laos and continued to provide relief and humanitarian assistance from aboardwith occasional visits INGOs which had used to operate in Vietnam came back to the unitedVietnam and open its scale of operation and many INGOs joined to the NGO community inVietnam The period from 1976 to 1978 could be considered high peak of NGO operation inVietnam with the participation of more than 70 organizations, the aid, medicine, food stuff fornew economic sector and agricultural and industrial renovation and development

Virtually all Western donors with the exception of Sweden supported the increased embargoimposed by the US in 1979 This decreased significantly the amount of assistance availablefor INGOs whishing to support Vietnam, who developed other strategies in order to raisefunds and provide humanitarian relief during severer food shortages In 1979, the Ministry ofFinance established AIDRECEP to facilitate assistance from foreign organizations, includingINGOs Form 1979 to 1988, the INGO operation in Vietnam was at the standstill The number

of INGOs did not decrease very much but they operated in hesitation The value of projectswas lower one fourth than those previous year This happened due to misunderstanding fromINGOs toward the issue relating to Cambodia at the moment During this time, the internalsituation and external factors, in particular the US-led embargo, limited the opportunities forINGOs to increase their support for Vietnam In relation to needs, INGOs support was low burthe solidarity shown was important and is well remembered by those who cite the Vietnameseexpression, “we remember best the mouthful of rice given when we were hungry”

During the mid to late 1980s, Vietnam began to broaden its international relations At the end

of 1986, the Communist Party of Vietnam initiated a policy of Doi Moi or Renovation This

change coincided with the collapse of the former socialist countries of the Soviet Union andEastern Europe; as a result, these countries substantially reduce their support for Vietnam’sdevelopment This change created new possibilities for INGOs wishing to support Vietnam

and many renewed efforts to have representatives in the country By 1988, the effects of Doi

Moi had begun, including increased interest from foreign investors In October 1988, CIDSE

became the first INGOs with a representative in Hanoi The 1989 withdrawal of Vietnamese

troops from Cambodia brought the beginning of re-normalisation with Western donorcountries.

In 1989, the People’s Aid Co-ordinating Committee (PACCOM) of the Vietnam Union ofFriendship Organizations (VUFO) was established to “work as a focal agency for internationalnon government organizations” PACCOM’s main function are to: work as a bridge betweenINGOs and Vietnamese partners and localities; facilitate INGOs activities and assist localpartners in their relation with INGOs; gather and share information concerning INGOsactivities in Vietnam; and recommend to the Government proper policies for INGOs inVietnam PACCOM is also responsible for processing permits for INGOs, who are requested

to register with the Committee for NGO Affairs through PACCOM In the early 1990s, theVietnamese Government gave AFSC, Action Aid, CARE, Oxfam Belgium, and the thenOxfam Great Britain permission to open offices in Hanoi and actively encouraged otherINGOs to come

For a long time, due to international isolation and lack of legal framework, INGOs workedfrom a distance The lack of a legal framework and legislative environment created challengesthat may have led to misunderstanding and occasional mistrust, for example, difficulties inobtaining visa, defining administrative procedures, working at the community level andvisiting remote areas

In 1993, in respond to the need for better sharing information about INGOs’ programmes, theNGO Resource Center was established in the officer of Radda Barnen

In 1996, the Committee for NGO Affairs was formed and comprised of six member atministerial level: Vietnam Union of Friendship Organizations (VUFO), as standing agency forforeign NGO affairs; the Ministry of Foreign Affairs (MOFA); the Ministry of Public Security(MOPS); the External Affairs Commission of the Communist Party of Vietnam; and theGovernment Committee for Organizations and Personnel (GCOP) Decision 340/TTG, anunder law, was issue to regulate INGO activities

PACCOM note that “by issuing permit, a clear legal framework for INGOs to operate inVietnam has been established, providing INGOs with official recognition” Indeed, althoughpermit procedures became clearer after Decision 340/TTG was issued, some concernsremained, and new ones arose In 1999, the Vietnamese Government issued Decision 28regulating the reception of INGO assistance and procedures for approving projects involvingVietnamese organizations During 2000, Decision 28 amendments were discussed todecentralize decision making and enable local authorities to approve projects at higher budgetlevels

In July 2000, the Committee for NGO Affairs was disbanded in line with public administrativereform while the role of VUFO and PACCOM did not change; GCOP was increasingly

become the central agency responsible for emerging local NGOs and was taking the lead in

developing appropriate regulations Thus, administrative responsibilities for internationalNGOs and local NGOs were separate.

Currently, INGOs apply for a permit with PACCOM and submit quarterly reports toPACCOM Projects with local partners must be approved by competent local authorities or theMinistry of Planning and Investment Implementing partners can be line ministries such asMOLISA, MOH, or MARD; mass organizations; universities; or local organizations

3.1.2 Current situation of the INGOs in Vietnam and their projects

The aid of INGOs has been implemented in different regions in the whole countries andfocused more on the sectors of education, health, social issue and economic development withtotal value of disbursement capital organizations by end of 2007 is at US$216 million(increased by over 200% against 2003)

The years of 2004 marked asignificant breakthroughwhen this was the first timeINGOs committed to aid thesum of USD 100 millions inthe Conference of Donors toVietnam Aid of INGOs inVietnam in 2004 hasconsiderably increased incomparison to previous ones

Source: NGO Resource Center, 2008

At the same time, the change in quality of the relationship between organizations withVietnam also took place

Here in some analyses of aid from INGOs:

3.1.2.1 By origin:

NGOs from North America (including USA and Canada) take for 42% of quantity oforganizations and 58% of total aid value The aid value from American NGOs has increased

by 170% against 2003 (from 46.7 million USD to 126 million USD)

NGOs from Europe accounts for 39% of quantity of organizations and over 28% of total aidvalue to Vietnam while organizations from Asia-Pacific takes account 19% of quantity oforganizations and 4% of aid value

Fig 1: Aid by INGOs through years

NGOs from North America have increased aid value and make up big portion in the total ofdisbursed aid,

Fig 2 : Aid from INGOs by origin (by end of 2007)

Source: NGO Resource Center, 2008

Project on vocational training, scholarship, teaching tools supplying are still paid in attention

On the top of priority list,projects on health have theaid value increasing by 20%against 2003 Some projects

on construction andequipment supplying for big

implemented and put intooperation Especially,projects on HIV/AIDS

126

60 30

58 28 4 0

10 20 30 40 50 60 70 80 90 100 110 120 130

% no.orgs

Aid value % of aid

value

North America Europe

Asian Pacific

attract attention fromAmerican NGOs.

Source: NGO Resource Center, 2008

In the sector of economic development, many projects has shifted from individual activitiessuch as income generation, credit support to the economic capacity building in overall.Approaches are flexible, more comprehensive and implemented by different interventionmethods to enhance the capacity of generating, controlling and maintaining family income.However, the aid value just stands in the level of 30.6 million USD

In the society development, INGOs activity participate into programs of house building for thepoor, taking care of children, preventing HIV/AIDS The total value has increased by 60%,from 20.2 million USD to 32.4 million USD in 2004 while the total aid value for environmentand emergency aid make up small portion in the total disbursed value (only 6%)

3.1.2.3 By region:

Aids from INGOs cover almost regions and provinces in Vietnam but still focus on sometraditional regions such as provinces in Centre and tend to shift to some provinces in CentralSouth region and Mekong delta region Some provinces still attract aid in high value such asHochiminh, Hanoi, Danang, Quang Tri, Quang Nam, Thua Thien-Hue while not much aid isinvested in some poor provinces in the mountainous areas such as Lai Chau, Dien Bien, TuyenQuang, Cao Bang

Aids from INGOs directly aimed at beneficiary and essential needs of community and localpeople Many organizations increased aids for policy lobby on faire trade, mutualunderstanding between Vietnamese people and others, supporting orange dioxin victims….Although some countries in South East Asia and South Asia were damaged by Tsunami andattracted attention from international donors but it is not likely to decrease in aid of INGOs inVietnam in 2005 Aids from American NGOs can be increased due to huge aid on HIV/AIDsfrom American Government

(See Annex 1 for more information on INGOs in Vietnam)

3.2 Monitoring and evaluation of INGOs in Vietnam

Current situation

As there is no recent and official study or report about this issue, a qualitative research,conducted with a series of interviews with the leaders and practitioners in the developmentprojects in Vietnam This research was conducted with a sample of 20 development expertsand practitioners from 20 international non-governmental organizations in Vietnam, whichhave been working in Vietnam for at least 5 years The interviewees include both programleaders at the organizations’ head offices and field offices

They were interviewed following a questionnaire of 5 key questions about the issue ofmonitoring and evaluation of development projects in Vietnam (See Annex 2 for the questionnaire of this qualitative research and Annex 3 for the list of respondents). From theresearch, there are below some overview about how M&E is implemented in internationalINGOs in Vietnam:

Understanding of M&E

Being asked about how they define M&E for the development projects, there are 17respondents can provide a clear definition of monitoring and evaluation The rest were mixedbetween monitoring and evaluation or consider M&E is solely evaluation These incorrectanswers go to the respondents from the local field sites

Perceptions about the importance of M&E

Generally, staff at head offices were greatly favor of M&E goals and objectives Theyperceived such activities to be one of the most important stages of the project cycle (if not themost important) A typical comment here was that “M&E is an internal tool for improvingstandards and strengthen practices; it is an increasingly essential component of the projectcycle”

For the perceptions of M&E in the field offices, it is quite revealing that the junior staff wereresponsible for undertaking daily reporting and monitoring activities Such reporting systemswere viewed by field staff as instruments through which senior managers could access theprogress of junior staff, as the following comment illustrates:

“…every week, I fill this report and give it to my supervisor at the end of the week, then every

month or so we meet to review what I have written and he assesses it and helps me understand what I have done wrong in my job…”

Clearly, staff at this level perceived M&E procedures as a highly sophisticated and technicalset of activities from which they were excluded by virtue of their inferior position.One respondent effectively summarized this perspective when he stated:

“…We still tend to think of M&E as a set of complex and specialized procedures that are beyond our understanding and to tell the truth, beyond our duties within this organization…”

It came as no surprise to learn that such activities held little interest for junior field staff and sowere undertaken without much enthusiasm Such widespread feelings at this level had beenfurther explained by them that they don’t know the purpose of the information collated theydon’ t see where or how it is used or don’ t learn anything from this process

Even some senior field staff also perceived the M&E process to be ‘too technical’ and

‘too formal’, in addition to being undertaken largely for the benefit of partners anddonors, not themselves Therefore, M&E is not considered at their top of priorities The focusshould be put a lot more in the implementation side

The M&E practices at their organizations

 As a consequence to this awareness, up to 60% of the asked people said that theirorganization has both monitoring and evaluation activities The rest of which sayingthat although their M&E is in place, it is not yet systematic and just short-termevaluation The M&E is only conducted when required by the donor, and only at theend of the project, but not put into the implementation plan Some of the M&Eprocesses of organizations just for us to provide the evaluation on results, but not therecommendations for improvements

 Almost all of the respondents stated that they implement the monitoring and evaluation

as it is requested by the donors Two thirds of them saying that they just do it at the end

of project and one third saying that the monitoring and evaluation is not put into theirimplementation plan

 There are 3 among 20 interviewees revealed that their organizations have a specializedM&E expert that is in charge of the M&E for their organization’s programs For therest, there are program staffs from the national to the ground level to join in theprocess There are 6 over 20 organizations they involve the local partners andbeneficiaries for monitoring and evaluation

 The methods used in the asked organizations to monitor and evaluate their programsinclude qualitative methods, quantitative methods or both For the qualitative method,the tools are listed including:

o Desk review: project reading materials, proposal, periodical report, trip report, activity report, other materials, etc

o In-depth interview of involved people

o Focused group discussion

CHAPTER 4 MONITORING AND EVALUATION OF THE

“SUPORT NETWORK FOR

PEOPLE WITH DISABILITIES”

4.1 General background of the program

4.1.1 Disability and Agent Orange issue in

Vietnam and Quang Ngai, program

location

The chemical dioxin, found in defoliants used during

the Vietnam War, is the source of many health

problems in Vietnam Dioxin was a chemical

ingredient in the millions of gallons of defoliants like

Agent Orange sprayed over the Vietnamese

countryside by the American military during the war between the U.S and Vietnam.According to VAVA, there currently are 4.8 million of people believed suffering from theeffects of dioxin poisoning in Vietnam.1

Quang Ngai is a southern central province lying on the coast in the middle of Vietnam Theprovince was devastated during the war According to local authorities, there 14,800 registereddioxin victims in Quang Ngai, and a total of 47,000 PWDs2 in the province According to thelatest survey done by Duc Pho District Health Care Center in 2006, there are 284 people withlimb deformities that medical personnel believe are related to dioxin, and 751 children under

16 with disabilities Most of these disabilities go untreated; the Duc Pho District Health CareCenter staff report that only a small portion (about 85) of the 284 people with limb deformitiesthey believe were congenital anomalies causes by dioxin received any treatment, therapy orfinancial help

Services for PWDs in Quang Ngai are inadequate Although the provincial and district healthcare directors have worked hard to improve medical care and care for PWDs, resourcelimitations have severely hampered their efforts At present, care for PWDs is limited toproviding some preliminary examinations, small surgeries and rudimentary physical therapy topatients who have recently received surgery

Most children with disabilities do not attend school Many parents say that they do not havetime to take their disabled children to school, but if their child was mobile, they would allowthem to attend These children get very little attention from education or health authorities,leaving the families to help them in any way they can

4.1.2 Program goals, objectives and activities

The Support Network for People with Disabilities (SN-PWD) is an initiative developed by theEast Meets West Foundation (EMW) and supported by the Ford Foundation and other donorsand organizations to help create channels of medical, educational and vocational services fordisabled people Launched in the Quang Ngai Province in January 2007, the program will end

by December 2011

The key outcome for this program, if successful, will be the creation of long-term, sustainablechannels by which PWDs in the program area can access health care and other services

Project Location Duc Pho and Son Tinh Districts, Quang Ngai Province, central Vietnam

Project Goal Improve the quality of life for PWDs in Quang Ngai province, with a

focus on Duc Pho and Son Tinh districts

1 Provide medical screening for PWDs

2 Provide corrective surgery for PWDs

1 Vietnam Association for the Victims of Agent Orange - VAVA's national survey on victims of dioxin exposure.

2 Quang Ngai Department of Labor, Invalids and Social Affairs's statistics.

Objectives 3 Provide treatment for non-surgical cases.

4 Provide vocational training for PWDS

5 Help children with disabilities get an education

6 Improve the Duc Pho Health Care Center

7 Improve the Quang Ngai Orthopedics Station

8 Improve the skills of medical personnel treating PWDs

Outcomes/Results 1 PWDs in the two target districts will be screened and

differentiated Provide screening to 7,761 PWDs

2 The PWDs in two districts in Quang Ngai in greatest need of corrective surgery will receive this surgery, allowing them greater freedom of movement, improved life options and better employment prospects Provide corrective surgery for 500 PWDs

3 Provide prosthetics, physical therapy or other medical or physicaltreatment to 300 PWDs who are not surgical candidates PWDs

in this area who are not good surgical candidates will get physical therapy and other assistance, which will improve their lives with some of the same effects as in (1)

4 Provide vocational training to 125 PWDs Eligible PWDs will be better able to earn a living to support themselves and their

families

5 Disabled children will be able to get an education, which will improve their long-term life prospects Provide educational support to 150 children with disabilities

6 Provide in-service training to 50 medical personnel from the medical facilities in Duc Pho and Son Tinh

7 Provide training for 300 commune health workers

8 The capacity of the Quang Ngai medical facilities serving PWDs will be greatly enhanced, leading to long-term and sustainable improvements in the lives of PWDs The number of patients treated at the two facilities will more than double as a result of the capacity improvements

Activities 1 Conduct screening of all PWDs in the two target districts.

2 Organize the list of PWDs into five categories:

a Those who are good surgical candidates

b Those who are good candidates for physical therapy and/

or prosthetics

c Those who would benefit from vocational training

d Those who are children in need of help getting an education

e Those whom the program cannot help at this time

3 Select 500 PWDs eligible for surgical correction, and refer them

to the appropriate medical facility, paying for their expenses

4 Select 300 eligible for non-surgical intervention, and arrange for their therapy

5 Select 150 children eligible for educational assistance, and develop appropriate interventions such as community tutoring, transportation to school, etc

6 Refurbish and equip the Rehabilitation Center at Duc Pho District Hospital

7 Build and equip an in-patient physical therapy ward at the built Quang Ngai Orthopedics Station

newly-8 Support in-service training for doctors and nurses assisting PWDs

Project partners: 1. Da Nang Provincial Orthopedics and Rehab Center

2 Quang Ngai Provincial Children, Family and Population Committee

3 Provincial Vietnam Association for Victims of Agent Orange

4 Duc Pho District Health Center (to be upgraded to Duc Pho District Hospital in 2007)

5 Quang Ngai Orthopedics Station in Son Tinh District, Quang Ngai

6 Duc Pho and Son Tinh District Children, Family and Population Committee

7 Da Nang and Quang Ngai Vocational Training Centers

Time fram January 2007 - December 2011 (five years)

Estimated budget Five years: $667,787

4.2 Monitoring and Evaluation of the Support Network for the People with

Disabilities

4.2.1 Measurable objectives and indicators

To be able to effectively monitor and evaluate the program, it is essential to set up theobjectives which is SMART (Specific, Stimulating, Measurable, Motivating and Achievable)and the indicators of success for both the outputs as well as the impact on beneficiaries

For the SN-PWD program, there are five identified desired outcomes For each, EMWdeveloped a set of baseline criteria and objectives as part of the implementation

The five identified desired outcomes are:

1 The PWDs in two districts in Quang Ngai in greatest need of corrective surgerywill receive this surgery

therapy and other assistance.

3 Eligible PWDs will be better able to earn a living to support themselves andtheir families

4 Disabled children will be able to get an education

5 The capacity of the Quang Ngai medical facilities serving PWDs will be greatlyenhanced This will thereby create long-lasting channels for PWDs to accesshealth care and other services after the SN-PWD finishes work

On the next page is a chart showing the objectives, the specific related activity, the specificnumerical goals, and the related outcomes

Objectives and Outcomes Analysis

Goal: To improve the quality of life for the people with disabilities (PWDs) in Quang Ngai province, with a focus on Duc Pho and Son Tinh Districts area

Indicator

Effect Indicator Notes

1 Provide medical

screening for PWDs District health facilities will screen all PWDs in the two

target districts so that they can be categorized for proper treatment

7,761 PWDs will receive screening and be assigned to acategory

PWDs in Quang Ngai will be organized for proper treatment and care

Medical screening will be paid for by the local health

authorities

2 Provide corrective

surgery for PWDs Some PWDs will be treated in Quang Ngai, and the rest

will be referred to Da Nang for surgery

500 PWDs will receive the surgery they need to correctdeformed limbs

PWDs will gain greatermobility and thus a better chance at getting

an education or gainful employment

The costs of medical treatment for referred patients will be paid by SN-PWD program

3 Provide treatment to

non-surgical cases

PWDs that are not good surgical candidates will be given physical therapy or prosthetics, or both, in QuangNgai Deaf children and adults will be given special education

300 PWDs will get therapy or

prosthetics

PWDs will receive the therapy they need to gain greater mobility and capacity so that they can attend school

or work

Costs for physical therapy will

be borne by the local medical authorities, except for special programs for deaf children to teach them to speak, read and write Prosthetics will be paid for by the program as part of the equipping of the Quang Ngai rehab center (objective

#7)

4 Provide vocational

training to PWDs PWDs that are in need of training will be given

practical training in suitable areas of work, such as tailoring, computers,

125 PWDs will receive training PWDs in the two project districts will

develop greater skills, income-earning possibilities and

This portion of the program will be coordinated with existing vocational training programs

- CFVG MBA 14 th Intake Page 28

-restaurant work, etc independence

5 Help children with

disabilities get an

education

A community-based tutoring program will be set up for 75 children in each district

Their needs will be assessed, and the proper help provided – books, special education forthe deaf, wheelchairs,

transportation to school, tutoring, etc

150 children will beenrolled in school

or otherwise offered a chance at

an education

Children will be able toget a good education, which will dramaticallyimprove their life chances

Community-based classes for deaf students, transport to school and other direct serviceswill be organized and paid for

by the SN-PWD The 150 children will be enrolled in year one and supported for fiveyears

6 Improve Duc Pho

Health Care Center

Refurbish and equip the hospital with better facilities for treating PWDs

The number of PWDs treated at thehospital will double

in three years, from 1,300 patients to 2,600 patients per year

The hospital will be much better able to provide medical treatment for PWDs on

a sustainable basis

The costs of treating new patients will be borne by the district medical system, gov’t social assistance agencies and/

or the patients themselves

7 Improve the Quang

Ngai Orthopedics Station Build an in-patient ward withfive rooms (including rest

room) and physical therapy equipment

The number of clients helped will triple in five years, from 500 per year

at present to an estimated 1,500 peryear in three years

The Quang Ngai Orthopedics Station will be much better able to provide corrective surgery and therapy for PWDs on a sustainable basis

The costs of treating new patients will be borne by the district medical system, gov’t social assistance agencies and/

or the patients themselves

8 Improve skills of

medical and other staff

serving PWDs, including

commune health workers

Provide opportunities for service training for medical staff in Duc Pho and Son Tinh Also provide training workshops for commune health workers

in-50 medical personnel will be trained, and 300 commune health workers

The medical staff in two districts will be better trained, and thus better able to assist PWDs with a variety ofneeds

Training will take place in the

Da Nang Orthopedics and Rehab Center for the medical personnel from the two facilities, and at local sites for the commune health workers

- CFVG MBA 14 th Intake Page 29

-4.2.2 M&E overall process, methodology and tools

A program of regular monitoring and evaluation has been used The method used to monitorand evaluate the SNPWD is a combination of both qualitative and quantitative methods Thekey tools include:

- Team weekly meetings, monthly meetings at district level, quarterly meetings atprovincial level (focus group discussions with the local partners and beneficiaries)

- Bi-monthly technical visits to the rehabilitation work at the field

- Frequent household visit (in-depth interviews with the beneficiaries plus observation)

- Annual review workshop

- Independent Consultant’s field trip for mid-term evaluation

- Database to count the number of beneficiaries

For a process, a benchmark survey will be done and then a monitoring and evaluation processput in place The M&E is reported on a quarterly basis for regular monitoring, and annuallyfor a complete program evaluation The process is below:

Benchmark Survey To obtain baseline data for comparative monitoring

Performance

Monitoring

Have two sub-components:

Monitoring of the direct services: This monitoring is

designed to ensure that the goals for providing direct servicesare met This will be reported in numerical format

Monitoring of the capacity-building component: To assess

the progress made in building durable channels for PWDs andCWDs to access medical, social and educational services

Evaluation Conducted annually and at the middle and the end of program

Reporting system On quarterly basis Include systematic compilation of

monitoring data and their aggregation for internal project useand external reporting A narrative report describes progressmade in improving capacity of local agencies and delivering onother goals

Benchmark Survey: When the project was approved, the Program Coordinator and his

assigned staff conducted a survey to collect baseline data for comparative monitoring andevaluation

The screening process is essential to the success of the program, since the screening willidentify both the PWDs who are most likely to benefit from the program, and also those who

- CFVG MBA 14 th Intake Page 30

-are considered priorities In conversations with local authorities, they emphasized that theirpriorities are to first offer treatment and other services to women, children, the poor and thedisadvantaged They also want to focus on victims of Agent Orange/dioxin, but this is mostlylikely not going to be part of the explicit criteria for this program (see logic model above) Thepriority of focusing on women, children and the poor is consistent with EMW’s approach.

Performance Monitoring: Regular field visits have been paid by the Program Director, who

relies on the Program Coordinator for regular reports These visits rely on the baseline dataand the planned indicators to assess the status of the whole project A quarterly report isproduced

During these monitoring visits, Customer Satisfaction Surveys are used to help EMW monitor,evaluate and improve the program, customer satisfaction surveys will be used This is acommon technique used by EMW to assess the level of satisfaction EMW clients have withour services All beneficiaries fill out the survey, either writing the answers themselves orthrough an interview format The responses will be part of the regular monitoring (reported on

a quarterly basis) and will be used to make changes in the program as the program is running

Evaluation: The annual evaluation is led by the Program Director She measured the results

based on the baseline data and the planned indicators in each objective area to draw upsuccessful factors as well as lessons learned

Mid-term Review: A mid-term independent evaluation was conducted by Mr Vu Ngoc

Khanh, an independent consultant and an M&E expert in June, 2009 to to assist the programstaff in changes and strategy that should be incorporated into the SNPWD’s second half andEMW future disability programming Since this is a new program with the possibility of asignificant expansion in later years, a mid-term evaluation was called for The detailedanalysis of the mid-term review will be provided in item 4.2.4

4.2.3 M&E personnel

The Monitoring and Evaluation (M&E) component of this project will be done incollaboration with program partners Of course, all stakeholders are involved, as outlined inthe partnership agreement and technical application

Program Director The Program Development Director on the EMW staff, will

provide overall supervision on the planning, implementation, monitoring and evaluation of the program

Program Coordinator The Program Coordinator be assigned to coordinate the

program and provide day to day management support and monitoring to the team based in Da Nang He is in charge of providing quarterly and annual reports, in consultation with theProgram Director He also manages daily work to make sure the activities meet the deadlines and expect outcome

Program Assistant A program Assistant is needed to provide logistic and

programmatic support to the program officer and program manager This assistant reports to the Program Officer on daily

- CFVG MBA 14 th Intake Page 31

-work and is responsible for a certain, of the daily -work deadlines and outcomes.

Direct beneficiaries Provide feedback on their levels of satisfaction

Local partners Joining in the research process by interviewing beneficiaries

and also provide their comments on stakeholders satisfaction survey

4.2.4 Monitoring meeting with local partner

At provincial level and district level, the program was led by a program managementcommittee (PMC) composed of representatives of relevant agencies At each level, regularmeeting of PMC was organised At provincial level, PMC’s members met every quarter while

at district level they met every month Through these meetings, PMB was able to updateinformation about PWDs situation and about program implementation progress, discussplanning, human resource, budget and other issues to address in order to solve any arisedproblems and catch up with the plan

In regard to the coordination between program partners and EMW, the roles andresponsibilities of PMC and EMW were also clarified to give a more dynamic environment foraction taking

PMC and EMW met regularly monthly Extraordinary meeting were also organised asnecessary when urgent decision was to be made (e.g to increase financial contribution fromSon Tinh district up to necessary level funding to cover the cost of facility upgrading)

4.2.5 Mid-term evaluation

The mid-term review plays a very important in the whole process of monitoring andevaluation of the SNPWD, as it is conducted at the very middle of the campaign by anindependent consultant and will help to provide reference for the formulation of strategies oradjustments for the second half of the program It is quite a comprehensive evaluation, thebelow information describes in further details the objectives, methodologies, process andresults of the evaluation

4.2.4.1 Objectives

The objectives of this mid-term evaluation are to:

- To review program progress and accomplishments against planned objectives

- To assess current capacity by the relevant local authorities in the providing andadministering of services to PWDs

- To recommend changes of strategy and emphasis that should be incorporated into theSN-PWD’s second half and EMW’s future disability programming

-was collected through questionnaires, key informant interviews, in-depth, structureddiscussions, focus group discussions and visits to beneficiary’s family The field work wascarried out in Duc Pho and Son Tinh districts, Quang Ngai Province

4.2.4.3 Activities

Questionnaires were used with representatives of PMC at provincial and district levels as well

as with local volunteers at communal level Three kinds of information were asked to thesegroups of informants

- their opinion about project activities

- their opinion about project impacts in term of changes for PWDs

- their recommendations for project

Focus group discussion was organised with provincial and district PMC’s members aboutchanges in institutional and individual capacity to administer and provide services for PWDs,and about individual commitment to implementation of support network to PWDs

Focus group discussions with volunteers were about their work, their success stories, theirdifficulties in work and solutions to overcome and recommendations

Semi-structured interviews were used with families with focus on support received andconditions of success and with teachers on their view of inclusive education and the benefices

of project support to CWDs

The field work was carried out in Duc Pho (Pho Ninh commune and Duc Pho town) and SonTinh districts (Tinh Minh commune and Son Tinh town) During field work, apart frommeeting was with district PMC and communal local volunteers, visits were made to familiesand children/people with disabilities

Most of group discussions and interviews were recorded with approbation of informants Aspecial introduction and commitment to confidentiality was made to informants by theconsultant before each meeting

4.2.4.4 Evaluation findings

The field surveys of the mid-term review provide the below findings, as well as the feedback

on how to improve the SNPWD:

Program approach:

In term of program approach, the combination of surgery, rehabilitation, education, vocationaltraining has given the program a comprehensive dimension of intervention approach Theprogram was also good at creating necessary ground conditions for program success and forsustainability through its support to facility upgrading and capacity building for local health

staff This combination has been seen by local partners as the most advanced feature of this

program in comparison with other programs addressing disability issue

- CFVG MBA 14 th Intake Page 33

-Program work method: In term of work method, it is satisfactory that the program has been

focusing, among others, on community-based rehabilitation The community-basedrehabilitation has been seen as a strong point of the program since the program was able to set

up a large network of local volunteers down to the village level The program method toaddress vocational training issue was a very innovative method that has been showing goodresults In regards to facility upgrading, the program has chosen to focus on upgradingfacilities at district level

Activities and results:

In term of activity implementation, the program has been going beyond the initial plan Moreactivities than it was planned have been implemented More resources than it was budgetedhave been mobilised For activities that were planned, most of indicators have been achieved

See Annex 4 for more details on the achievements in reflection with the planned indicators.

Below are the qualitative comments on the strengths, weaknesses of the program:

Current capacity by the relevant local authorities in the providing and administering services

to PWDs: The SNPWD is the most extensive network in Vietnam with support volunteers in

each village Therefore, it enables services administrators to be effectively updated aboutnumber of PWDs and their general situation EMW is also working in close coordination withthe program management committee (PMC) regular meetings, to ensure that the network canwell integrate into available services

In the course of program implementation, the program management team as servicesadministrators have worked out with EMW to come up with new activities to address theseneeds Where the current resources can not meet with the needs identified, local partners andEMW team were able to develop proposals and to obtain external funding The programmanagement committee was able to mobilise funds to match the funding gap of the upgrading

of the health care centres in Duc Pho and Son Tinh districts due to the increased costs ofconstruction

Through their participation in program, program management team members believe that theyhave learned great experiences in program design and program cost They already think aboutdeveloping, at the end of the program life, a new proposal with ambition to cover moredistricts and submit it to the Provincial People Committee for funding

Capacity to provide services to PWDs: The screening was organised by the local partners with

participation of local volunteers and local health services under supervision by the EMW.Local volunteers are all confident that they are able to play the role of information providerand counsellors for usual issues In order to equip local volunteers with knowledge and skills

to support PWDs’ rehabilitation, the program has organized theoretical and practical training.The training and coaching have enabled local volunteers to not only effectively participate inactivities organised by the program but also proactively take action to monitor local situation

in regards to PWDs, to provide timely advises and rehabilitation support to PWDs and tocoordinate with district partners to improve services to PWDs

- CFVG MBA 14 th Intake Page 34

-CWDs account for a quite high percentage of PWDs The program has supported -CWDs withscholarship and learning aids to enable them to go to school or to continue their education.Interviewed teachers have showed that they understood the benefices of inclusive educationand really support its implementation A part from having provided CWDs with education,teachers have also cared about how other children have interacted with CWDs They weretherefore able to convince other children to play with CWDs

Challenges for a better service to PWDs:

Although the capacity of services administrators and local services providers have beenimproved through out the process of program implementation, the mid-term evaluation alsoshow that they still face many challenges:

 Time is the first challenge mentioned by services administrators and lack of funding tocarry out all the services that may help PWDs is another challenge

 Services administrators are also concerned about the accessibility and availability ofservices for PWDs as well as the affordability of services

 In regards to the implementation of rehabilitation at community level the challenge is

to maintain the local support network and to keep local volunteers motivated

 Local volunteers also find the job they are doing for PWDs is sometimes reallychallenging due to the high expectation of PWDs and their families Today, eachvolunteer is in charge of providing services to PWDs in at least one village Thismakes much more difficult the outreaching to all PWDs

 PWDs have different needs As volunteer used to deal with one or two cases that theyhave selected on the basis of high success probability, volunteers face the challenge ofdealing with a new case that is different from cases they are used to

 Since the practice of community-based rehabilitation is new, in order to avoid failurethat may negatively affect future work, local volunteer usually select a patient whoseresults of rehabilitation can be significantly and rapidly visible For more serious cases,they have learned to refer them to a higher level of care or specialists

Conclusions, lessons learnt and Recommendations

- CFVG MBA 14 th Intake Page 35

-available services and capacity to coordinate, integrate these -available services The programhas also impacted local capacity to develop and implement actions for the benefits of PWDs

Strategic recommendations:

 Keep focus on two districts, while exploring opportunities for expansion

 Develop local partners’ capacity for fund raising for future expansion

 Strengthen local support network

 Consider link with family economic development

 Keep using the current institutional setting

 Providing support to create better and safer play environment for children

 Connecting trainees, trainers with an authorised agency and lobby for a certificatedelivery at the end of the training period wherever and whenever it is possible

 In order to improve monitoring and supervision system and better assess impact andresults, quarterly assessment mission is recommended

 Communication activities also need to be improved in order to create betterunderstanding between different stakeholders in order to facilitate cooperation andlong-term commitment to better service for PWDs

(See Annex 4 for a full report of this mid-term review)

CHAPTER 5

- CFVG MBA 14 th Intake Page 36

-EVALUATION AND RECOMMENDATIONS FOR THE

MONITORING AND EVALUATION OF THE

“SUPORT NETWORK FOR PEOPLE WITH DISABILITIES” 5.1 Evaluation

5.1.1 Strengths

- The most important strength that the monitoring and evaluation work of the Support Network for the People with Disabilities is that it is a result-based M&E, which is an advanced way of monitoring and evaluation, which is different with traditional M&E that is applied by a lot of current organizations:

Results include outputs together with outcomes and impacts We do this to stress the focus onanswering the “so what” question Building a rehabilitation center, providing a surgery, ortraining rural clinic workers does not answer the “so what” question These are outputs andnow one goes on to say “so what.” What are the results of having this rehabilitation centerbuilt, this surgery provided, or these trained clinic workers?

Vs the traditional one, which focus only on provision of inputs and production of outputs,result-based M&E which EMW is applying for SNPWD is more focused on the perceptions ofchanges among stakeholders, with systematic reporting with more qualitative and quantitativeinformation on the progress toward outcomes, thanks to the data collected on outputs andwhether they can contribute toward achievement of outcome

- The monitoring and evaluation for the SNPWD is successful because it is a participatory M&E

The program is able to involve both whom is affecting (program staff, local partners) as well

as is affected by the impacts of the program (direct beneficiaries) in the project managementand M&E processes in particular (measuring, recording, communicating information) Thishelps fulfilling a range of purposes: empowerment of stakeholders to take action, improvedpublic accountability and improved information provision for strategic planning at differentlevels Through involvement of community and program stakeholders, community memberscan articulate their priorities and criticism of program strategies

Involving local people in project provides an opportunity to the program manager of SNPWD

to assess deficiencies in the project design – if the work plans were realistic, if local funding isadequate and whether the project is actually owned by the people This itself is anachievement in capacity building at local level, which will result in the program’ssustainability

- Practical tools which are applicable in local context

All the M&E tools (formats of meetings, questionnaire survey, focus group interview manual,meeting minutes, format of reports,…) are very practical and applicable in local context Forsome organizations, who have an M&E system which is ideal and very systematic, but whenapplied in Vietnam, they reach a lot of difficulties

- CFVG MBA 14 th Intake Page 37

Monitoring and evaluation of the SNPWD is supported by an enforced and professional management system of East Meets West Foundation

All the management levels (from Program Director, Program Coordinator and ProgramAssistant) are very professional in the project management job The monitoring and evaluationprocess is comprehensive and in place since the beginning, starting from the planning,implementation, conclusion Every step of the project is well and periodically monitored andevaluated from different levels of management, to make sure the program is on the right track.This professionalism is thanks to the expertise in project management that East Meets Westhas gained for the last 21 years

- Feasible timeline, indicators for success agreed in advance with the local partners

It is thanks to the participatory monitoring and evaluation model that SNPWD is applying andthe screening survey The screening process is essential to the success of the program, sincethe screening will identify both the PWDs who are most likely to benefit from the program,and also those who are considered priorities In conversations with local authorities, theyemphasized that their priorities are to first offer treatment and other services to women,children, the poor and the disadvantaged Then, all parties can agree from the beginning

- Follow-up actions after M&E

The monitoring results and reports are carefully reviewed by the program manager and theProgram Director of the organization Adjustments for activities as well as the expect outcomewill be done if necessary, as well as the strategic decisions can be made if required Thanks toeffective monitoring results of the SNPWD project in Quang Ngai, strategic decisions weremade to expand and reduplicate the project into two similar projects in Thai Binh province inlate 2007 and Danang in 2008

- Under the program, services administrators were able to identify problems that might prevent the services to be appropriately implemented and to make decision that helped solve the problems

For example, when it was noted that Son Tinh district has some delay in implementation ofsurgery activities, the program management committee has taken initiative to organize ameeting with EMW to address this issue in order to accelerate the process to catch up with theinitial plan They have proceeded to a survey to know the reasons They found out thatplanning surgical operations for children during the school year was not appropriate becausesome families were reluctant to send their children to surgeries during the school year, fearing

of their children dropping out of school later They therefore made decision to “organisesurgeries for children during summer time so that they can attend” Also they havedetermined to replace the current person in charge of this activity by another staff who is moredynamic and active in order to respond more effectively to the program’s expectations Every year, the program management committee was able to organise monitoring visitstogether with EMW and DNOCH to assess the effects of surgical operations and of otherprogram support They have also cooperated with DNORC to organise training and follow-up

- CFVG MBA 14 th Intake Page 38

-for local volunteers The training and follow-up activities provided opportunities to evaluatelocal volunteers’ skills and to upgrade them accordingly

5.1.2 Shortcomings, difficulties and challenges

- Capacity of the local partners some is still limited

Capacity constraints are often more acute at field level, where understaffed and resourced local offices find it difficult to process and share the information coming from otherdonors Weaknesses persist in the capacity of several evaluation units, in terms of both thefinance and staff needed for planning and managing the evaluations Because of the lack ofcapacity in the local people’s evaluation function, the program managers have insufficientresources to gauge the impact of the activities

under Difficulties keep local partners – their motivation working for M&E

In regards to the implementation of monitoring and evaluation from the community level thechallenge is to maintain the local support network and to keep local volunteers motivated Themonitoring and information system sometimes suffered from lack of motivation of localvolunteers due to lack of financial incentives Service administrators know that if they canarrange for some minor fund to refresh the community volunteers with the knowledge andskills, they can keep this network sustainable They may need to submit a proposal to thefunding agency within the health service for this funding

- There is a need for comprehensive strategies and systematic tools for M&E

The current M&E model that EMW is applying for the SNPWD program is based on theexpertise of the program leaders and staff It is participatory and results-based but the tools arebeing used is just limited at basic qualitative research and analysis It lacks of advanced toolscurrently used widely all over the world for large organizations, such as modeling, rankingand prioritization, etc

- The monitoring and information system sometimes suffered from lack of motivation of local volunteers due to lack of financial incentives Service administrators know that if

they can arrange for some minor fund to refresh the community volunteers with theknowledge and skills, they can keep this network sustainable

5.2 Recommendations

In order to improve the monitoring and evaluation of the East Meets West’s Support Networkfor the People with Disabilities, it is necessary to overcome these above mentionedshortcomings and difficulties that exist in the current M&E model and process In this part, theauthor proposes some solutions with a hope that they will help to improve the implementation

of M&E and then the project management of the program

Strategies and tools

- CFVG MBA 14 th Intake Page 39

In order to improve monitoring and supervision system and better assess impact andresults, quarterly assessment mission is recommended The assessment shall focus onconcrete results in order to better draw lessons learnt

- Communication activities also need to be improved in order to create betterunderstanding between different stakeholders in order to facilitate cooperation andlong-term commitment to better service for PWDs Contents, means and targetedgroups of communication shall be regularly reviewed through a consultation process inorder to improve the quality and effectiveness

Personnel

- Yet the inclusion of the local partners in evaluation requires a strengthening of theirevaluation and statistical capacities Then, East Meets West program managers shouldtake time and cost money to provide training (theoretical training) for the localpartners to increase both the capacity and awareness about the importance ofmonitoring and evaluation and how it benefits the project management in general

- There is a plan from the local partner in requesting the funding for the M&E activitiesshould be provided to the government, so as to motivate the M&E local network

- Further increase the involvement of the local people to every steps for the monitoringand evaluation Demonstration from the program staffs to the local people should also

be used during the time the program staffs work together with the local people It is thebest way of capacity building

- An M&E expert should be employed for the whole organization He/she will be theone to create the systematic M&E tools for the whole organization, set the standards,increase the awareness of the whole organization and provide the necessary trainingfor each program staff on how to do the monitoring and evaluation the best way

- CFVG MBA 14 th Intake Page 40