the factors needed to form a scientifically appropriate and successful curriculum for a collaborated project between Topscore Educational Consultancy Company and Doan Thi Diem Primary School

Research Process...9 CHAPTER 1: LITERATURE REVIEW...11 1.1 History and Evolution of Curriculum Development for Primary Level...11 1.2 Curriculum Development Tools for Primary Level...12

Table of Contents

INTRODUCTION 3

1 Research Background 3

2 Research Question 5

3 Research Objectives 8

4 Research Process 9

CHAPTER 1: LITERATURE REVIEW 11

1.1 History and Evolution of Curriculum Development for Primary Level 11

1.2 Curriculum Development Tools for Primary Level 12

1.2.1 Bradley’s Effectiveness Model for curriculum Development Indicators 13 1.2.2 Tyler’s Objectives-Centered Model 15

1.2.3 Stufflebeam’s Context, Input, Process, Product Model 16

1.2.4 Scriven’s Goal-Free Model 18

1.2.5 Stake’s Responsive Model 20

1.2.6 Eisner’s Connoisseurship Model 21

1.2.7 Value-Added Assessment 23

1.2 The Crossing between Curriculum Development and Management Tools 24

1.2.1 The importance of management in education 24

1.2.2 The Logical Framework Approach and its practice to enhance managerial activities at schools 26

1.3 Steps to establish evaluation system for curriculum 31

1.4 Criteria in setting the evaluation system 33

CHAPTER 2: METHODOLOGY 37

2.1 Research Methodology 37

2.2 Sample 38

2.3 Instrumentation 39

2.4 Data Collection 41

2.5 Method of Analysis 42

2.6 Summary 42

CHAPTER 3: DATA ANALYSIS 43

3.1 Overview of Cambridge International Education program 43

3.2 Data Analysis 45

3.2.1 The process of identifying problems 45

3.2.2 The series of surveys to the establishment of evaluation system 49

CHAPTER 4: RECOMMENDATION 60

4.1 The establishment of evaluation system 60

4.1.1 The first-level evaluation system 60

4.1.2 The second-level evaluation system 68

4.2 Challenges to program evaluation 69

4.3 Recommendations 71

CONCLUSION 74

1 Limitations of the research 74

2 Conclusion 74

1 Research Background

There is no doubt in considering education as an essential part of our life.One believes that educating and being educated is not only a great source to equiponeself with sufficient knowledge and skills, but also a necessary tool to thrive onlater in life However, for the past few decades, we no longer just encounter withthe question of whether a child is deprived of education or not We are confrontingwith how to best improve our existing educational system Lately, a good deal ofattention has been paid to how to evaluate existing and upcoming curriculum andeducational projects so that they could benefit students decently As a result, anumber of theories in the domain of education and teaching have becomeincreasingly refined to evaluate and assess a curriculum is beneficial for students orshould it be modified and improved

Curriculum, as being defined in Oxford Dictionary (Oxford University Press,2006), is first and foremost the courses that are taught by a school, college, etcetera In education, a curriculum is broadly defined as the totality of studentexperiences that occur in the educational process The term frequently alludesparticularly to an arranged grouping of guidelines, or to a perspective of theunderstudy's experiences in accordance with the educator's or schools instructionalobjectives Curriculum may fuse the arranged association of students withinstructional substance, materials, assets, and courses of action for assessing theaccomplishment of instructive objectives Additionally, curriculum split into severalcategories, such as the implicit and explicit, as well as the official curriculum andextra-curricular activities

There have been several notable works by Western educators and researchers

in this particular field of education since 1940s and each one of them providedifferent outlook on how to conduct the best curriculum for primary pupil students.Interestingly, mentioning evaluation and assessment, it is immediate to draw a

connection to managing and controlling, which is an important part of management

knowledge and practices

Managerial evaluation theories have proved to serve the utmost course- how

to best utilize existing resources to bring out the most effective and efficient results

An issue emerging among the educators and managers is that whether one shouldenable school systems to examine and critique their curriculum developmentprocedures with managerial tools Those in charge for curriculum planning surelyneed help to employ a good enough process for establishing goals and determiningthe appropriateness for their particular situation they are facing with, to applymanagerial philosophies in creating what is best for their students, in other words,their clients

Among the tremendous variety of existing managerial theories in controllingand evaluating for projects, one could easily note that Logical Framework Approach(developed by U.S Agency for International Development, 1969), which is amanagement tool commonly used for designing, monitoring, and evaluatingdevelopment projects, has gained popularity in both private business sector as well

as educational institutes The Logical Framework Approach continues to gainadherents, though it is a management tool invented more than 40 years ago

This thesis reports the findings of a thorough study to establish the factorsneeded to form a scientifically appropriate and successful curriculum for acollaborated project between Topscore Educational Consultancy Company andDoan Thi Diem Primary School Subsequently, these factors are in hope to beapplied in other educational institutions and establishments in Hanoi and inVietnam Emphasis is placed on the establishment of an evaluation system created

by the combination of existing educational and managerial tools, as well as therequirements of sufficient quality control during the development process In thatsense, central to the research study is exploration of the sharing role andresponsibility of participating members and experts, which take up a cruciallyimportant part in this educational and training project

So often that the educational management, or management in general, hasbeen brought down to too simple factors such as lack of funding to facilitateactivities, however, it is not the case being examined in this research It is

observable that the central factor in the failed progress of own or semi own projects lies in the lack of dedicated research and cooperation amongparticipating members In Vietnam, funding has never been enough due tocontinuous demands from education reformation and to the fact that Vietnam stillstands in the rank of low-income countries Despite that, one can notice the role ofgood management here, that being said, management refers to maximize or properutilization and make best use of limited and a scarce resource In that sense, goodresearch and cooperation among membership that enables the working consistentlyand discretely towards the betterment of curriculum plays a great role in the success

State-of the project

To assist in the possible improvement of this project, it is required theunderstanding of the essence of the work, including responsibility, staffing,collection, policies and procedures, funding and structure of academic curriculum

My research, therefore, has incorporated interviews with Cambridge InternationalEducation program staff and a questionnaire survey to all the participating studentsand their parents

2 Research Question

Great interest has been shown in the past decade in the curriculum field.Most of the emphasis has appeared to centre around multi-ethnic approaches, newacademic programs, building autonomy, new school organizational patterns, andmedia technology rather than around curriculum development procedures itself.Consequently, school systems have found ample help in determining which newprograms to adopt, often these programs have been package proposals completefrom goals to evaluation New patterns of school organization also have beenrecommended, beginning with such changes as adding classes for prekindergarten

on through un-grading the elementary and secondary schools and the making ofsuggestions for a middle school which would change the more traditional juniorhigh structure

By the same token, nowadays, Vietnam’s society has shown its great concern

of educational system and curriculum reformation It is observable that the number

of international programs conducted in metropolitan cities such as Hanoi or Ho ChiMinh City has been increasing relentlessly The involvement of English-taughtprograms demonstrates a great deal of parents’ concern for advanced educationalpath for their children, especially in order to aid them in the context ofglobalization

Among the leading schools for primary education, Doan Thi Diem primaryschool stands as a prestigious education establishment The school integrates severalinternational programs with A from England, B from USA and C from Singapore,with hope to provide students with a wide range of choices that matches theirparent’s satisfaction and requirements The international program in collaborationwith Top Score Vietnam, an educational consultancy company, using curriculumadoption and teaching materials from Cambridge University and its teachingconsultancy department, is called Cambridge International Education program(hereby is abbreviated as CIE at times)

During the last few years, the CIE program is incorporated into someselective schools in Hanoi, including primary schools Doan Thi Diem By 2010, theprogram was put into some schools in the city on the basis of voluntary registration

of parents and students To meet the high quality of training, CIE program focused

on recruiting teachers, in which the criteria of expertise and teaching experience arehighly competitive

Initially, the program used to be a collaborative project between Doan ThiDiem Primary School and EMG educational consultancy company However, due

to the lack of due responsibilities provided from EMG, the company itself has beenexcluded from the program and then the control was taken over by Top Scoreeducational consultancy The CIE program, hence, is now expected to learn fromthe experienced of the past failing incidence and hopes to advance the program’squality to the next level, especially in terms of curriculum

After two weeks of adapting and observing, it occurred to me that most ofthe problems this program has faced with lies in its internal process of assessing andcontrolling the quality of its curriculum development progress Curriculum, as being

understood in this research, is an organizational pattern for structuring leaning,which is the pre-planned dimension of the instructional program

The purposes of the study reported here, therefore, are to identify criteria forcurriculum development that are crucial, functional and observable so that it couldalign with the educational goals of the school itself as well as the individual goals ofeach students; and to test these criteria in existing school system Through initialanalysis, it is uncovered that there are three basic problems that were seriouslyhindering not only meaningful evaluation of this project, but also theirimplementation

Firstly, the planning was too vague where the objectives were multiple andnot clearly related to project activities There has not existed yet the clear picture ofwhat the project would look like if it were successful Thus, evaluators could notcompare – in an objective manner – what was planned with what actually happened.Secondly, and perhaps most importantly, the evaluation was an adversary process:with the absence of clear targets and frequent disagreements (even among projectteam members) as to just what the project was about, evaluators ended up usingtheir own judgment as to what they thought were “good things” and “bad things”

The subsequent evaluation results would then frequently become a basis forfurther argument about what was good or bad, rather than resulting in constructiveactions for project improvement Educational curriculum, on the other hand,requires a group of experts to examine scientifically and thoroughly its quality TheCIE program needs a council of experienced educational experts that could help it

to toss light on two questions: one, do planned courses, programs, activities, andlearning opportunities as developed and organized actually produce desired results;and two, how can the curriculum offerings best be improved

Through that understanding and observation being made at CIE project, thisresearch hopes to provide a tool for curriculum evaluation by examining and testing

a combined tool between education and contemporary management

3 Research Objectives

Having a keen interest in education as a tool for sustainable development, theresearcher has come to realize having an academic background of economics andmanagement as well as a concern in primary education itself is not a conflict Onthe other hand, this crossing gives interesting perspectives and outlooks on problemidentification and problem solving

The two main objectives of this thesis is firstly, to identify existing problemsrelated to CIE Program at Doan Thi Diem Primary School- a program to integrateinternational standard curriculum developed by Cambridge University intoVietnamese educational system Secondly, through the examination of theseproblems and pitfalls, the researcher hopes to give out solutions to two issues: towhat extent the current developed curriculum has fulfilled the educational goals ofDoan Thi Diem Primary School; and the initiatives to improve the current quality of

curriculum using neat tools from managerial science, with help of a comprehensive

evaluation system

In the context of constantly changing competition, even an educationalinstitution now must strive to find out its prevailing points to satisfying itscustomers, while staying true to the nature and significant of true and righteouseducation In that sense, this report is as well hoped to contribute as an insightfulpaperwork to this important project of Doan Thi Diem Primary School

The research’s mere ambition is not to provide a thorough inspection of allaspects covered by these abovementioned theories; but to put some moreperspectives into the case of this CIE Program with hope to enhance its operationalactivities Even though the time of researching at Doan Thi Diem Primary Schoolwas tentatively short and future research is indeed needed to consult the program inmuch greater details, this research is still expected to stand as a suggestion from theview of a business student to the program to perform in its most effective andsuccessful way

4 Research Process

To fulfil the research objectives, the following session dwells on thebackground that forms the academic side of this report There are two schools oftheories, namely Curriculum Development tools and Logical Framework as amanagement tool that are explained and described in an adequate brief

After examining these theories, the author spends the rest and majormagnitude of this report to analyse the situation and context of CIE Project inaccordance with the preceding theories The analysis is solely established in thebasis of the internship I have conducted at Doan Thi Diem Primary School fromDecember 2014 to April 2015 At the end of the report, solutions and suggestionswere generated in the form of a comprehensive evaluation system

There stood three major steps in this research, including identifying problems(determining research questions), interviewing and questioning (determiningresearch objectives and research process) and giving out solutions (determiningresearch conclusion)

Throughout these three major steps, a host of surveys, questionnaires andinterviews would take place with the participation of all different stakeholders ofCambridge International Educational program, which shall be listed in the nextsession of this report The inquiry documents and the overall process aimed to beshort and concise in order to gain the information needed without having tooveranalyse pages of data, which could help to prevent an extended period of time

Figure 1.1: Research process at Cambridge International Education program

Interview and give questionnaires to other stakeholders

Interview School Board Members and Project managers to identify Project's objectives Interview parents and students to identify their expectations and levels of satisfaction

Give solutions

Establish an evaluation system combining stakeholders' opinions Give out some initatives to obtain best

results

CHAPTER 1: LITERATURE REVIEW

1.1 History and Evolution of Curriculum Development for Primary Level

Curriculum assessment in general has a long history As Guba and Lincoln(1981) pointed out, a Chinese head in 2200 B.C obliged that his open authoritiesexhibit their capability in formal competency tests In the United States, thesympathy toward assessing schools can be followed at any rate as far back as theproposals of the Committee of Ten, which toward the end of the nineteenth centuryset maybe the first illustration of "evaluative gauges" for a country's auxiliaryschools (United States’ National Education Association, 1969)

As of late, then again, the enthusiasm for educational module assessmentspecifically has appeared to increment especially Society's emphasis on instructiveresponsibility, the specialists' requests for instructive change, and the instructor'sattendant requirement for confirmation of results have all added to the currententhusiasm for speculations and systems for educational module assessment.Unfortunately, much of this interest seems to have resulted in an ill- conceivedobsession with test results and test results only Therefore, a more extensiveviewpoint and more expanded methodologies appear essential

Initially, primary school was synonymous with essential instruction, takingyoungsters from kindergarten through evaluation As the populace developed andgot to be progressively urban and rural rather than country, the one-room schoolbuilding offered path to the multi-room school building, which got to be variousschools The change of rudimentary training is clear With a consistent, unfalteringclimb in differing qualities in a huge number of nationwide grade schools, theinstructive methodology of educators must adjust

While school understudies of the twenty first century took the fundamentalclasses in primary school (Social Studies, Science, Language Arts, Math, and soon.), numerous schools today are changing their curricula and joining classes, forexample, Mandarin Chinese training Despite the fact that the ordinary instructiveclasses are still by and by, an alternate method is being met by directors and

instructors so as to successfully show all understudies, and stay aware of a changingand advancing society

Curriculum evaluation, therefore, should be understood as the process of

collecting data on a curriculum to determine its value or worth with the aim ofdeciding whether to adopt, reject, or revise the programme, which is concerned byvarious parties For example, the public want to know whether the curriculumimplemented has achieved its aims and objectives; teachers want to know whetherwhat they are doing in the classroom is effective; and the developer or plannerwants to know how to improve the curriculum product

1.2 Curriculum Development Tools for Primary Level

Educational curriculum development is a methodology of enhancing theeducational module Different methodologies have been utilized as a part ofcreating curricula Ordinarily utilized methodologies comprise of analysis (i.e needexamination, errand investigation), planning (i.e target outline), selecting (i.e.picking fitting learning/showing routines and proper evaluation strategy)development (i.e development of the educational module usage board/ educationalmodule assessment panel) and checking (i.e educational module survey advisorygroup)

Effectiveness Model for Curriculum Development Indicators, for instance, amodern theory of curriculum development, holds that teachers as educationalexperts actively construct subjective feedbacks of what they are teaching based ontheir constant interaction on the daily basis with students (Bradley, 1985).According to this paradigm, an adequately effective curriculum must meet theexpectation of being consistent, responsive and responsible to its clients Thissophisticated set of indicators, however, was not the pioneer in the field ofeducational evaluation

Many other theories have long been standing as utile tools for curriculumassessment, such as Objectives- Centred Model (Tyler, 1950) providing a relativelyeasy steps for overall controlling of curriculum development; Context, Input,Process and Product Model (Stufflebeam, 1971) with tremendous improvements

from the aforementioned theory; Goal- Free Model (Scriven, 1972) being the first toquestion the assumption that goals or objectives are crucial in the evaluationprocess; Responsive Model (Stake, 1975) making a major contribution by notingthat the concerns of the stakeholders should be paramount first and ConnoisseurshipModel (Eisner, 1979) which emphasizes qualitative appreciation As the movement

of constantly refined curriculum effectiveness and accuracy that has been there formore than half of the century, all of these ideas are bombarding those responsiblefor decision making in such matters

In this research, there are six of the most significant works being presentedand scanned as the primary research background Each of these mentioned tools

shall be presented through its (1) definition, (2) characteristics of each model; (3)

advantages and disadvantage of the model, with hope to form a big picture ofexisting curriculum evaluation systems in the world

1.2.1 Bradley’s Effectiveness Model for curriculum Development Indicators

The Bradley’s effective model for curriculum development indicators is a set

of measurable checkpoints, with simple responses (yes or no) to identify whetherthe current curriculum plan is set properly or not The full version of this model ispresented in the table below

Table 1.1: Bradley’s Effectiveness Model for curriculum Development

Indicators

No Vertical

curriculum

continuity

The course of study reflects a K-12 format that enablesteachers to have quick and constant access to what is beingtaught in the grade levels below and above them Also,upward spiraling prevents undue or useless curricularrepetition

for the same grade level

Curriculum

priority

Philosophical and financial commitments are evident.Clerical assistance is provided and reasonable stipends arepaid to teachers for work during the summer months inaddition, curriculum topics appear on school board agendas,administrative meetings agendas, and building=staffmeeting agendas

Broad

involvement

Buildings in the district have teacher representatives on thecurricular committees, elementary, middle level or juniorhigh, and high school principals (or designees) arerepresented; and school board members are apprised of andapprove the course of study

Long-range

planning

Each program in the district is included in the 5-tearsequence and review cycle Also, a philosophy of educationand theory of curriculum permeate the entire school district

Planned Tangible evidence shows that the internal and external

change publics accept the developed program course of study for

the school district The process of developing a course ofstudy for each program or discipline in a school district is nolonger one of determining how to do it, but one ofdetermining how to do it better

Source: Glatthorn, 2015

The indicators for effective curriculum development represent workingcharacteristics that any complex educational institution must have in order to beresponsive and responsible to its clients (Wiles J, 2002) Further, the measurementcan also be oriented to meet the needs of school regardless of scales, which canfocus on a specific evaluation of a curriculum aspect, such as literature, language,math, or any different content area The obvious advantage of this model is itssimplicity and readiness to use However, being as simple as it is, the model ismostly appropriate for overall assessment and not detailed enough to serve as athorough tool of curriculum evaluation

1.2.2 Tyler’s Objectives-Centered Model

One of the soonest curriculum evaluation models, which keeps on affectingnumerous evaluation activities, was that proposed by Ralph Tyler (1950) in hiswork Basic Principles of Curriculum and Instruction As elucidated in this work andused as a piece of evaluation attempts, the Tyler approach moved sensibly andpurposely through a couple of related steps:

(1) Begin with the behavioral objectives that have been previously determined.Those targets ought to indicate both the substance of learning and the understudyconduct expected as “Demonstrate familiarity with dependable sources ofinformation on questions relating to nutrition.”

(2) Identify the situations that will give the student the opportunity to express thebehavior embodied in the objective and that evoke or encourage this behavior.(3) Select, modify, or build suitable assessment instruments, and check theinstruments for objectivity, dependability, and legitimacy

(4) Use the instruments to obtain summarized or appraised results

(5) Compare the results obtained from several instruments before and after givenperiods in order to estimate the amount of change taking place

(6) Analyze the results in order to determine strengths and weaknesses of thecurriculum and to identify possible explanations about the reason for this particularpattern of strengths and weaknesses

(7) Use the results to make the necessary modifications in the curriculum(Glatthorn, 1987)

The Tyler model has several advantages, firstly as it is relatively easy tounderstand and apply It focuses attention on curricular strengths and weaknesses,rather than being concerned merely with the performance of each student The moststriking matter here is it also emphasizes the importance of a continuing cycle ofappraisal, examination, and constant improvement

However, according to Guba and Lincoln (1981), it still bears plenty ofpitfalls, namely: not suggesting how the objectives themselves should be evaluatednor providing standards or suggest how standards should be developed Else, itsaccentuation on the earlier explanation of targets may confine creativity incurriculum development, which appears to place undue accentuation on the pre-assessment and post-appraisal, disregarding totally the requirement fordevelopmental evaluation By the same token, Baron and Boschee (1995), in theirbook Authentic Assessment: The Key to Unlocking Student Success, stress that “weare encountering fundamental changes in the way we view and conduct assessment

in schools” (p.1)

1.2.3 Stufflebeam’s Context, Input, Process, Product Model

These obvious weaknesses in the Tyler model led several evaluationexperts in the late 1960s and early 1970s to criticize the Tyler model and tooffer their own alternatives The alternative that had the greatest impact was thatdeveloped by a Phi Delta Kappa committee chaired by Daniel Stufflebeam (1971).This model seemed to appeal to educational leaders because it emphasized theimportance of producing evaluative data for decision making; in fact, decision

making was the sole justification for evaluation, in the view of the Phi Delta Kappacommittee.

To service the needs of decision makers, the Stufflebeam model provides ameans for generating data relating to four stages of program operation: contextevaluation, which continuously assesses needs and problems in the context to helpdecision makers deter- mine goals and objectives; input evaluation, whichassesses alternative means for achieving those goals to help decision makers chooseoptimal means; process evaluation, which monitors the processes both to ensurethat the means are actually being implemented and to make the necessarymodifications; and product evaluation, which compares actual ends with intendedends and leads to a series of recycling decisions

During each of these four stages, specific steps are taken:

(1) The kinds of decisions are identified

(2) The kinds of data needed to make those decisions are identified

(3) Those data are collected

(4) The criteria for determining quality are established

(5) The data are analyzed on the basis of those criteria

(6) The needed information is provided to decision makers (Glatthorn, 1987, pp.273–274)

The context, input, process, product (CIPP) model, as it has come to becalled, has several attractive features for those interested in curriculum evaluation.Its emphasis on decision making seems appropriate for administrators concernedwith improving curricula Its concern for the formative aspects of evaluationremedies a serious deficiency in the Tyler model Finally, the detailed guidelinesand forms created by the committee provide step- by-step guidance for users TheCIPP model, however, has some serious drawbacks associated with it Its mainweakness seems to be its failure to recognize the complexity of the decision-makingprocess in organizations It assumes more rationality than exists in such situationsand ignores the political factors that play a large part in these decisions Therefore,

as Guba and Lincoln (1981) noted, it seems difficult to implement and expensive tomaintain.

1.2.4 Scriven’s Goal-Free Model

Michael Scriven (1972) was the first to question the assumption that goals orobjectives are crucial in the educational evaluation process After his involvement

in several evaluation projects where so-called side effects seemed more significantthan the original objectives, he began to question the seemingly arbitrary distinctionbetween intended and unintended effects His goal-free evaluation model was theoutcome of this dissatisfaction In conducting a goal-free evaluation, the evaluator

functions as an unbiased observer who begins by generating a profile of needs for

the group served by a given program Only then, by using methods that areprimarily qualitative in nature, the evaluator assesses the actual effects of theprogram If a program has an effect that is responsive to one of the identified needs,then the program is perceived as useful

Goal-free evaluator, therefore, avoids learning the stated purpose, goals orintended achievements, of the program prior to or during the evaluation Instead, heobserves and measures actual processes and outcomes; and interviews programconsumers, which prevents only looking at the program as it pertains to the intendedgoals at the risk of overlooking many positive and/or negative unintended side-effects The evaluator, subsequently, asks: “What does the program actually do?”rather than “what does the program intend to do?” As being stated in his mostnotable work, “Merit is determined by relating program effects to the relevant needs

of the impacted population, (Scriven, 1991 p 180).” a comprehensive needsassessment should be conducted simultaneously with data collection “Theevaluator should provide experiential accounts of program activity so that readers ofthe report can, through naturalistic generalization, arrive at their own judgments ofquality in addition to those the evaluator provides, (Stake, 2004, p 215).”

With this new approach, there stand a host of critical advantages for theutilization of this tool Firstly, it may identify unintended positive and negative side-effects and other context specific information As a supplement to a traditional

evaluation, it serves as a form of triangulating both data collection methods and datasources Secondly, it circumvents the traditional outcome evaluation and thedifficulty of identifying true current goals and true original goals, and then definingand weighing them

By the same token, it is, therefore, less intrusive to the program andpotentially less costly to the client By reducing interaction with program staff, it isless susceptible to social, perceptual, and cognitive biases; is reversible; anevaluation may begin goal- free and later become goal-based using the goal-freedata for preliminary investigative purposes Moreover, it is less subject to biasintroduced by intentionally or unintentionally trying to satisfy the client because it

is not explicit in what the client is attempting to do; it offers fewer opportunities forevaluator bias or corruption because the evaluator is unable to clearly determineways of cheating For the evaluator, it requires increased effort, identifiesincompetence, and enhances the balance of power among the evaluator and client

Lastly, and perhaps most importantly, its focuses on human experience andwhat people actually do and feel, allows for understanding how programimplementer deal with its non-programmed decisions (Stake, 2004), which should

be understood as decisions regarding relatively novel problems, or problems that anindividual, group, organization, or entity has never encountered (George & Jones,2000)

Scriven’s main contribution, obviously, was to redirect the attention ofevaluators and administrators to the importance of unintended effects—a redirectionthat seems especially useful in education If a mathematics program achieves itsobjectives of improving computational skills but has the unintended effect ofdiminishing interest in mathematics, then it cannot be judged completely successful.Scriven’s emphasis on qualitative methods also seemed to come at an opportunemoment, when there was increasing dissatisfaction in the research community withthe dominance of quantitative methodologies

As Scriven himself notes, however, goal-free evaluation should be used tocomplement, not supplant, goal-based assessments Used alone, it cannot provide

sufficient information for the decision maker Some critics have faulted Scriven fornot providing more explicit directions for developing and implementing the goal-free model; as a consequence, it probably can be used only by experts who do notrequire explicit guidance in assessing needs and detecting effects.

1.2.5 Stake’s Responsive Model

Robert Stake (1975) made a major contribution to curriculum evaluation inhis development of the responsive model, because the responsive model is basedexplicitly on the assumption that the concerns of the stakeholders—those forwhom the evaluation is done—should be paramount in determining the evaluationissues He made the point this way: “To emphasize evaluation issues that areimportant for each particular program, I recommend the responsive evaluationapproach” It is an approach that trades off some measurement precision in order toincrease the usefulness of the findings to persons in and around the program Aneducational evaluation is a responsive evaluation “if it orients more directly toprogram activities than to program intents; responds to audience requirements forinformation; and if the different value perspectives present are referred to inreporting the success and failure of the program” (p.14) Stake recommends aninteractive and recursive evaluation process that embodies these steps:

(1) The evaluator meets with clients, staff, and audiences to gain a sense of theirperspectives on and intentions regarding the evaluation

(2) The evaluator draws on such discussions and the analysis of any documents

to determine the scope of the evaluation project

(3) The evaluator observes the program closely to get a sense of its operationand to note any unintended deviations from announced intents

(4) The evaluator discovers the stated and real purposes of the project and theconcerns that various audiences have about it and the evaluation

(5) The evaluator identifies the issues and problems with which the evaluationshould be concerned For each issue and problem, the evaluator develops anevaluation design, specifying the kinds of data needed

(6) The evaluator selects the means needed to acquire the data desired Mostoften, the means will be human observers or judges.

(7) The evaluator implements the data-collection procedures

(8) The evaluator organizes the information into themes and prepares

“portrayals” that communicate in natural ways the thematic reports The portrayalsmay involve videotapes, artifacts, case studies, or other “faithful representations.”(9) By again being sensitive to the concerns of the stakeholders, the evaluatordecides which audiences require which reports and chooses formats mostappropriate for given audiences (Glatthorn, 1987)

Clearly, the chief advantage of the responsive model is its sensitivity toclients By identifying their concerns and being sensitive to their values, byinvolving them closely throughout the evaluation, and by adapting the form ofreports to meet their needs, the model, if effectively used, should result inevaluations of high utility to clients The responsive model also has the virtue offlexibility: The evaluator is able to choose from a variety of methodologies onceclient concerns have been identified Its chief weakness would seem to be itssusceptibility to manipulation by clients, who in expressing their concerns mightattempt to draw attention away from weaknesses they did not want exposed

1.2.6 Eisner’s Connoisseurship Model

Elliot Eisner (1979) drew from his background in aesthetics and arteducation in developing his “connoisseurship” model, an approach to evaluationthat emphasizes qualitative appreciation The Eisner model is built on two closelyrelated constructs: connoisseurship and criticism Connoisseurship, in Eisner’sterms, is the art of appreciation—recognizing and appreciating through perceptualmemory, drawing from experience to appreciate what is significant It is the abilityboth to perceive the particulars of educational life and to under- stand how thoseparticulars form part of a classroom structure Criticism, to Eisner, is the art ofdisclosing qualities of an entity that connoisseurship perceives In such a disclosure,the educational critic is more likely to use what Eisner calls “non-discursive”—a

language that is metaphorical, connotative, and symbolic It uses linguistic forms topresent, rather than represent, conception or feeling.

Educational criticism, in Eisner’s formulation, has three aspects Thedescriptive aspect is an attempt to characterize and portray the relevant qualities ofeducational life—the rules, the regularities, the underlying architecture Theinterpretive aspect uses ideas from the social sciences to explore meanings anddevelop alternative explanations—to explicate social phenomena The evaluativeaspect makes judgments to improve the educational processes and providesgrounds for the value choices made so that others might better disagree

The chief contribution of the Eisner model is that it breaks sharply with thetraditional scientific models and offers a radically different view of what evaluationmight be In doing so, it broadens the evaluator’s perspective and enriches his or herrepertoire by drawing from a rich tradition of artistic criticism Its critics havefaulted it for its lack of methodological rigor, although Eisner has attempted torefute such charges Critics have also argued that use of the model requires a greatdeal of expertise, noting the seeming elitism implied in the term connoisseurship

In summary, the innovative practices to which many educators aspire canaccommodate and build on more traditional mandates (Ferrero, 2006) Although themodels above seem sharply distinct from one another, some evidence of congruenceexists in current theories of evaluation This congruence is quite evident in theASCD monograph Applied Strategies for Curriculum Evaluation (Brandt, 1981),

in which seven experts in evaluation were asked to explain how their “evaluationmodel” would be used in evaluating a secondary humanities course While themodels proposed by the experts (Stake, Scriven, Eisner, and Worthen) differed inmany of their details, several common emphases emerged in the approaches:Study the context, determine client concerns, use qualitative methods, assessopportunity cost (what other opportunities the student is missing by taking thiscourse), be sensitive to unintended effects, and develop different reports fordifferent audiences

By using these common emphases, along with insights generated fromanalyzing other models, it is possible to develop a list of criteria that can be used inboth assessing and developing evaluation models Districts with sufficient resources

to employ an expert consultant can use the criteria to assess the model proposed bythe consultant; districts developing a homegrown process can use the criteria todirect their own work

The criteria will obviously result in an eclectic approach to evaluation, onethat draws from the strengths of several different models Such an eclectic processhas been used successfully in evaluating a field of study; this same process also can

be used to evaluate a course of study with the scope of the evaluation reduced.1.2.7 Value-Added Assessment

One typical type of assessment in the management of school is Added Assessment, which happens during the continuous refining process ofcurriculum planning and execution Value-added assessment is a technique ofusing test data to determine the value that teachers add to the learning of eachstudent It focuses on how test data can help each child academically This type

Value-of assessment makes it possible to isolate the impact Value-of the individual teacherand to respond with appropriate rewards or corrective training FormerUniversity of Tennessee statistician William Sanders, who helped pioneer value-added research, noted that this method offers a way to ensure that as fewchildren as possible suffer a disadvantage in learning Sanders (2001) believedthat “under effective teachers, students could make gains no matter what theirability or achievement levels were when they started” (p 3)

The official and standardized format of this assessment shall be presented

as follows:

(1) Each spring (or any other selected season), students take a referenced test in five core subjects (math, science, reading, language, and socialstudies) Non-redundant test items are added each year in an attempt todiscourage blatant “teaching to the test.”

norm-(2) Each fall (or any other selected season), districts receive a report cardbroken down by school and grade showing progress in each subject in thecontext of 3-year averages The system records student achievement gains inscale score points and in the form of comparisons to local, state, and nationalaverages School and district reports are reported to the public.

(3) Each teacher receives a report card—one that is not released to thegeneral public— showing the year-to-year progress of his or her students.Supervisors also receive these reports It is this application of these data thatholds the greatest promise for building a better teacher

This annual assessment helps students and especially their parents to keeptrack of the curriculum execution, a first-hand feedback comes straightly fromschool’s most important stakeholders The practice of this added yet importantself- evaluation also allows public to get access to the current situation ofteaching quality of the program However, in Vietnam, this valuable assessmenthas not been popularized to the point that only a few privileged schools such asprivate schools, foreign invested schools are reported to practice

1.2 The Crossing between Curriculum Development and Management Tools

1.2.1 The importance of management in education

It has frequently been observed that most of the world’s educationalinstitutions have been applied different forms of management tools in their dailyactivities (Deshmukh, 2010) The reason is that managerial tools along with itsnature provide us with facilities to best utilize limited resources to get the targetedgoals

Managerial evaluation is a central part of the academic mission of manyschools and a collective endeavor of all education institutions at large Theseevaluation activities shape the thinking of teachers and advance the public body ofknowledge that is conveyed in the classroom Management practices are embeddedwith values pertaining to the very definition of management, its objectives,priorities and sources of accountability During the first half of the twentiethcentury, schools struggled to make themselves aligned with different management

tools (Gordon and Howell, 1959) Over the course of the next five decades, schoolswent on to make substantial progress in building the prescribed body of knowledge

by adopting rigorous evaluation methods from allied disciplines (from the socialsciences to economics, mathematics or statistics) and applying them (at times in atrans-disciplinary way) to answering critical management questions

In the last decade, several scholars have criticized the current direction ofeducational evaluation Some of the criticisms (Pfeffer and Fong, 2002; Bennis andO'Toole, 2005; Mintzberg, 2004) have focused on the seemingly growing gapbetween academic research on educational evaluation and practice, which couldthreaten the relevance of the new knowledge for professional practice Another line

of criticism, which is central to the application of the Principles of ResponsibleManagement Education, has to do with the impact of management values This type

of criticism was most notably articulated by the late Sumantra Ghoshal (2005).Ghoshal argued that today’s dominant theories and frameworks have contributed tocreating, reinforcing and perpetuating harmful values among students

In summary, decades after the publication of the Gordon and Howell report(1959), not only have it is not yet developed the prescribed set of standards ofprofessional conduct but have in fact produced a set of theories and tools that mayhave implicitly created a wrong set of standards It is not clear whether currentmanagement frameworks and tools are of much help to schools through thetransition or whether it is through trial and error that schools progress through thedifferent stages and adopt different attitudes towards managerial responsibility.What is clear however is the growing need among educational leaders forframeworks and tools that will help them address the new objectives of social andenvironmental stewardship in a more systematic and effective way

Management along with effective leadership are, therefore essential ifschools are to achieve the wide-ranging objectives set for them by their manystakeholders, notably the governments which provide most of the funding for publiceducational institutions as the case seen in Vietnam In an increasingly globaleconomy, an educated workforce is vital to maintain and enhance competitiveness

Society expects schools, especially primary schools in particular, to prepare peoplefor readiness a rapidly changing environment Teachers, and their leaders andmanagers, are the people who are required to deliver higher educational standards.The concept of management has been joined, or superseded, by the language ofleadership but the activities undertaken by principals and senior staff resist suchlabels

In that sense, management is practiced in many countries, expanding thescope and scale of leadership and providing greater potential for direct and indirectinfluences on school and pupil outcomes Successful leaders are increasinglyfocused on learning, the central and unique focus of educational organizations Asthese environmental pressures intensify, leaders and managers require greaterunderstanding, skill and resilience to sustain their institutions Heads, principals andsenior staff need an appreciation of the theory, as well as the practice, ofeducational management Competence comprises an appreciation of concepts aswell as a penchant for successful action The next chapter examines the nature oftheory in educational leadership and management, and its contribution to goodpractice

1.2.2 The Logical Framework Approach and its practice to enhance managerial activities at schools

Peter Drucker (1969) once affirmed that management is the setting ofobjectives This much is certain that in case an institution or organization have noobjectives, then the relative value of any course of action cannot be compared toalternative courses of action Therefore, among thousands of managerial tools in theworld, there stands an important place for a good objective tracking model

Among those tools, the Logical Framework Approach is known to be amongthe most frequently used tool, especially in small and medium developmentalprojects It is a set of interlocking concepts which must be used together in adynamic fashion to develop a well-designed, objectively-described and evaluableproject

In 1969, the U.S Agency for International Development commissioned PCIstaff to analyze its project evaluation system That analysis uncovered three basicproblems that were seriously hindering not only meaningful evaluation of projects,but also their implementation.

(1) Planning was too vague: Objectives were multiple and not clearly related

to project activities There was no clear picture of what the project would look like

if it were successful Thus, evaluators could not compare – in an objective manner –what was planned with what actually happened

(2) The management responsibility was unclear: Project managers were

committed to the fact that projects must be justified in terms of their ultimate

benefits (impact) yet were reluctant to be considered responsible for impact; there

were too many important factors outside their control They found it difficult toarticulate what they should be responsible for, and ended up not accepting anyresponsibility for results

(3) Evaluation was an adversary process: With the absence of clear targets

and frequent disagreements (even among project team members) as to just what theproject is about, evaluators ended up using their own judgment as to what theythought were “good things” and “bad things” The subsequent evaluation resultswould then frequently become a basis for further argument about what was good orbad, rather than resulting in constructive actions for project improvement

The Logical Framework Approach to project design and evaluation wasspecifically developed in response to the above problems According to the report ofPractical Concepts Incorporated in the USA (1980), it encourages collaborationfrom the outset and helps avoid adversary relationships in both project formulationand evaluation by:

(1) Fostering of clearly stated, explicit and measureable description of whatwill happen if the project is successful;

(2) Clarifying what a project manager should be responsible for accomplishingand why;

(3) Displaying the key elements of project design and their relationships toeach other in a way that facilitates project analysis;

(4) Changing the focus of evaluation from “who is to blame?” to “what is themost realistic plan for this project for the future based on the best evidence availablenow?” This approach makes the project manager a primary user of evaluationresults The Logical Framework requires clear objectives and then bases evaluation

on evidence

The Logical Framework is a way of organizing information and activities sothat a number of different points of view can be brought to bear at the same time,completing rather than opposing each other (Drucker P, 1972) To simplifyprograms, it is essential to first recognize that there are three basic levels of

responsibility: Inputs – the resources consumed and activities undertaken, Outputs –

the things managers are committed to produce These must be stated as results Incase of failure to produce those results, then the burden of proof is on the manager

to “show cause” as to why he or she failed, Purpose – the reason to produce the

outputs At this point, one adds to the three-level management hierarchy a fourth,

superior level, called Goal Goal is define as the higher-level objective immediately

above project purpose Goal thus relates our project aspirations to aspirations ofthose for whom cur activities have no intrinsic interest

Moreover, it is critical to establish an appropriate set of criteria for thedetermination of indicators According to Handbook for the logical frameworkapproach (The Norwegian Agency for Development Cooperation, 1999)., there arefour simple yet important criteria that should be fulfilled first-hand in the choosing

of indicators for an evaluation system based on the Logical Framework Approach

(1) Indicators measure what is important: The indicators must measure what is

important in the objective

(2) Indicators must be plausible: The indicators we select must be so closely

related to what we are trying to measure that we are confident our project was animportant factor in the observable results

(3) Indicators must be targeted: Indicators must be targeted in terms of quantity,

quality, and time (QQT) If any of these three are missing we cannot be entirelyobjective about whether we have been successful or not One must note that notevery indicator can include all three factors (QQT)

(4) Indicators are independent: Indicators that demonstrate the achievement of

an objective at one specific level cannot he used to demonstrate achievement at thenext higher level Although this appears to be one of the simplest concepts ofLogical Framework methodology, it is also one of the most common weaknesses inLogical Framework designs There is a common tendency to demonstrateachievement of a result by measuring the means used to achieve the result Forexample, it is frequently claimed that “school building constructed” and “teacherstrained” (outputs) demonstrate improved quality of education in the school(purpose)

At this point, the following part shall be contributed to the advantages aswell as the limitations of using Logical Framework Approach (abbreviated in thetable as LFA) Knowing these strengths and weaknesses of this model in practice,the according solution and utilization shall be explained further in the next part ofthis chapter

Table 1.2: Advantages and Limitations of Logical Framework Approach

Pros • It ensures that fundamental questions are asked and weaknesses are

analyzed, in order to provide decision makers with better and morerelevant information

• It guides systematic and logical analysis of the inter-related keyelements which constitute a well-designed project

• It improves planning by highlighting linkages between projectelements and external factors

• It provides a better basis for systematic monitoring and analysis ofthe effects of projects

• It facilitates common understanding and better communication

between decision makers, managers and other parties involved in theproject.

• Management and administration benefit from standardizedprocedures for collecting and assessing information

• The use of LFA and systematic monitoring ensures continuity ofapproach when original project staff are replaced

• As more institutions adopt the LFA concept it may facilitatecommunication between governments and donor agencies Widespreaduse of the

• LFA format makes it easier to undertake both sectional studies andcomparative studies in general

Source: Norwegian Agency for Development Cooperation (NORAD), 1999

It is important to note that this approach has many advantages in it as beingnot difficult to use and not requiring a degree in mathematics or the use ofcomputers On the other hand, it relies on the user’s experience with developmentprojects as well as a sense of what constitutes good management and intuition

One should not forgo the nature of this approach as well, because it does notprovide answers or make decisions; but it organizes information in such a way thatthe important questions can be asked, project weaknesses can be identified, and

decision-makers can make decisions based on their increased insight and knowledge(Practical Concepts Incorporated’s guide, 1979).

1.3 Steps to establish evaluation system for curriculum

Having effective and sophisticated tools for setting a good evaluationsystem is not enough A thorough strategy for establishing an evaluation systemfor curriculum development as well has been refined throughout the years bycountless educational institutions and educators around the world Among themstands one of the most notable works, which shows and explain comprehensivelyand cohesively on evaluation strategies, which is a well- developed strategybetween educational knowledge and sophisticated management tools (Worthen,1981)

Figure 1.1: Steps to establish evaluation system for curriculum

Source: Worthen, 1981

According to this theory, the following are strategies that successfuladministrators use in developing assessment and evaluation programs, includingthree major steps In this section, the most essential steps to establish anevaluation system for curriculum development shall be presented in order to give

a big picture of the actual researching process of this report

Firstly, setting goals and indicators is an important must do Theevaluation and assessment process must be associated closely with the missionstatement and objectives of the educational establishment, since that clarity in fact

Identifying target stakeholders involved in the process

Design the evaluation system

Design the evaluation system

Refining and improving the evaluation system

Refining and improving the evaluation system

allows schools to make priorities and decisions to afford maximum pursuit of itsown mission Indicators of curriculum integration for the purposes of evaluation,therefore, should be formed during the early planning stages of the program

Secondly, one should take due care of identifying target populations, which

is understood as the focus on targeting specifically external and internalpopulation groups External groups are included with parents and community,while other internal stakeholders whose presences represent the direct educationalactivities such as administrators, teachers, and students (Douglas E, 2000).Hence, the phrase of identifying all possible stakeholders helps focus specifically

on these target areas and how they relate to school and curriculum

Through this second step, one could start to design an evaluation system,considering two main matters First, the implementation should be left open tochanges, in another word, be flexible (Gardiner L, 1990) This represents itself incase of additional data become clear, they should, then, be integrated into a newdeveloped design Second, the final evaluation system should be reported in ways

that will facilitate needs of different stakeholders, which means it must be (1) easy

to comprehend (2) open for more initiatives given by the public and (3)

communicated widely and clearly

This process does not stop here, but continues to refine itself times totimes, so that people related to it are all able to measure several benefits ofeffective curriculum as well as improve on unresolved matters Usually, indicators

of success related directly student achievement as well as performance in schoolsstands as an important factor

However, it is predicted that future research will be based more on theimpact of technology or teaching and learning process quality, so that it couldgive a more complete picture (Ferrero J, 2006) One way or another, one thing isshared commonly between past and future method, is throughout the entireevaluation process, the focus for curriculum developers should be on how to bestcombining appropriate strategies with measurable results indicating positivecorrelations with teaching and learning

1.4 Criteria in setting the evaluation system

1.4.1 Twenty-four criteria for the development of curriculum

According to Jack J Wickert (1973), the evaluation system contains differentcore criteria for an accurate and effective curriculum development This set ofcriteria play an essential role within this research as they form the structure of the

evaluation system as well as the survey and questionnaire’s gist A final list of 24

was thus developed, the following criteria were used in the evaluation:

(1) There is a clear statement of the philosophy (mission) of the school

(2) Written information is available which clearly states the directions and/ orresponsibilities for individuals and committees appointed to do work in curricularareas

(3) The curriculum tasks to be done are understood by the respective groups andcommittees

(4) A planned strategy is in operation for curriculum planning which establishes

a sequence of steps or tasks as well as a timetable for curriculum planning and

evaluation covering all aspects from goals to means of implementation

(5) Administrative expedients for curriculum development are provided, such asreleased time, physical facilities, appropriate materials, clerical help, anddiscretionary funds

(6) School organization and instructional patterns and facilities are determined

by the curriculum rather than vice versa

(7) There is a central curriculum committee for the school district which consists

of professional representatives of every school level and which is responsible forcoordinating all curriculum development decisions

(8) One specific person on the central administrative staff is responsible forcoordinating curriculum and instructional planning and evaluation

(9) Individualism is encouraged For example: (a) the individual school isconsidered the basis operational and planning unit for curriculum improvement; thecentral office may develop curricula adapted to their particular community and

pupils (b) Individual teaching styles may also be encouraged within the framework

(16) Open communication, both oral and written, provides for reactions from allconcerned groups and individuals

(17) Provision is made for studying the learner’s needs as an integral phase in theimprovement of curriculum and instruction

(18) Experimentation and action research are employed as integral parts of thecurriculum improvement process

(19) An evaluation is carried on to determine the effectiveness of the curriculumplanning

(20) Effective status leadership is evident on all levels of curriculumdevelopment A leader is thought of as one who moves the group under hisresponsibility toward mutually agreed-upon goals

(21) Emergent leadership which helps to clarify goals is encouraged

(22) Provisions are made for professional growth of staff members through suchactivities as: attendance at local and national conferences, school visits, the study ofnew materials, and in service programs in developing skills in curriculumimprovement

(23) Curriculum improvement is viewed as a change in attitudes, perceptions,values, understandings and skills of the staff as well as a change in the educationalprogram

(24) Objectives for the learner are written in behavioral terms in all subject areas.Twenty-four criteria for the development of curriculum were identified Each

criterion was accepted on the basis that is was (a) crucial- it must stand alone and being decisive and critical to any list formulated; (b) functional- it must be understood and applicable by a user; (c) observable- it must be readily identifiable

as a philosophical or practical operant

In his research, Wickert developed an extensive survey of the literaturewhich was conducted in an attempt to formulate the 24 criteria for curriculumdevelopment A preliminary list of over 90 guidelines, suggestions and criteria forcurriculum change in schools was developed from ideas found in the works of notedwriters in the curriculum filed From this preliminary list a refinement was made,using the three standards described earlier namely cruciality, functionality andobservability

However, to apply this theory in Vietnamese context, one should carefullyexclude irrelevant or inapplicable indicators into assessment system This shall bementioned in details in the next section of this research

1.4.2 Curriculum Program Checklist

In the book Curriculum Development (Bilbao et al, 2008; Glatthorn, 4th ed,2015) a simple checklist to form a good curriculum has been established, and thenhas been used in a host of schools in the Western educational system (Glatthord, 4th

ed, 2015)

The most advantageous feature of this checklist is that it is useful toinstructive planners attempting to gauge the accomplishment of their classroomcurricular project The checklist for the curricular project above is easy tounderstand and gives a fast assessment of program components, which help figureout if key parts are display in the classroom program

Table 1.3: Curriculum program checklist

Checklis

t

Curriculum program components

Does a good curriculum have to provide evidence ofadministrative and school board support?

Does a good curriculum plan have to incorporate a missionstatement?

Does a good curriculum plan have to establish an advisorycommittee?

Does a good curriculum plan have to facilitate the involvement ofparents?

Does a good curriculum plan have to utilize student learneroutcomes as a measure?

Does a good curriculum plan have to have an evaluation tool thatprovides for the collection of qualitative data?

Source: Bilbao, Purita P., Lucido, Paz I., Iringan, Tomasa C., and Javier, Rodrigo B., 2008.

These two tools shall be used further in the research process in the form ofintegration into questionnaires and interviews with hope to form a comprehensiveunderstanding about Cambridge International Education program at Doan Thi DiemPrimary School

CHAPTER 2: METHODOLOGY

The purpose of this study is to examine the perceptions of Doan Thi DiemPrimary School’s CIE program’s stakeholders, including selected project managers,teachers, school board members, parents and students in terms of the quality andcondition, maintenance, and improvement and renovation of ongoing curriculumdevelopment

The four purposes of this chapter are to (1) describe the research methodology of this study, (2) explain the sample selection, (3) describe the procedure used in designing the instrument and collecting the data, and (4) provide

an explanation of the statistical procedures used to analyse the data

According to Leary (1995), there are distinct advantages in using aquestionnaire vs an interview methodology: questionnaires are less expensive andeasier to administer than personal interviews; they lend themselves to groupadministration; and, they allow confidentiality to be assured Robson (1993)indicates that mailed surveys are extremely efficient at providing information in arelatively brief time period at low cost to the researcher It is noted that this applies

in the case of adult surveyees only, not for the students whose age ranges from six

to ten years old With this group, a physical questionnaire was required