(LUẬN VĂN THẠC SĨ) Evaluation of Career Exploration project in College Success course through students’ change in attitudes and knowledge A case study

VIETNAM NATIONAL UNIVERSITY- HANOI UNIVERSITY OF LANGUAGES & INTERNATIONAL STUDIES FACULTY OF POST- GRADUATE NGUYỄN THỊ CHI EVALUATION OF CAREER EXPLORATION PROJECT IN COLLEGE SUCCESS

VIETNAM NATIONAL UNIVERSITY- HANOI UNIVERSITY OF LANGUAGES & INTERNATIONAL STUDIES

FACULTY OF POST- GRADUATE

NGUYỄN THỊ CHI

EVALUATION OF CAREER EXPLORATION PROJECT IN COLLEGE SUCCESS COURSE THROUGH STUDENTS’ CHANGE IN ATTITUDES AND KNOWLEDGE:

VIETNAM NATIONAL UNIVERSITY- HANOI UNIVERSITY OF LANGUAGES & INTERNATIONAL STUDIES

FACULTY OF POST- GRADUATE

NGUYỄN THỊ CHI

EVALUATION OF CAREER EXPLORATION PROJECT IN COLLEGE SUCCESS COURSE THROUGH STUDENTS’ CHANGE IN ATTITUDES AND KNOWLEDGE:

TABLE OF CONTENTS

CANDIDATE‟S STATEMENT ……… ii

ACKNOWLEDGEMENTS ………iii

ABSTRACT ………iv

TABLE OF CONTENTS ……….v

LIST OF TABLES & FIGURES ……… viii

PART A: INTRODUCTION……… 1

I Statement of the problem and rationale for the evaluation… ……… 1

II Scope of the study ……… ……….4

III Aims of the revaluation and research questions ……….5

IV Significance of the study………6

V Methods of the study ……….6

PART B: DEVELOPMENT ……… 8

CHAPTER 1: LITERATURE REVIEW ……….8

1.1 Career exploration and related terms … ………8

1.2 Importance of career exploration ……… … 10

1.3 Research on career exploration ……… ……… 11

1.3.1 Correlation studies ………11

1.3.2 Evaluation ……….15

1.3.2.1 Significance and framework of career program evaluation ……… 15

1.3.2.2 Types of career exploration and findings of specific evaluation ……… 19

CHAPTER 2: RESEARCH METHODOLOGY……….……… 32

2.1 The context………… ……… 32

2.2 Participants ……… ………33

2.3 Data collection methods and procedures… ……… 36

2.3.1 Questionnaire ……….36

2.3.2 Test ……… 37

2.3.3 Interview ………38

2.4 Data analysis ……… 39

CHAPTER 3: DATA ANALYSIS AND DISCUSSION ……… 41

3.1 Single-case analysis and discussion ……… 41

3.1.1 Case 1 (A) ……… ……….41

3.1.1.1 Background of the case ……… 41

3.1.1.2 Changes in attitudes ……… 44

3.1.1.3 Changes in knowledge ……… 48

3.1.2 Case 2 (L) ……… ……… 51

3.2.1.1 Background of the case ……….51

3.2.1.2 Changes in attitudes ……… 54

3.2.1.3 Changes in knowledge ……… 59

3.1.3 Case 3 (H) ……… 61

3.3.1.1 Background of the case ……… 61

3.3.1.2 Changes in attitudes ……… 64

3.3.1.3 Changes in knowledge ……… 69

3.2 Cross-case analysis and discussion ……… …71

PART C: CONCLUSION ……… ……… 78

1 Summary of the study ……… ……….………78

2 Contributions of the evaluation ……….……… 79

3 Limitations of the study ………80

4 Suggestions for further study………80

REFERENCES ……… 81 APPENDICES………I

Appendix 1A: Questionnaire for choosing cases (English version)……….I Appendix 1B: Questionnaire for choosing cases (Vietnamese version)… II Appendix 2A: Pre-Questionnaire (English version) …… ………III Appendix 2B: Pre-Questionnaire (Vietnamese version) ……… VII Appendix 3A: Post-Questionnaire (English version) ………XII Appendix 3B: Post-Questionnaire (Vietnamese version) … ………… XVII Appendix 4A: Pre-Interview scheme (English & Vietnamese version) XXII Appendix 4B: Post-Interview (English & Vietnamese version) …… XXIII Appendix 5A: Pre-Test (English version) …… ……… XXVI Appendix 5B: Pre-Test (Vietnamese version) ……… XXVI Appendix 6A: Post-Test (English version) …… ……… XXVIII Appendix 6B: Post-Test (Vietnamese version) ………XXIX

LIST OF TABLES & FIGURES

Figure 1: The general scope of educational evaluation ……….16

Figure 2: Process of career decision-making ……….20

Table 1: Framework of holistic program evaluation ……….16

Table 2: Results of reason survey ……… 35

Table 3: A‟s changes in attitudes ……… 44

Table 4: Development of A‟s attitudes from her perception ……… 47

Table 5: A‟s changes in knowledge……… ………… 49

Table 6: Development of A‟s knowledge from her perception ……… 51

Table 7: L‟s changes in attitudes ………54

Table 8: Development of L‟s attitudes from her perception ……… 59

Table 9: L‟s changes in knowledge ………59

Table 10: H‟s changes in attitudes ……….65

Table 11: Summary of changes of three cases ……… 72

PART A: INTRODUCTION

1 Statement of the problem and rationale for the study

Evaluation is a tool which helps systematically identify the „merit‟ or

„worth‟ of a program so as to reduce „uncertainty in decision‟ and „facilitate positive change‟ for advantages (Mertens, 1999, as cited in Mertens, 2005, p.48) In other words, program evaluation is really critical; however,

according to several scholars (Poel, 2009; Royse, Thyer, and Padgett, 2010) it has rarely been undertaken The reasons for avoiding it have not been

officially revealed by researchers, but they may include the concern that

conducting evaluations take time and attention away from the course content The situation may be the same with University of International Languages and Studies (ULIS), where a large number of courses are internally designed

by its staff, but just a few evaluations of their programs and projects have been undertaken As such, it is considerably beneficial to take evaluative research into consideration for the case of ULIS

„College Success: Taking Charge of Your Personal and Academic Growth‟ is a new compulsory course specifically designed for first-year

students at ULIS So far the course has run for three years During the course, students are expected to participate in a variety of practical tasks that promote

a number of necessary academic and soft skills (Study skills, 2009; Phung et

al., 2011) The content of the course was revised once in 2010 when the force recognized that the fundamental objectives of version 2009 were to provide students with opportunities to practise some vital skills such as note-taking, citing, and presentation skills while most of the tasks were like

task-reading exercises Specifically, students were required to read some texts and then answer some questions as a follow-up activity, which does not reflect the principle of learning by doing stated in the course description More

importantly, the original course just focused on the academic skills which are integrated in other courses For example, the Reading course does involve practising note-taking skills; the Writing course does involve practice on referencing and avoiding plagiarism The „tasks‟ designed for the original

version of College Success are somehow the same as the exercises in the two

courses mentioned above Version 2010 is similar to version 2009 in that it is for first-year students to develop a number of essential skills so that they can better understand themselves and their environment, but it bears remarkable differences The first difference lies in the objectives Besides training in academic skills; the new version creates opportunities for students‟ personal growth and provides them with career orientation The second difference lies

in the task process Students are not required to do the exercises but take action in making some specific products such as posters, talk shows, and job-related reports The practice of the needed academic skills is integrated into the process of making these products For instance, the note-taking skills are integrated into the task of creating a poster of a role model In so doing, students can practise and/ or revise one of the academic skills – note-taking (select what information to be displayed on their poster) At the same time, they have an opportunity to share their own stories about what they truly love and value in their life Whereby, they can be aware of various values in

human life and the needs of putting efforts into studying and working for great achievements Actually, it is impossible for students to acquire and master all the skills expected in this course; students need time to absorb and put them in practice for further improvement Thus, the fundamental aim of the course is to raise students‟ awareness of some necessary skills that they need for both academic purposes and personal growth As such, the new version is hoped to work better with the given aim Until 2011, the original

course title, Study Skills, has been changed into „College Success: Taking Charge of Your Personal and Academic Growth,‟ which better reflects the

nature of the new version of the course

Within the framework of the course, there are two projects which take

place at the same time during the second half of the course – Career

exploration and Dream jobs Students are divided into groups and asked to choose one of the two projects for their process While Dream jobs creates an

opportunity for students to better understand the job they have wished to do

in the future by expressing their dream to the others, Career exploration is a

tool to enrich students‟ knowledge of the various career options available to them and suitable to their major Further, the projects both help students be self-aware of their personality traits, work-related strengths and weaknesses, have a sense of cooperative work, and practise a number of academic skills, including information searching, processing, presentation, and teamwork

(Phung, et al., 2011a) It is assumed that the students who choose Dream jobs

are clear about the career path they want to follow in the future, whereas there

is a fact that many students, when applying for ULIS, still are not sure about their expected jobs These students just study as scheduled and do not seem to

know where to start with their career In this regard, Career exploration is a

true opportunity for them to kick off their start; therefore, the evaluator would like to investigate into its impacts on the students who choose this project

Moreover, Career exploration was implemented for the first time in 2010 and

has informally received conflicting oral comments from students Some

claimed that the project was interesting, but others complained that they were burdened with the task and had to spend such a good deal of time for a mark

As one of the project designers as well as a teacher involving in its process,

the evaluator is encouraged to conduct a formal in-depth evaluation of the project

2 Scope of the study

According to Scriven (1991), the evaluation is implemented on the basis of „relevant standards of merit, worth, or value‟ (p.47, as cited in

Mertens, 2005) In other words, the evaluation process normally involves the comparison between the outcomes with the intended objectives Royse et al (2010) share the same opinion when they categorize evaluation as one type of research that helps define whether the needs are met or the participants‟ goals

are attained In the case of Career exploration project, students‟ achievement

is compared to the planned goals in respect of attitudes, knowledge, and skills The first two goals are those expected to be acquired by students

during their involvement without any lectures, while the third involves the teaching and practice in class By the nature of the project, its primary aim is

to promote students‟ changes in attitudes and knowledge in relation to their possible career path As such, this study is just to evaluate students‟ changes

in attitudes and knowledge while the evaluation of skills attainment and/ or enhancement is excluded As regards attitudes, students are expected to

develop: self-awareness of personality traits, work-related strengths and weaknesses, activeness and criticality in utilizing sources, and cooperative sense in teamwork In terms of knowledge, students are expected to have a grasp of the important features of relevant jobs and their possible career paths (Phung et al 2011, p.23) Any other emerging attitude and knowledge

discovered during the course of the study will also be included

As a matter of fact, it is difficult for students to become different in their behaviors right after the project, so achievement or „change‟ in this study refers to the students‟ awareness of the expected attitudes and

knowledge that are assumed by the course designers to be needed for

students‟ learning at college and for their future career Job performance or personal adjustment was excluded from this scope Moreover, in this

evaluation, „change‟ is divided into two levels: „acquire‟ and „develop‟

„Acquire‟ refers to the attitudes and knowledge that students became aware of

in the course of doing the project In other words, they had never learned the

„acquired‟ attitudes and knowledge before the project Meanwhile, „develop‟ refers to the attitudes and knowledge that students had learned from

elsewhere other than from the project, but thanks to the project their

awareness of these attitudes and knowledge was heightened

Another important matter is how the project and the related evaluation can be located in the field of English Teaching Methodology In answer to this question, the approach on which the designers based should be

mentioned: competence-based approach In other words, the designers would like to build up learners‟ competences as the outcome of the course or the project itself Competences can be understood as the application of skills, knowledge, strategies, attitudes, etc in doing some tasks to a standard Tasks

in language teaching require learners to pay primary attention to the meaning and make use of their linguistic sources to process and understand language Tasks designed within the project require students to process the provided guidelines which were written in English in order to make a certain product

In this respect, language (specifically English) serves as a means to help

learners achieve some expected outcomes With regard to this study, it did not evaluate the achieved competences; neither did it assess students‟ completion

of specific tasks What was focused on were the attitudes and knowledge which are considered part of language learning process Having adequate attitudes and knowledge benefits students in bettering their learning; being

aware of students‟ attitudes and knowledge benefits teachers in helping them

to the fullest Putting differently, the project itself as well as the evaluation was considered practical in English Teaching Methodology

3 Aims of the study and research questions

The first aim of the study is to find out whether students achieve the

goals in terms of attitudes and knowledge in Career exploration project If

the achievement exists, the specific attitudes and pieces of knowledge are also expected to be discovered Also, attempts were made to see whether students could obtain any other attitudes and pieces of knowledge out of the expected ones

In brief, the study seeks answers to the following questions:

1 In terms of attitudes, after completing the project of career

exploration,

a was there any change in students‟ attitudes?

b what attitudes did the students acquire?

c what attitudes did the students develop? To what extent were those attitudes developed as perceived by the students?

2 In terms of knowledge, after completing the project of career

exploration,

a was there any change in students‟ knowledge?

b what knowledge did the students acquire?

c what knowledge did the students develop? To what extent was that knowledge developed as perceived by the students?

4 Significance of the study

The study is a significant endeavor in investigating the impacts of the project on students‟ attitudes and knowledge in respect of their career path Such an outcome determines whether or not the project is successful and/ or indicates any improvement Additionally, the project is not of rigid theories but the exploration of the real jobs, so it is important for teachers in-charge to provide proper guidance and support with real-life experience so as to engage students in the tasks and foster their awareness of the tasks‟ values (Phung, et al., 2011, p.i) Therefore, understanding of the impacts may help teachers in-charge to look back at their decision on guidance and support for the further improvement of the project

5 Methods of the study

Case study was employed in this study which was to discover

individual students‟ overall attainment in terms of attitudes and knowledge after they accomplished the project It can be argued that in order to evaluate

a project, an overview of as many participants‟ achievement should be drawn

as possible However, the goal of the project is to help students partly develop their personal growth (Phung et al., 2011) and individuals are different;

accordingly, the fact that a person can, to a certain extent, grow thanks to the project also proves its certain success As a result, case study is considered adequate for the evaluation

Three cases for the evaluation were selected among the first-year

students who attended College Success Course in the second semester of the academic year 2011 – 2012 of ULIS In order to investigate students‟ change

in attitudes and knowledge, three instruments namely questionnaire, test and interview were used and delivered to students both before and after the

project

PART B - DEVELOPMENT

CHAPTER 1 – LITERATURE REVIEW

1.1 Career exploration and related terms

The career guidance, according to Hughes and Karp (2004), dates back

to the 20th century when vocational education was strongly supported and promoted These scholars also acknowledge that Frank Parson, father of the vocational guidance movement, was the first person to follow scientific and systematic model of vocational assessment:

In the wise choice of a vocation, there are three broad factors:

(1) a clear understanding of yourself, your aptitudes, abilities,

interests, ambitions, resources, limitations, and their causes; (2)

a knowledge of the requirements and conditions of success,

advantages and disadvantages, compensation, opportunities, and

prospects in different lines of works; (3) true reasoning on the

relations of these two groups of facts (Philips & Pazienza, 1998,

p.2, as cited in Hughes & Karp, 2004)

Basing on this model, career guidance was first designed as the

assessment of individual differences by „experts who would determine the person‟s best occupational direction (Philips & Pazienza, 1998, as cited in Hughes & Karp, 2004) Not until 1920s and 1930s did the role of counseling start, and so far it has been a trend in career orientation The Comprehensive Guidance Program was developed in the early 1970s and a lot of reforms have been implemented since then

Career orientation is a general term that refers to any types of activities

to help students have better understanding of themselves and the world of work and/ or prepare them with essential occupational knowledge and skills This term and the others, such as career development, career education, career guidance, and career counseling, can be used interchangeably (Maddy-

Bernstein, 2000, cited in Hughes & Karp, 2004) although they may be used

differently by various programmers For example, career guidance is often understood as „an umbrella term‟ (Hughes & Karp, 2004) encompassing many services for students‟ personal and professional growth while career counseling is only one portion of career guidance, focusing on the interaction between trained professionals and students so as to help them develop their career path(s) Accordingly, career orientation can be divided into two

categories: one is career counseling or advising and the other is interventions The former includes the activities that provide direction and planning skills for students, and the latter refers to career courses designed to promote

student knowledge and skills relevant to the world of work

One emerging term in the late 1970s from the theory of career choice and counseling is career exploration which is considered as a type of

„information-seeking behavior‟ or as a „career problem-solving behavior‟ Along with its development, career exploration has been referred to as „a complex psychological process, which sustains the search of information, as well as hypothesis testing about self and environment, in order to attain career goals‟ (Hall, 1998 cited in Zikic & Hall, 2009; Taveira & Moreno, 2003) Putting differently, career exploration is considered to be a critical step in the career decision-making process

Many other terms which can be used in the field of career education and guidance as suggested by Athanasou (2007) include: “group counseling, computer-assisted guidance, one-to-one counseling services, career advisory services, telephone career information and advice, employment services, career information, recruitment programs, supported employment programs, human resources development programs, outplacement services,

rehabilitation programs, vocational assessment services, and vocational

guidance” (p.3) However, whatever term is used and although the

implementation of specific programs associated with each term is different, the overall objective of these programs is to involve individuals in the

exploration of the world of work as well as offer them a chance to identify their own potentials in certain career paths

1.2 The importance of career exploration

There is no wonder that career exploration is of particular importance

in eliminating initial confusion about what a person should prepare in terms

of academic and vocational concerns and what occupations he/ she can follow after graduation Once things are made clear at the very beginning, he/ she can better plan towards the goal According to Porfeli & Skorikov (2009), the selection of a suitable career is presumed to depend heavily on the nature of the orientation program and the extent to which a person engage in it (p.47) Schneider & Stevenson (1999) also assert that career development is „vitally important for today‟s youths‟ (Hughes & Karp, p.2, 2004) They argue that the youths now have been more motivated and ambitious for their education and profession, but many of them have been directionless; hence the demand for purposeful guidance has increased Occupational Outlook Quarterly

(1999) shares the view that students often wonder why they need to study the subjects that seem unrelated to their career goals and that they often feel confused when it comes to making a decision on their career path Although career exploration takes time, many believe that it brings positive impacts on the determination of students‟ future plans Zikic & Hall (2009) posit that during the process of career exploration the individual is likely to become more self-aware of his or her likes and dislikes and possibly figure out the focus of his or her exploration Accordingly, the person‟s adaptability to the world of work can be achieved and/ or increase (p.186) These scholars also discuss the outcomes of career orientation during a person‟s employment, at

which time it helps him or her to discover more occupational opportunities and become more employable with the increase in adaptability (Fugate,

Kinichi & Ashforth, 2004; Savickas, 2007)

1.3 Research on career exploration

Two most popular trends of research on career exploration, up till now, have been correlation studies and impact evaluation Within the scope of this study, only a small number of examples of correlation studies concerning career exploration were presented along with significant findings, whereas most of the space was dedicated to the summary of career program

evaluation

1.3.1 Correlation studies

One example of correlation studies is „Five-factor model of personality and career exploration‟ by Reed, Bruch, Haase (2004) The researchers

examine whether there is any relation between the dimensions of the

five-factor model (FFM) of personality namely neuroticism, extraversion,

openness to experience, agreeableness, conscientiousness and specific career exploration variables which include self-exploration, career information seeking, stress regarding career exploration, career search self-efficacy

(p.224-225) The correlation study was conducted with the sample of 204 undergraduates at the average age of 20, including 2% freshmen, 46%

sophomores, 44% junior, and 8& seniors The participants were required to complete a questionnaire booklet comprising a form to assess the five

personality factors and a series of items to assess the relation between those factors and career exploration variables Results from the correlation analysis revealed the association between conscientiousness/ extraversion/ low

neuroticism and career search self-efficacy/ career information seeking The results also proved that openness was associated with a lack of career

information seeking and that neuroticism/ openness was related to

self-exploration In this regard, counseling experts are advised to consider using the FFM when engaging in career counseling because it can be “a useful tool for teaching clients how their personality style might facilitate or inhibit their career development and assessment” (Costa, McCrae & Key, 1995, as cited in Reed, Bruch, Haase, 2004, p.235) This tool can also help counselor in

optimizing the clients‟ career self-efficacy by facilitating good conditions for beneficial factors or providing more detailed plans and guidance to minimize the impacts of detrimental factors However, the investigators do not intend to generalize such relationship between personality factors and career

exploration variables to other population and suggest that additional research

is needed (p.235-236)

Nauta‟s (2007) also investigates into the relationship between the five big personality dimensions and college students‟ career exploration

However, the investigation of these two sets of variables was not

simultaneously; instead the investigator scored the five big personality

dimensions of the participants and used the results to predict their career exploration behaviors (self-exploration and environment exploration) After approximately 18 months, the participants were re-contacted and do a career-exploration self-report which was put into comparison with the previous prediction The so doing revealed that none of the predictors was associated with subsequent environmental exploration and that the findings related to the association between personality traits and self-exploration were not totally consistent with those in Reed et al (2004) Specifically, openness was still seen to be positively associated with self-exploration while extraversion was negatively associated with such exploration This raises a wonder whether extraversion just has short-term benefits in promoting exploration In

addition, the relationship between career interests and self-efficacy (which, as proposed by Holland‟s (1997), include realistic, investigative, artistic, social, enterprising, and conventional) and career exploration behaviors was

examined with the same procedures The results showed that whereas

realistic, artistic, and conventional interests and artistic self-efficacy had positive association with self-exploration, investigating and enterprising interests were negatively associated with such exploration „Recognizing such tendencies early in students‟ careers would allow for interventions to be used

to increase the likelihood that they will engage in self-exploration prior to committing to a college major or career‟ (Nauta, 2007, p.176) Moreover, understanding of the mentioned tendencies, career counselors and/ or support staff could design suitable activities to facilitate self-exploration among

students

Another example of correlation studies concerning these issues is

„Personality traits, vocational interests, and career exploration: a

cross-cultural comparison between American and Hong Kong students” by Fan, Cheung, Leong, and Cheung (2012) This study also discusses the

relationship between the personality predictors and career exploration, but it was put in the context of two different cultures (Western and Eastern

counterparts) For such a feature, Fan at el (2012) did not employ the factor model to assess personalities; instead they based on the Cross-cultural (Chinese) Personality Assessment Inventory-2 (CPAI-2), which consists of four factors namely Social Potency/ Expansiveness, Dependability,

five-Accommodation, and Interpersonal Relatedness To a certain extent, these four factors are similar to five dimensions of the five-factor model; however, the investigators would like to use a tool that could cover indigenous

personality dimensions of Eastern culture (Hong Kong students) In

particular, „the IR (Interpersonal Relatedness) factor assesses the aspect of formalistic relationship orientation in collective societies The coverage of IR

is broader than the meaning expressed by Agreeableness in the Big Five and includes the qualities of humanity and social propriety in Confucian

philosophy.‟ (p.109) As regards vocational interests, the scholars did not merely examine the relation between different types and career exploration behaviors as Nauta had done, but hypothesized that vocational interests

mediated the association between the personality and career exploration Their thorough analysis revealed that the CPAI-2 indigenous personality dimensions derived in the Chinese cultural context predicted career

exploration of Hong Kong students only Significant cultural differences were also found in the contributions of the other two universal dimensions,

Accommodation and Dependability The Accommodation personality factor was more important to career exploration in Hong Kong context whereas Dependability was more important in American context Furthermore, there existed certain mediation Enterprise interest type mediated the association between Social Potency and career exploration in Hong Kong sample;

Artistic interest type was the mediator in the American sample The findings

of this investigation raise significant awareness of cultural differences in career guidance and the possible mediation of vocational interests in the

contribution of personality traits to career exploration, possibly leading to better design of counseling plans and/ or career courses

1.3.2 Evaluation

1.3.2.1 Significance and framework of career program evaluation

Evaluation of career program outcomes is crucial as it can help to

better the effects of wider-scale applications as well as limit the unexpected impacts on the participants According to Reese and Miller (2010, p.107), the

use of outcome data can serve as an important catalyst for improving career interventions So as to prove their argument, the scholars conducted a two-

year study to assess whether modifications made to the course using the

previous year‟s outcome data as a baseline resulted in subsequent

improvements The study used a pre-post group design that compared

students in a career course to a quasi-control group In both years of making modification and examine the related outcomes in comparison with those of the original course, the scholars found that there was increase in students‟

career decision-making self-efficacy Although the number of participants

was small (35 students enrolled in the career course and 38 students were of quasi-control group), the consistency of the results across both years appeared

to minimize such limitation In a broader sense, outcome evaluation of a

career course can bridge the gap between research and practitioners in terms

of providing career guidance In addition, Athanasou (2007) suggests that

evaluation can help to recognize unintended effects, for example social

enjoyment, group cohesion or work escape (p.3), whereby certain

modification can be decided The scholar summarizes the general scope of

program evaluation as follows:

Figure 1: The general scope of educational evaluation (Athanasou, 2000, as cited in Athanasou 2007, p.4)

Anathasou also suggests a model as a holistic evaluation framework for career education and guidance, including six steps: ethics, coverage, cost,

objective(s), effects, and stakeholders The detailed description of those steps can be seen in the following table:

Table 1: Framework of holistic program evaluation (p.29)

The above framework, from my perspective, is not only useful for

those who intend to have a holistic look at a program, but it is also useful in determining evaluation design for each dimension As regards the fourth step, Kirkpatrick (1996) provides the hierarchy of evaluation, starting with

reaction, learning, behavior and moving up to results:

- Reaction – program participants‟ estimates of satisfaction

provide an immediate level of evaluation

- Learning – the extent of learning (i.e., skill acquisition,

attitude change) that had been achieved is also of interest

- Behavior – the extent to which learning has generalized to

the work situation or there has been a transfer of skills is

assessed at this level through follow-up after a program

- Results – the wider impact of a program in the community or

the organization is observed at this level of the hierarchy

(p.25-26) Although there are documented criticism of the validity of the

hierarchy and the relationship between the levels (Anathasou, 2007, p.27),

this is still considered a source of reference when researchers decide the

scope and the design of their evaluation In particular, the evaluation of

Career Exploration project in ULIS did not aim at whether they transferred

the skills learnt from the project; hence field observation was not really

necessary for the evaluation It was also not an evaluation of the wider impact

in the community, so the evaluation could be limited to certain goals instead

of looking at each and every aspect of the project

Regarding the effectiveness of career interventions, Brown & Krane (2000) did not discuss how to determine the effect as Athanasou (2007), but suggested what should be included as important predictors According to them, in order to evaluate the impact of career courses, three out of the

following five components should be assessed: whether the courses (a) allow clients to clarify career and life goals in writing; (b) provide clients with individualized interpretations and feedback; (c) provide current information

on the risks and rewards of selected occupations and career fields; (d) include study of models and mentors who demonstrate effective career behavior; and (e) assist in developing support network for pursuing career aspirations

(Folsom and Reardon, 2003, p.426)

Another scheme to evaluate career interventions is based on outputs and outcomes (Burck, 1978; Burck & Peterson, 1975; Peterson & Bruck, 1982; Peterson, Sampson & Reardon, 1991) In this scheme, outputs refer to the skills, knowledge and attitudes acquired by participants as the result of an intervention whereas outcomes refer to the effects of course satisfaction, deciding on a major and timely graduation from college (Folsom & Reardon,

Career Ideas for Kids Who Like Computer by Diane Lindsey Reeves and

Peter Kent In Vietnam, a series of books by Kim Dong Publisher introducing the overview of certain types of jobs have also been available Career-related

work books such as Career Plan Project Workbook by McGraw-Hill

Companies have been delivered In the following part, a review of

effectiveness of three most popular forms of career exploration, based career exploration and career courses, will be provided

Internet-1.3.2.2.1 Internet-based career exploration

Up till now, a large number of universities and/ or institutions in other countries have designed useful online tools and offline programs to help students become aware of their own strengths and weaknesses, get to know clusters of available jobs, as well as figure out whether or not their interests and aptitudes are suitable with certain careers One example of online career decision-making tools can be found on the website

http://acrn.ovae.org/decision.htm by America‟s Career Resource Network This tool serves to help teachers, counselors, parents assist their children as well as help students themselves in identifying an appropriate career direction and selecting or developing an education plan that will provide the

knowledge and skills needed to succeed within their chosen career field The process of career decision making goes through six steps as follows:

Figure 2: Process of career decision-making (http://acrn.ovae.org/decision.htm)

As can be seen from figure 2, when users roll the cursor over each step, they can find the detailed description beginning with the goal(s) that are expected

to be attained and then several links to necessary reading texts or exploration

activities For example, the first step, engaging, aims at identifying the

relation between career choice and educational achievement and its importance In order to reach this goal, users are provided with the readings

of education challenge, economic challenge, and career development After

going through this first step, users are motivated to investigate their interests,

abilities, needs, and occupational choices (understanding), to explore educational requirements and look at several occupations (exploring), to

create a personal career profile based on what they have learned about

themselves and potential occupational options (evaluating), to make

educational plans for each occupation selected (acting), and to review the

decision so as to see how comfortable they are with the choices and whether they are ready to proceed with implementation or need to go back to an

earlier step in the decision-making process (reflecting) Associated with each

of these five steps are a guidance reading text and a number of questions to help users in their exploration

Another useful website that provides guidance for students to find out the possible paths themselves is

http://www.centralia.edu/students/counseling/careers.html by Centralia

College, the oldest community in the state of Washington The process of career direction and education plan mentioned in these two websites is also applied in various career courses implemented by universities, such as Ferris State University, Johnson & Wales University, and Utrecht University for both high school and college students

According to Boyce & Rainie (2002), Kommers & Rainie (2002), individuals are increasingly using computers for the purpose of career

exploration and planning They estimated that one of every five Americans has searched online for occupational information and that more than four million users do so every day (cited in Gore Jr, Bobek, Robbins, and Shayne,

2006, p.422) Robinson, Meyer, Prince, McLean, and Low (2000) agree that the Internet provides good access to „a vast realm of career and educational information‟, creating a space for self-directed career exploration; on the other hand, they wonder whether students use the source in a meaningful way (p.38) The wonder drives the scholars to the consideration into the case of Career Exploration Links, which is expected to address the needs of the wide range of college students (p.47) The provision of detailed description of the model helps other scholars and instructors to be well aware of the effort to

engage students in active interaction with the site The home page involves four choices which link to the second page; along with the choices is a short introductory text which assists students in recognizing their needs at a time:

(Robinson et al., 2000, p.47) With this model, students can go through the site assistance step by step based on their primary interests; in so doing, they are able to picture a comprehensive path to their undergraduate majors or career choices The alternative is to access the database of undergraduate majors, graduate

program, and occupational information links directly The keyword search is also established, allowing users to enter a brief search statement and retrieve

a display of occupational headings and their associated links (p.48-52) In this regard, it can be said that the model provides students with alternatives to career-related assistance; on the other hand, it poses a challenge in ensuring the proper utilization of students As admitted by Robinson et al (2000), the big disadvantage of models like Career Exploration Links is „the loss of a professional counselor or instructor to insure that the assessment and

occupational information is not presented or interpreted in ways that are

harmful to the career development of the individual‟ (p.52)

This disadvantage is also recognized by Gore Jr, Bobek, Robbins, and Shayne (2006) after they investigated a typology of users of a Web-delivered computer career guidance application among 87,293 high school and college

students residing in the United States during an eight-month period in

2002-2003 After three times of investigation, Gore Jr et al classified the students into three clusters, including General Browsers, Focused Users, and In-depth Users, and the first cluster accounted for high percentage The General

Browser cluster typically spent fewer than 30 minutes in the system and

might return for one additional session The Focused User cluster typically spent between 30 minutes and one hour in the system; moreover, the focused their exploration in specific areas as self-assessment and occupations The In-depth Users were those who spent more than one hour in the system and recorded multiple visits to self-assessment, occupation, major, and school sections of the system (p.422-428) It can be said that the extent to which students exploit the system without any interventions varies In this regard, the number of General Browsers could be reduced and that they might be more fully engaged in the system if they had received an adequate

orientation Gore Jr et al (2006) suggested that teachers and counselors be more active in preparing students for using the system, monitoring their use, and engaging them in processing the results of their exploration Many other leading scholars (Reile & Harris-Bowlsbey, 2000; Sampson & Lumsden, 2000) also posit he combination of using the Web-delivered computer career guidance application with counseling relationship for better outcomes (cited

in Gore Jr et al., 2006, p.335)

According to Gati and Peretz (2011), computerized career inventions cannot replace face-to-face human counseling Human are complex while computerized systems are just useful for providing specific interventions for a limited set of problems (p.270) Nonetheless, it is undeniable that online career tools are, to a certain extent, helpful in providing sources of majors and career information as well as self-assessment inventories, all of which

facilitate individuals‟ exploration As a result, there had merged a current trend which regards online career tools and career counseling complementary (Vocopher, Glavin & Savickas, 2010, as cited in Gati & Peretz, 2011, p.268)

In this case, career experts can support online tools by providing in-depth analysis of the results (either a printed copy or online feedback) Expert

judgment and recommendation can also be employed for more complicated problems or aspects Last but not least, professionals can help students more prepared before they start the exploration, which is highly likely lead to

optimum efficiency of the whole process (p.268-270)

Unlike the situation in other countries, online career decision-making tools and career courses have still been limited in Vietnam A large number of articles on www.dantri.com.vn and www.vietbao.vn since 2006 have

criticized that career orientation or career mentoring has not been paid

enough attention by the authorities Although there have been several

workshops for high school and university students to direct them towards the right career paths, the activities have been limited to theory and/ or personal counseling An online source of career information and counseling are

available on the website http://huongnghiep.vn/Default.htm, but guidance on the self-development process is limited

1.3.2.2.2 Career courses and projects

Career development courses have been used in colleges and

universities in the United States to assist students with educational and career planning since 1911 when several hours of instruction on vocational guidance was integrated in freshmen orientation courses (Maverick, 1926, as cited in Folsom & Reardon, 2003, p.421) A great number of career courses have continued to be designed and practiced United States in many years later, which has been proved by series of surveys of career course prevalence

(Hoppock, 1932; 1956; 1961; Haney & Howland, 1978; Reardon, Zunker, Dyal, 1979; Collins, 1998; to name but a few as cited in Folsom & Reardon,

2003, p.422-424) The content of career courses in the decades before 1970s was not revealed by the researchers, but afterwards the main content of the career development courses have tended to cover three major areas: (a) career choice factors, (b) career information, and (c) job-seeking techniques (Devlin,

1974, as cited in Folsom & Reardon, 2003, p.423) Then, Ripley (1975)

suggested that a combination of a restricted job market and a growing desire within higher education for contribution to the holistic development of

students was the primary reason for the remarkable development of career courses in the era (Folsom & Reardon, 2003, p.423) This trend has been followed by Britain, then many nations in Europe (Watts, Dartois & Plant, 1988) as well as several others in Asia such as Hong Kong, Japan, the

Philippines, Taiwan, and South Korea – (Organization for Economic

Cooperation & Development, 1996) with the assumption that career

education has the potential to make education at all levels more “career

relevant” (Herr, 1987, as cited in Peng, 1999, p 276)

In response to the prevalence of career courses, great effort has been made to learn about the impacts of these courses on both students‟ outputs (skills, knowledge and attitudes acquired by participants as the result of an intervention) and outcomes (more positive career-planning thoughts,

increased career decidedness, and greater career maturity) (Folsom &

(1999) However, two of those studies showed mixed results Weisman

(1988) used Career Decision Scale (Osipow et al,, 1976) and the Career

Decision-Making Self-Efficacy Scale (Taylor & Betz, 1983) to examine the impact of a career course on self efficacy and career decidedness The results indicated no significant changes in the output measure Meanwhile, Quinn and Lewis (1989) incorporated such career instruction into an existing upper division business personnel and organization behavior course, and they found career certainty increased for the course treatment group in comparison to the control group (Folsom and Reardon 2003, p.437) Although, neither of the studies mentioned the course content and exact level of students, except for the fact that the level of participants in the course introduced by Quinn and Lewis (1989) was higher than those in the course reported by Weisman

(1998), the mixed results raise a question about the factors that work behind the influence of the course

Another output interest of many researchers was in students‟ attitudes towards career planning Remarkably, Folsom and Reardon (2003) found 13 studies of this concern in the period of 1976 and 2001 None of studies (i.e., Ware, 1981; Stonewater & Daniels, 1983; Kern, 1995) prove negative effect

of career course on this dimension However, there is a likelihood of a

controversial debate between the effect of career counseling and career course

on career planning Whereas Davis and Horne (1986) found no significant different between the group counseling and career course treatments, the early studies (Super, Thompson, Lindeman, Jordaan, & Myers, 1981;

Babcock & Kaufman, 1976) indicated that the career development course was more effective than counseling

As regards the outcomes of career courses, the scholars mainly focused

on students‟ course satisfaction and the rate of students who stayed at schools

after their participation in the course (Touchton et al., 1977; Hepperner

Krause, 1979; Reardon & Rogan, 1981; Bechtol, 1978; Goodson, 1982;

Folsom, 2000) The matter of retention may be of special concern to many postsecondary institutions in United States because it bears an impact of

student graduation rates Noel, Levitz, and Saluri (1985) estimated that the percentage of college students who dropped out of school for their

undecidedness could be up to 75% By means of sound analysis, Folsom and Reardon (2003) found that 13 out of 15 studies reporting positive gain in measured outcome variables and 2 studies reporting no changes in outcome variable These findings are similar to the studies summarized earlier about output variables (Folsom and Reardon, 2003, p.444-445) Again, a question about the factors that worked behind these procedures was raised In order to have thorough understanding of the impact, the context and content of the course, counseling as well as the studies are needed to be grasped Even, further correlation studies have a role to play in this case According to Zikic and Hall (2009), there are some cases in which career exploration fails to achieve the desired outcomes Firstly, it is when the individuals are too

indecisive and/ or dependent on the external factors while they themselves do not make enough effort to explore what they really want to Another barrier to the success of career exploration can be the lack of opportunities and/ or resources In particular, there exist few sources of relevant information

relating the chosen career or students can hardly find opportunities to learn from real-life experience These unfavorable conditions more or less inhibit the individuals from holding positive attitude and behavior toward the

exploration (p.182-187)

With the same endeavor to understand the effects of career courses,

Peng did research on The impact of career education courses on career belief

and career decision making among business college students in Taiwan in

1999 and Effectiveness of two different career education courses on career decidedness for college freshmen in 2001 The former was conducted by

comparing the business college students‟ career beliefs and career decision making based on their participation in one-semester career education course with those who did not enroll in the course The results supported the

literature that career courses had positive effect on students‟ career decision making, but showed no changes in career beliefs There was a possibility that

a semester long career course did not provide sufficient time to change

students‟ beliefs in careers (Peng, 1999, p 278,287) The latter compared the influence of two career courses One was a cognitive restructuring

intervention which aimed to strengthen rational attitudes related to career decision making In this approach, the instructor served as a discussion leader and facilitator rather than lecturer as in the other course, a career skills

training intervention, which focused on teaching the factors of students‟ career decision making (personality, interest, aptitude, and value) Two

similar important requirements of these two courses were that students had to find a mentor for experience learning and that they themselves had to

construct a plan based on the content of the courses Participants of the

research were 164 freshmen from Taipei College of Business in Taiwan; these students were divided into three groups: 24 students in experimental group 1; 44 in experimental group 2 and the rest in control group (p.32-33) Like the study in 1999, this study also supported the literature that career education courses have a positive impact on career decision making

Moreover, the equal results were found in both approaches to career courses Remarkably, the mentoring activity did have great influence on students‟ understanding of profession as well as the road to the achievements (p.40)

As stated by Ware (1981), many of career courses at his time were designed for freshmen and sophomores who had decided on their majors and the results indicated increases in career decision making, in knowledge of self-awareness and of the world of work Therefore, he would like to carry out a study to investigate the effectiveness of courses planning for junior and senior students of psychology majors (p.67) The researcher employed several form to examine different dimensions of course effectiveness: the first form was to investigate students‟ degree of self knowledge, job opportunity,

resume writing, and interview skills; the second for students‟ specific

information response; the third for career decision making; and the last for career maturity After the procedure of pre-post test, he found that the

increase in students‟ self knowledge, job opportunities and skills as well as in career maturity was consistent with the course objectives The course also encouraged positive thoughts and actions toward acquire career-related

information However, there was no significant change in career decision making (p.70) Although the study proved consistent findings with the

literature, it might still fail to provide in-depth understanding of changes in investigated dimensions

As proved by the literature that college students often did not have accurate knowledge of careers in their discipline (Warner & Bradley, 1991; Ellis & Cantrell, 1994; and Nauta, 2000, as cited in Nauta, 2003, p.268) and that career research and presentations could boost students‟ understanding of career information (Ware, 1988; Nauta; 2000), Nauta (2002) conducted a career research project for undergraduate psychology She believed that

career research project was useful because it required students to be active in seeking source materials; thereby they could learn from the process of

investigation (p.290) Her project involved several steps: first, students were

required to search information on the career they would like to pursue or desire to know about and write an individual report; next, the oral

presentations were delivered to classmates (students could join a group if they searched the same occupation) She did not do any formal evaluation of the project, but students‟ feedback at the end of the project showed positive

effects Some of her students admitted that hearing others‟ oral presentations was useful but it was hard for them to absorb so much information in a short period of time and suggested that they should be replaced by poster

presentations Also based on the students‟ feedback, Nauta recommended the career research project be employed prior to senior students to support

students in evaluating their academic majors and careers at the very early stage in their education Thanks to the early assessment, students could

maximize their preparation for the work in the future

In conclusion, after the discussion on different terms related to career exploration and the importance of such an intervention, a wide range of

studies related to the issue was reviewed It could be observed from the

review that one of concerns among a good number of scholars was about the relationship between personality traits, career interests and career exploration, and their investigations prove a certain association between these set of

variables Another concern of scholars was about the outputs and outcomes of career exploration programs, especially the impacts of the interventions on learners‟ attitudes towards the world of work and their decision-making

efficacy Moreover, a common framework of program evaluation was

provided All in all, the review serves as the background; thereby a

framework for the current study was established and discussion on its

findings could be offered

principle of learning by doing It means that students do not have to do any

exercises but complete a number of tasks for some products, and their

awareness of important skills are raised during the process of implementing the task The course is divided into 12 lessons with six themes which include

human values and self-identification, student-related administrative

procedures and college study requirements at ULIS, extra-curriculum

activities, problem solving in respect of personal relationship, study and finance, career orientation, and presentation skills The lessons on human values and self-identification help students to be aware of their own self and

the other selves so that they can better themselves in a proper direction These lessons also help them to be aware of the values of effort for success The

lessons on student-related administrative procedures and college study

requirements and extra-curriculum activities provide students with some

basic knowledge of their study environment so that they can prepare well to fulfill the requirements as well as arrange the plan for more non-academic

enjoyment in their student life The series of lessons on problem solving

creates a platform for students to express themselves and share their own problems The fundamental aim of these lessons is not to solve the students‟ problems but orient them towards the possible measures that they can apply

to find the solutions themselves Career orientation and presentation skills

are integrated into two projects called Dream jobs and Career exploration

By the nature of their names, the projects both aims to encourage students to discover the career world so that they can well prepare for the future job

requirements However, they are still different in that Dream jobs focuses on the job students have dreamt of while Career exploration creates an

opportunity for the discovery of various career options available Students are not to conduct the two projects at the same time but choose one of them

2.2 Participants

The cases for the evaluation were selected based on the reason for their choice of project option As mentioned above, there are two project options –

Dream jobs and Career exploration, of which the latter is assumed to be more

complex as it involves two interviews for career-related information while the former does not Thus, the reason why they choose the project may reflect their initial thoughts of it The revelation of these thoughts somehow leads to that of their attitudes and knowledge in relation to their chosen career

First the evaluator piloted asking a number of students about the

reason(s) for their choice In this case, as the evaluator was once in charge of

College Success class she piloted asking all the students choosing Career Exploration in her own class She has had no intention of involving her

students in this study so as to avoid the effects of teacher-students

relationship It is the fact that she and her students have had good

relationship, as a result, the students would provide the answers that

contented their teachers which may lead the students to contenting their

teacher; it is possible that such a case contributes to the evaluation invalidity

After gathering all the information from the pilot, the evaluator

designed a short questionnaire on the reasons for choosing the project option The answer choices were based on both the project description and the

responses of the evaluators‟ students The detailed questionnaire for case choosing can be seen in Appendix 1 Following are just the summary of the reasons stated in the questionnaire:

I chose Career Exploration because

a I am more interested in this option than the other

b I hope that I will be more active by doing this project

c I want to learn experience from real persons

d I think there is no difference in choosing this option or the other; in both cases I will have to do a research and make a presentation

e This creates an opportunity to do a search on my ideal work place

f I choose it because all of my group members choose it

g Others (please specify)………

The questionnaire was delivered to more than 350 first-year students in

13 groups attending College Success Course in the second semester year

2011-2012 However, nearly half of the students returned the questionnaires One reason was that a number of students were not willing to participate in the evaluation Another number of students did not finish the questionnaire until the project started In this case, the questionnaires could no longer be used as the students could not be accessed before the process Moreover,

some of the groups were behind the schedule, which affected the process of the evaluation; therefore, the students of these groups were excluded from the selecting procedure right from the beginning

After collecting back the students‟ responses, the evaluator classified the students into different categories of reasons The detailed result of the survey of students‟ reasons for their choice can be seen in the following table:

Category Number

of cases

Reasons for their choice

Table 2: Result of reason survey

As can be seen from the table, most of the respondents chose the reasons related to their interest (reason a), their expectation of being more active (reason b) and/ or their willing to talk with experienced people about the required issues (reason c) and do a search for their own work place

(reason e) As stated above, the researcher assumed that different reasons could have different impacts on their intention and reaction to the

intervention Therefore, the cases were from different categories, and at least three cases were selected: one student who chose the project for one reason, one student with two reasons, and the rest with more than two reasons In fact, the researcher did contact more than three cases, but some of them refuse to participate in the next stages of the evaluation, the other drop from the process after they did the questionnaire for the first time At that time, no more students were contacted as the project started Finally, the researcher decided to follow only three cases that were most committed to the

however, it is revealed in ranges of behaviors and speeches As such,

Krosnick, Judd, & Wittenbrink (2005) claim that the evaluation of attitude