assessment of critical thinking in third-year efl students' writings at hanam teachers' training college = đánh giá tư duy phê phán thể hiện trong bài viết của sinh viên năm thứ 3 ngành tiếng anh

Types of fallacies and the reflection of the influence of familiarity on the quality of students’ critical thought Appendix D: Holistic rating scale to mark an argumentative essay XXXL

- -

ĐOÀN THANH HƯỜNG

ASSESSMENT OF CRITICAL THINKING IN THIRD-YEAR EFL STUDENTS’ WRITINGS AT HANAM TEACHERS’

TRAINING COLLEGE

(Đánh giá tư duy phê phán thể hiện trong bài viết của sinh viên năm thứ

3 ngành tiếng Anh trường Cao đẳng sư phạm Hà Nam)

M.A Minor Thesis

ĐOÀN THANH HƯỜNG

ASSESSMENT OF CRITICAL THINKING IN THIRD-YEAR

EFL STUDENTS’ WRITINGS AT HANAM TEACHERS’

TRAINING COLLEGE

(Đánh giá tư duy phê phán thể hiện trong bài viết của sinh viên năm thứ

3 ngành tiếng Anh trường Cao đẳng sư phạm Hà Nam)

M.A Minor Thesis

Field: English Linguistics

Code: 60 22 15

Supervisor: Phùng Hà Thanh, M.A

HANOI – 2010

2.1.1 Critical thinking and logic 5

2.1.2 Critical thinking and cognitive psychology 7

2.1.3 Critical thinking and tool to gain desirable outcome 8

2.1.4 Critical thinking and critical inquiry 9

2.1.5 Critical thinking and universal intellectual standards 9

3.3.1 Criteria of assessment 20

3.3.2 Procedures of assessment 21

4.1.1 Inter-rater reliability 22

4.2 The results of the assessment of critical thinking 23

4.2.1 The display of critical thinking in students’ argumentative writings 23

4.2.2 The numbers of arguments, evidence, refutations, and fallacies in 36 students’ argumentative writings

familiarity on the quality of students’ critical thought

4.3.2 Opposition recognition and refutation and the reflection of the influence

of familiarity on the quality of students’ critical thought

32

4.3.3 Types of fallacies and the reflection of the influence of familiarity on the

quality of students’ critical thought

Appendix D: Holistic rating scale to mark an argumentative essay XXXL

LIST OF FIGURE AND TABLES

Page

Table 1 Correlation of scores given by rater 1 and rater 2 on all 36 writing samples 22

Table 2 Correlation of scores given by rater 1 and rater 2 on writing samples on

Table 4 Average score (Median) and score with highest frequency of appearance

(Mode) of all 36 writings given by each rater

24

Table 5 Average scores (Median) and scores with highest frequency of appearance

(Mode) of each group of writing pieces regarding topic given

24

Table 6 Total number and average number of each element of critical thinking 25

Table 7 Total numbers and differences of elements of critical thinking in writing

samples on familiar and unfamiliar topics

25

CHAPTER I: INTRODUCTION

This chapter presents the statement of the problem, research problems, research questions, scope of the study, significance of the study, and design of the study

1.1 Statement of the problem

From Socratic time, the ability to reason well has been regarded an important outcome of education and training, and since 1980s, ‗critical thinking‘ has become a ‗buzzword‘ in the intellectual circles (Fisher, 2001) Critical thinking is essential for everyone to perform well not only in educational systems, but also in future workplaces, and social and interpersonal contexts Dewey (1933) pointed out learning to think is the central purpose of education Scriven (1985) acknowledged training in critical thinking should be the primary task of education They are among those who recognize human beings do not innately possess the ability of thinking critically but acquire it through education

Although ‗critical thinking‘ is universally accepted as a goal of education as it refers to quality thinking, there remains a controversy around the ‗critical thinking‘ concept across borders As many scholars have claimed, critical thinking took roots in Western societies, especially from the Greek culture with its related schools of academic thoughts and their renowned philosophers Atkinson (1997) believed that critical thinking is a tacit, indefinable, socio-cultural practice and behavior that individuals in the U.S or Western society subconsciously and naturally acquire Nevertheless, many others have argued that the social structures and values that developed in the East represented a different value system from the West have created different conceptions of critical thinking There is

‗individualism‘ in the West that inspires public debate, logical science with the focus on objective observation; and there also is ‗collectivism‘ in the East leads to the individuality suppression, more holistic view of reality with the focus on the group (Nisbett, 2003; Davidson, 1998) These differences have put the ‗critical thinking‘ concept within the matter of ‗culture specific‘ The students from the East may not apparently show up the ability to reason well, but those claimed they lack critical thinking should reconsider the cultural aspects Gieve (1998) introduced the notions of monological critical thinking and dialogical thinking, which pointed out the influences of education background and cultural traits on the way Western and Eastern students exhibit their critical thinking skills With this approach, Gieve (1998) stated that critical thinking is hardly irrelevant or ill-suited to

non-Western ESL students and it has a significant and far-reaching value for students in any society and culture With his opinion, we return to the consensus among discussions related, that human thinking is flawed and has to be improved with the support of education Thus, Western or non-Western students all have to undergo a learning process before claiming themselves ‗critical thinkers‘

As the central target of education in general, the development of critical thinking skills has also become a key goal for educators in first and second language education in particular Teachers of English in EFL (English as Foreign Language) contexts, however, are often constrained by the linguistic skills of their students in Asian countries like China, Japan, Vietnam, etc For example of writing skill, the learners‘ writing pieces always seem to disappoint the instructors, especially when it comes to argumentative essays Vietnamese students, even at tertiary level, have been claimed to lack critical thinking since Eastern learners of English are often characterized as lacking critical thinking characteristics due to those perceptions mentioned above Since both native speakers and non-native speakers have to exert great effort to become ‗critical thinkers of English‘, this claim should be examined carefully Courses on argumentative writings and critical thinking share the focus on arguments and argumentation In most English language curriculums in Vietnam, there is always a part for argumentative writing practice that integrated background of Western arguments and critical thinking skills To find out if Vietnamese students can reason well the way native learners do or not, it is recommended to study their argumentative writings

As an EFL teacher from Hanam Teachers‘ Training College, the researcher of this paper understand the importance of critical thinking to third year students majoring in English since they are about to finish their study with much paper work to be done and communication to be set Thus, while assessing the students‘ skills of analyzing and reasoning in a case study, the researcher wanted to explore whether third year EFL students here displayed features of critical thinking in their English compositions and then, whether content familiarity an influential factor on students‘ thinking process

1.2 Research problems

This study focused on investigating and assessing the display of critical thinking in the argumentative compositions of third year EFL students at Hanam Teachers‘ Training

College Based on the evaluative views of both Western and non-Western raters, it would reveal the extent to which Vietnamese students in their native culture context display their critical thoughts Then, the study would also examine the influence of familiarity on the quality of those students‘ critical thought

1.4 Scope of the study

Within the scope of this study, the researcher wanted to explore the well-know Western concept of critical thinking Since the knowledge related to this concept is boundlessly varied, the researcher‘s attention only rested on the aspect that critical thinking refers to reasoning ability in argumentation That led to her choice of the argumentative writings as the key samples for the study among many other types of writing tasks taught in the Hanam Teachers‘ Training College‘s writing curriculum The reason was that argumentative writings get students involved directly and actively in argumentation process by forming reasons, making inductions, drawing conclusions, and applying them to the discussed issues

Accordingly, the study investigated the argumentative writings by 36 Vietnamese college students from a same classroom All the students are in the first semester of their third year

studying They were all participating in a course called ―English Writing 5‖, which

introducing writing skills for their fifth semester The samples were the participants‘ compulsory writing tasks, which were told be part of their performance evaluation, assigned by the instructor of the course

1.5 Significance of the study

Through this study, the researcher hoped to gather and present useful knowledge about critical thinking for those who have keen interest People who come across this research might gain some knowledge about the nature of critical thinking with different conceptions across disciplines and insights which they might find helpful

It is hopefully that the research results would be practical for students, lecturers, and researchers For the students, the study might generally raise their awareness of the importance of critical thinking and its influence on such specific linguistic skill as writing For the teachers, it may provide original understandings of critical thinking and some suggestions on the employment of critical thinking in teaching argumentative essays For the researchers, the paper would be a referential case for further studies on the related issues

1.6 Design of the study

This study collected 36 argumentative essays written by third-year EFL students at Hanam Teachers‘ Training College Those writings were then analyzed based on criteria informed

by the research problems and research questions mentioned above The correlative statistics provided the correlation coefficients to check the inter-rater reliability in assessing students‘ argumentative writings quality in general as well as on each given topic

in particular The descriptive statistics provided quantitative analysis to support the overall assessment on students‘ reasoning ability displayed in their argumentative writings They also presented the raters‘ qualitative assessment on numbers, types and differences of critical thinking elements in two groups of essays on two different topics

CHAPTER II: LITERATURE REVIEW

This chapter consists of three parts: conceptions of critical thinking; critical thinking instruction, and argumentative writing and critical thinking

2.1 Conceptions of critical thinking

Critical thinking is a rich concept that has been developing throughout the past 2500 years from Socrates time with an overwhelming number of definitions Presenting a defining statement of critical thinking is difficult as it presents different meanings for different people and can be explored using different approaches Thus, in order to develop an overview on conceptions of critical thinking, it is essential to examine prominent conceptions of critical thinking from different angles There were some references to literature but the emphasis was on language acquisition and the opinions of practitioners and learners

2.1.1 Critical thinking and logic

First of all, we should refer critical thinking to the communicative competence, which has been examined under three main categories of skills, understandings and dispositions rooted from logical basis The pioneer scholar who has the influential role on logically conceptualizing critical thinking in early days was John Dewey He coined the term critical

thinking in the 1930s under the name of ―reflective thinking‖ and defined it as "active,

persistent, and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it and the further conclusions to which it tends" (as cited

in Fisher 2001, p.2) As commonly known, logic is the study of inference and in logic instructions with such key terms as argument, premise, claim, reason and conclusion Arguments are claims defended with reasons They are composed of one or more premises, which are statements offered as reasons for accepting another statement or a further unspoken one, and a conclusion, which is statement supported by reasons With this well-known definition, Dewey apparently referred to the basic logical structure of argument with all the elements paraphrased, ‗any belief‘ as ‗premise‘, ‗grounds‘ as ‗reasons‘ and the

‗further conclusions‘ as inferences of our beliefs

As acknowledged by de Bono (1982), most schools followed Dewey approach in equating thinking with logic The logic related conception largely focuses on the analysis of the arguments and the construction of argument Along with time and the scholastic development, informal logic is sometimes presented as a theoretical alternative to formal logic since it is concerned with the content and context of arguments more than their form Formal logicians when treating arguments always assume that the premises are right and only focus on the form of the argument Informal logicians, in contrast, take pain to investigate whether the premises in the arguments are true or not and employ a fallacy theory to check on their work Accordingly, a well-formed argument can appear to be logical but makes no sense while there is a good argument in which the conclusion is supported by the premises even though it does not follow necessary form as validity requirements

Many relate critical thinking to informal logic due to its practical application to analyzing argumentation It is the attempt to develop logic to assess, analyze and improve ordinary language, or everyday reasoning The development of informal logic is tied to educational goals: by the desire to develop ways of analyzing ordinary reasoning which can inform general education To this extent, the goals of informal logic overlap with those of the Critical Thinking Movement in 1980s, which aims to inform and improve public reasoning, discussion and debate by promoting models of education which emphasize critical inquiry

While critical thinking will include evaluation of arguments and hence require skills of argumentation including informal logic, critical thinking requires additional abilities not supplied by informal logic, such as the ability to obtain and assess information and to clarify meaning Many, especially Robert H Ennis (1985) believed that critical thinking requires certain dispositions Understood in this way, "critical thinking" is a broad term for the attitudes and skills that are involved in analyzing and evaluating arguments

The list of dispositions includes such things as being open-minded, paying attention to the total situation, seeking reasons, and trying to be well-informed The four general sets of abilities that are constitutive of critical thinking are clarity-related abilities, inference-related abilities, abilities related to establishing a sound basis for inference, and abilities involved in going about decision making in an orderly and useful way, often called problem solving When combined with the critical thinking dispositions, these four

categories are intended to cover comprehensively the process of deciding what to believe

or do, and he defined ―Critical thinking is reasonable, reflective thinking that is focused on

deciding what to believe or do‖ (Ennis, 1985, p.45)

2.1.2 Critical thinking and cognitive psychology

The second noticeable conception of critical thinking derived from cognitive psychology and the well-known Bloom‘s Taxonomy, which argues for a development of different thinking levels and equates critical thinking with higher-order thinking Bloom‘s (1956) identified six major cognitive categories, which have provided the basis for future taxonomies

Figure: Bloom’s Taxonomy

Bloom defined higher order thinking skills as those that require analysis, synthesis, and evaluation Someone exhibiting cognitive skills of this level will categorize or classify information, comparing and contrasting it in order to make a decision Other characteristics

of higher order thinking include: combining, creating, designing, developing, evaluating, justifying, and measuring A student in an upper-division course should be able to demonstrate all of these cognitive skills when thinking and reasoning through problems Critical thinking skills are an integral part of both higher and lower order thinking as defined by Bloom Critical thinking itself is defined as having skills to generate

information (lower order thinking) and using those skills to guide behavior (higher order thinking) Critically thinking about a set of facts or other information in order to make an informed decision requires the thinker to go through the six levels of cognitive thinking defined by Bloom: knowledge, comprehension, application, analysis, synthesis, and evaluation Major or life-changing decisions that are made without going through this process may not be the best and may exhibit less than satisfactory results This conception is a good beginning, but it has problems One is that the levels are not hierarchical, as suggested by the theory, but rather are interdependent, and they should come under a spiral process, not a linear one For example, although synthesis and evaluation generally do require analysis, analysis generally requires synthesis and evaluation (Ennis, 1981)

2.1.3 Critical thinking and tool to gain desirable outcome

The third way to conceptualize critical thinking is to combine logic and cognitive psychology approaches Halpern (1996) defined critical thinking as ―the use of those cognitive skills or strategies that increase the probability of a desirable outcome It is purposeful, reasonable, and goal-directed.‖ (p.33-34) It is also known as directed thinking, against non-directed thinking, which is defined as daydreams, nightdreams, and other sorts

of thinking that are not engaged in for a specific purpose or do not involve the use of critical thinking skills It is used to describe thinking that is purposeful, reasoned, and goal directed—the kind of thinking involved in solving problems, formulating inferences, calculating likelihoods, and making decisions when the thinker is using skills that are thoughtful and effective for the particular context and type of thinking task In a more

extensive observation, Halpern (2003) emphasizes that ‗all the cognitive levels are under

the power of that conception, and the “critical” element of critical thinking denotes the evaluation of thought processes and their outcomes‘ (as cited in Phung, in press) Thus, the

final and most worthy target of thinking is to achieve a desirable outcome However, the problem is that, Halpern (2003) also noted that desirable decisions were embedded in a

system of values Since ‗desirable‘ is a vague term, something that is desirable to one

person might be undesirable or even conflicting to others One‘s desire has to be framed within certain ethical and social boundaries She also acknowledges that thinking is only a means to solve specific problems in particular context Thus, as Phung (in press) pointed

out, this conception does not fit in educational perspective since it touched upon daily life matter and varied in different contexts

2.1.4 Critical thinking and critical inquiry

The fourth conceptualization of critical thinking is more or less related to the critical inquiry and critical literacy movement, which can be roughly understood as any kind of investigation that attends to power relations and seeks to change the current situation of oppression (Crotty, 1998, as cited in Phung, in press) As observed by Temple (2005), the critical literacy movement contains many dimensions that people who advocated for critical thinking found useful; however, the critical thinking movement promoted many

practices and principles that should be maintained, even as we entertain new practices One

of the dimensions mentioned above is the premise that language is always used in some context that includes power relationships Thus, language becomes a form of politics All texts, including scripts for movies, television shows, and advertisements, are written by someone for a purpose Since those purposes are not often transparent, readers need to develop and exercise their critical faculties to filter what they understand and believe from

texts Temple (2005) also pointed out that ―We are constantly assaulted by language that is

not just unclear, but often deliberately deceptive and manipulative Students need tools for unmasking the true purposes of language within a particular context so they can both understand its true meaning and, as necessary, free themselves from its pernicious effects.‖

Proponents of this conception with a view to embed socio-political component into critical thinking provides opportunities to take advantage of the achievements not just for college-level philosophy classes, but for all readers, even at lower levels They passionately believe that the study of the practical, even political, uses of language should be brought into the school curriculum These new focuses include sensitivity toward both the grammar of obfuscation and the rules that ascribe roles and meanings to language and its users in social settings Critical discourse analysis is a subject matter that deals with these targets In a broader scale of education, as in critical pedagogy, those who promote critical thinking in their classrooms will find those insights and tools useful and practical

2.1.5 Critical thinking and universal intellectual standards

And the last, but never the least to mention here is the conception of critical thinking

developed by philosopher Richard Paul He defined critical thinking as ―mode of

thinking about any subject, content, or problem in which the thinker improves the quality of his or her thinking by skillfully taking charge of the structures inherent in thinking and imposing intellectual standards upon them." (Paul, Fisher and Nosich, 1993, p.4) As observed by

Fisher (2001), this definition is interesting because it draws attention to a feature of critical thinking on which teachers and researchers in the field seem to be largely agreed That is, the only realistic way to develop one‘s critical thinking ability is through ‗thinking about one‘s thinking‘, and consciously aiming to improve it by reference to some model of good thinking in that domain

Paul and Elder (2006), meanwhile, offered a list of what they call "elements of thought" with purpose, information, inferences/conclusions, concepts, assumptions, points of view, implications/consequences, and questions Along with those are nine suggested qualities

that make messages optimally useful including clarity, accuracy, precision, relevance,

depth, breadth, logic, significance, and fairness By implication, they refer to quality

questions of specification, amplification, kind/category, verification/validation degree, magnitude, motive, detail, and questions designed to probe, challenge, and motivate However, what make this conception remarkable is that unlike those other ones which merely touch upon one or two fields, such as logic, psychology or socio-political, this conception is regarded transdisciplinary since the proponents can be said to set out the shared criteria of critical thinking for all scholastic fields else where in the world

To sum up this part, the researcher would like to refer to two remarkable observations about dimensions of critical thinking Out of the two, the latter is more preferable since it has covered up all the conceptions we have mentioned so far

First, according to Moon‘s (2004) there are dimensions of breadth, longitudinal, and technical Breadth dimension refers to narrow views of critical thinking which focus primarily on the language of argument and reasoning, or broader, viewing the characteristics of the critical thinker as leading to a way of operating in the world Longitudinal dimension, which acknowledges a person‘s past and encourages development through construction of past and new knowledge It involves studies of critical thinking at

a particular educational stage Technical dimension is often characterized by lists of skills

or procedures (which may even be expected to occur in a particular order) There is also a dimension that relates to the way in which critical thinking is viewed in relation to the discipline of the learners and a dimension that relates to the origins of the researcher or writer Cognitive psychologists are likely to have a different view of critical thinking (often

as a form of problem solving) from the philosophers (who may see it as a form of logic) from the educationalists

From a quite different angle, Phung (in press) proposed a reconceptualization of critical thinking as a six-dimensional construct that is psychological, logical, semiotic, socio-political, methodological and educational because these dimensions correspond to the factors regulating our thinking She reasoned:

Focusing on informal logical reasoning the logical dimension points to the fact that thinking is an inference process The psychological dimension acknowledges thinking

as a psychological process, urging thinkers to understand the mental functions of the human brain The semiotic dimension denotes that thoughts are stored in and conveyed through languages The socio-political dimension refers to the influences of social power relations on thinking Systematic strategies and principles applied to thinking form the methodological dimension, which consists of two overlapping components One includes strategies and principles that are used to guide thinking in every situation They cover all the above-mentioned dimensions and supposed to be insights from the most relevant branches of studies such as informal logic, cognitive psychology, semiotics, and critical discourse analysis The other refers to technical theories used to solve technical problems Phung (in press)

2.2 Critical thinking instruction

Almost everyone agrees that one of the main goals of education, at whatever level, is to

help develop students‘ general thinking skills, including their critical thinking skills in

particular Almost all of them also agree that students do not acquire these skills as much

as they could and should That is why teaching critical thinking has become more and more urgent worldwide However, there is a big distance between defining what critical thinking

is and how to teach it One cannot just push all the related theoretical wordings of conceptions into their learners‘ head to turn them into critical thinkers Still, as van Gelder (2005) pointed out, college instructors often made the mistake of thinking that they could teach critical thinking skills by teaching the theory of critical thinking He recognized that grasp of theory was an absolute necessity for advanced critical thinking but it was wrong when one only taught theory, or overemphasized theory relative to practice Skills are not a natural outcome of theory and must be naturally develop through practice However,

practice is more effective when supplemented by appropriate levels of theoretical understanding That is the reason why for years, keen educators have been trying hard to find proper way to adopt the theoretical understanding of critical thinking into their instructions to gain the most effective and productive effects

As we have examined, the fifth critical thinking conception with Paul‘s model of universal intellectual standards have addressed comprehensively all aspects of quality thinking

Critical thinking from this view, as Phung (in press) pointed out “is not only about

cognitive capabilities but also about moral commitment to the critical standards and traits‖ Accordingly, when students are taught of critical thinking, besides the knowledge

and thinking skills, they also have to learn about the intellectual standards, as well as ethical and social values This is great but quite an ambitious goal for educators of critical thinking to reach because despite the universality of these intellectual standards, their application in a specific context still depends on cultural features and other conditions Moreover, a thorough instruction of any subject requires systems of clear procedures to evaluate the learners‘ achievements, which is vaguely shown in Paul and Elder‘s theory Their criteria are not widely applied into every critical thinking course book ever existed because of the different cultural awareness, for example, criteria of fairness, clarity and precision seemed to be viewed differently in Eastern and Western cultures

The same situation happens to Bloom‘s taxonomy and the cognitive psychology related conception It is hard when we try to press it into a conceptual framework for teaching

critical thinking, as Ennis (1985) observed, ―the five concepts Bloom‟s introduced are too

vague to guide us in developing and judging critical thinking assessment [ ] What do you assess when you test for ability to analyze?‖(p.45) As a result, since Bloom‘s taxonomy is

suitable for illustrating critical thinking as higher-order thinking, it fails to provide the basics to construct a relevant instruction for teaching and learning critical thinking as a subject matter

The third and the fourth conceptions mentioned above seem more or less depart from the teaching and learning activities to be use as main source or background for instruction

designing The one proposed by Halpern as observed by Phung (in press) ―too broad and

too narrow from an educational perspective”, and the socio-political one is still on the way

to find its place in school curriculum That is the reason why at present there are not many

course book or syllabus of critical thinking constructed based on those instructions If there are any, they are rarely used or used as complementary materials

It is unarguably that there exists many other conceptions of critical thinking and some of those may be developed and used in teaching critical thinking context However, similar to the ones mentioned lately, they do not occupy as large a proportion as logic in critical thinking instruction and assessment As Scriven and Paul (2004) observed, thinking is a natural process, but left to itself, it is often biased, distorted, partial, uninformed, and potentially prejudiced; excellence in thought must be cultivated Thus, generally, critical thinking instructions, either focusing on skills only or on skills integrated with subject content, often aim at teaching students how to think critically, how to reason well Especially in language teaching, though many criteria or standards proposed by other conceptions are necessarily important parts of critical thinking, reasoning always takes first place Logic deals with the quality of reasoning and the argument, and up to now, logic has still been most dominant referential basis for constructing and designing critical thinking course and course book The first reason is that the application of logic was seen to maximize the ‗objectivity‘ of critique and argument When people teach critical thinking skills based on logical basis, they benefit from available logic theory, which is a tool that

show ‗what is implicit in the concepts used [in argument] and to expose contradiction‘ de

Bono (1982, p.77, as cited in Moon, 2004, p.38) He also implied that logic may have held its popularity because, while it was considered that students should be taught how to think,

other than logic there seemed to be little to grasp hold of and to then teach – ―what set of

rules for thinking was there, apart from logic?‖ (de Bono, 1982, as cited in Moon, 2004,

p.39) The second reason, logic, together with the emergence of its branch informal logic, provides quite an exhaustive set of procedures for evaluating arguments with detailed explanation of given terms like premise, conclusion, etc and a full theory of fallacies As a result, both the coach and the coached are beneficial from it The instructors find it easier

to teach their learners with clear-cut concepts; the learners can evaluate their own achievements via comprehensible criteria of diversified test and assessment forms The educational targets of the critical instruction thus will be systematically and effectively assessed The criteria of informal logic for appraising critical thinking is sure more clearly defined than those by Bloom or Paul and Elder

With those reasons mentioned above, though approaching critical thinking from logic (or more exactly, from informal logic) is quite a narrow dimension, which focuses only on the language of argument and reasoning, instructions designed on logical basis are still considered the most proper one used and to be used on a widest scale

2.3 Argumentative writing and critical thinking

This part aims at the relation between argumentative writing and critical thinking in learning language and the way people use argumentative writing to assess critical thinking First of all, it is necessary to look at the definition provided by Baker & Brizee (2007,

para.1) ―Argumentative writing is the act of forming reasons, making inductions, drawing

conclusions, and applying them to the case in discussion; the operation of inferring propositions, not known or admitted as true, from facts or principles known, admitted, or proved to be true.” Accordingly, the key function of an argumentative essay is to show

that your opinion, theory, or hypothesis about something is correct or more truthful than others It is never easy to acquire skilled argumentation One has an opinion does not mean one can debate it successfully with someone else even though their point seems so rational and logical The definition has clearly explained the process of reasoning from the known

or assumed to the unknown forming reasons, making inductions, drawing conclusions, and

applying them to the discussed case Without doing this you do not have an argument, you

have only an assertion, an essay that is just your unsubstantiated opinion

Structurally, argumentative essays often begin with a statement of your assertion with its timeliness, significance, and relevance in relation to some phenomenon Next, it reviews critically the literature about that phenomenon The accompanied part illustrates how your assertion is "righter" than others‘, including more reliable or valid methods that you employed to explain the case In another definition by Purdue OWL writing laboratory,

‗argumentative essay is a type of writing that requires students to go through steps of

investigating topic, collecting, generating and evaluating evidences to finally establish their own stances in an appropriate manner‟ Accordingly, when writing an argumentative

essay, writers focus on presenting their argument successfully and convincingly to make others to agree with their facts, share their values and accept their conclusions Both those two definitions called for the writer‘s ability to produce not just an understandable but

logical and convincing paper Thus, the ability of reasoning and refuting well are what a good argumentative essay writer should possess

Since the ultimate goal of writing argumentative essay is to persuade readers to believe in writers‘ point; writing then should be a real reflection of writers‘ thought and ideas The strategy of stating arguments, giving relevant supporting evidences and examples as well

as manners of refuting the counter-argument that a critical thinker employs will all be presented in his argumentative essay This implies that a good thinker makes a good writer and a good critical writer makes a good argumentative paper Sachs (2004) pointed out the interrelation between critical thinking and writing, especially argumentative writing, a process of doing critical thinking and a product communicating the results of critical

thinking As such, one can use written argumentative texts to measure critical thinking

skill Since this paper deals with the assessment of critical thinking in writing, it will now

provide a look of several general knowledge standardized essays tests for critical thinking

As observed by Stapleton (2001), although critical thinking has received much attention in the second language learning field, little attention has been paid to the critical thinking assessment test The existing critical thinking test tend to be test specific with criteria laid out that specifically pertain to the content of the test itself McPeck (1990) claimed to know of at least 26 tests designed to measure critical thinking ability, but they are often limited to multiple-choice instruments that do not allow any probing of reasoning behind the examinee‘s answer The textual analysis and scoring guides for testing critical thinking quality in written discourse seem quite rare One of the most widely used assessments of critical thinking in writing is the Ennis-Weir Critical Thinking Essay Test It is a general

test of critical thinking in the context of argumentation and regarded as a diagnostic device

to identify specific area of reasoning or argumentation (Stapleton, 2001, p.514) ―It is the only critical thinking test that asks students for a written response judging the quality of reasoning in a piece of writing‖ (Hatcher, 1995, p.27, as cited in Paul, 2001, p.515)

According to the test‘s grading guides, there is a list of area of critical thinking competence:

 Getting to the point

 Seeing reasons and assumptions

 Stating one‘s point

 Offering good reasons

 Seeing other possibilities (including other possible explanations)

 Reversal of an if-then (or other conditional relationship)

 The straw person fallacy

 Overgeneralization

 Excessive skepticism

 Creditability problems

 The use of emotive language to persuade

Responding to Ennis-Weir Test of Critical Thinking, Stapleton (2001) agreed that general concepts of critical thinking can be derived from this model, but like many other tests of this sort, he thought those criteria seem content specific He then proposed a model to address that lack of adequate critical thinking tests by offering a scheme to assess any argumentative passage Raters follow this model to identify key elements of critical

thinking displayed in the given writing, assessing each paper for (a) number of arguments,

(b) extent of evidence, (c) recognition of opposing arguments, (d) corresponding refutations, and (e) number of fallacies

Both two models possess through time have proved their effectiveness In this study, their most applicable points will then be combined to make a measuring tool of critical thinking

in students‘ argumentative writing Critical thinking could be assessed via argumentative essays in terms of elements taken from Stapleton‘s proposal with related criteria provided

by Ennis-Weir The process of assessing with the detailed explanation of the model will be discussed in the next chapter

CHAPTER III: METHODOLOGY

This chapter recites the methodology conducted in the study, including samples and sampling, data collection, and data analysis

3.1 Samples and Sampling

3.1.1 Samples

The samples are purposively selected since the study only focused on one of many types of writing tasks the students had learnt during all compulsory writing courses The pieces chosen are written by third year students since at the time the study was carrying, they

were working on argumentative writing in their Writing 5 course The instructor of the

course, who worked independently with the researcher of this study, provided all 36 papers The writing tasks were carried out under time pressure during class hours to be submitted for the course‘s evaluation requirements The whole set of 36 writing pieces will

third-call Writing 5, and were being taught by a same teacher with same curriculum, activities

and evaluation system

3.1.1.2 Topics

With the approval and assistance from the instructor of the Writing 5 course, 36 students

were accidentally divided into two sub-groups of equal number then each group was assigned with writing topics prepared beforehand by the researcher There are two topics, one of which was familiar to the majority of Vietnamese people while the other deemed unfamiliar Accordingly, half of the randomly selected participants worked on the familiar topic and the other half on the unfamiliar one One noticeable factor is that prior to the testing time, students had been provided with several topics including the two topics used

in this study

3.1.1.3 Raters

Due the subjectivity required in any assessing process, the study relied on the working of two separate raters to read and score all the writing samples Those two raters were deliberately invited based on their cultural background, working experience and specialized teaching areas The first rater (or rater A for convenience) is a Vietnamese teacher who has been teaching critical thinking for years The second rater (rater B) is an experienced Australian teacher who has taught English to both native and non-native learners Both neither work at the college where this research was conducted nor have direct contact with the students here

3.2 Data Collection

3.2.1 Instrument

Best and Kahn (1993, as cited in Ghonaim, 2005) regarded documents among the most significant sources for data collection This source helps bring more insightful information, thus increasing the reliability of the study findings Since the study focused on evaluating students‘ real reflection of critical thinking ability in their writings rather than their perceptions in general, the researchers made a decision to use students‘ essay tests as a main instrument

3.2.2 Procedures

3.2.2.1 Design of the writing tasks

Before explaining the writing tasks as main source to collect data for this study, the

researcher would like to clarify the term ―familiarity‖ It relates to theories on schemata

and knowledge structures Rumelhart (as cited in Stapleton, 2001) described schemata as prototypes of memory arising from familiar experiences that individuals use to interpret related knowledge When people are given information, those who already know can develop their knowledge by organizing related principles and notions and link to its application On the other hand, the schemata of people who find the information new may contain certain ideas about the situation but lack knowledge of related principles and their application

The familiar topic requires students to respond to the statement “Tobacco companies

should compensate smokers who have become ill as a result of smoking” The issue of

smokers‘ demand of compensation was selected because of the following reasons Firstly,

it was always one of the most frequently used topics in writing at any level, although the question was modified a little Secondly, people keep talking about smoking largely on

media means with its pros and cons, and the public attention has recently directed to the rights of consumers with many as cases in which the buyers and users filed the producers All of the information sources related to the issue seemed close and rich enough for the students to get used to the tone of such kind of debate and easily develop their own arguments The third reason was that among many topics assigned as homework, when

informally asked by the researcher „which topic would you be most ready to work on?‘ 33

out of 36 students picked the topic on smoking

For the unfamiliar topic, the legalization of prostitution was chosen Students were asked

to respond to the statement “Prostitution should be legalized because it brings a lot of

benefits” This topic was deemed unfamiliar because prostitution is forbidden in Vietnam

and people quite often avoided talking about this As an Eastern culture, the question of whether or not should prostitution be legal hardly ever occur to Vietnamese, let alone discuss about it openly Another reason for choosing it was the fact that few students voluntarily worked on such topic

It is believed that those two topics would provide participants a good opportunity to use critical thinking skills to explore the topics‘ complexities As Stapleton (2001) pointed out,

a potential criticism of using familiar content to elicit critical thinking is that it encourages well-rehearsed reasons and evidence absorbed through exposure to the media, schooling, and parents Such exposure can build prejudices that tend to hinder effective critical thinking because they can block out alternative viewpoints On the other hand, Glaser (1984) claimed that people having little familiarity with a topic lack the schemata with which to infer further knowledge (as cited in Stapleton, 2001) As such, it is presumed that the benefits of schemata evoked by a familiar topic to outweigh its potential hindrance Therefore, the main aim of choosing both a familiar and unfamiliar topic was to explore how students‘ rich schemata with regards to compensation for smokers with their schemata

on legalization of prostitution, which was supposed to be poorer It was hypothesized that rich schemata would enhance critical thinking abilities

3.2.2.2 Essay tests administered and collected

Before doing the argumentative writing tests, the students were given several topics to prepare at home, some of them were considered familiar to them and some were unfamiliar As agreed by the course instructor, the researcher inserted the two to-be-tested topics into the students‘ homework They had been advised to collect information from

Internet and read materials related to the assigned topics before actually composing their own writings on one of them in the test

The essay tests were carried out at the end of the course, with the time allowed of 60 minutes and under the strictly observation of the instructor and the researcher The participants did not know which of the given topics they were going to work on until the testing time The ones who wrote on familiar topic and the ones who wrote on unfamiliar topic were randomly selected

When the testing time was up, the students submitted their work The researcher collected all the 36 writing pieces were then made into identical hard copies and soft copies of Microsoft Word files for the purposive retention and the assessment process of the study

3.3 Data analysis

3.3.1 Criteria of assessment

To analyze the data collected, the study mainly based on the key elements of critical thinking proposed by Stapleton with reference to Ennis-Weir‘s criteria:

Arguments: Arguments are claims supported by reasons; unsupported claims are merely

opinions If the reason supporting a claim was deemed inadequate, then the argument was considered flawed

Evidence: The ways a writer provides evidence to support a reason can come in many

forms, including personal experience, research studies, statistics, citation, analogies, pointing out consequences, and precisely defining words Pieces of evidence are isolated and categorized accordingly

Recognition of opposition and Refutation: They are participants‘ recognition of the

multisided nature of the issues in question and their attempt to refute them

Fallacies: Fallacies are simply understood as flaws in arguments They describes the

different types of errors in reasoning, occur when the reason does not adequately support the claim in one or the other way Ramage and Bean (as cited in Stapleton, 2001) identify three broad types of fallacies based on appeals to pathos, ethos, and logos Pathos fallacies are flaws in the relationship between what is argued and the audience They include

appealing to stirring symbols, provincialism, appealing to emotional premises, and red herring Ethos fallacies are flaws in the relationship of the argument and the character of

those involve in the argument) They are appeal to false authorities, attacking the

character of the arguer, and straw person Logos fallacies are flaws in the relationship

between the claim and the reasons or evidence in an argument They are named as

irrelevance, false analogy, hasty generalization, slippery slope, oversimplication, and begging the question The specific types of fallacies found in the students‘ writings will be

explained and analyzed in the next chapter, with reference to Dowden (2007)

3.3.2 Procedures of assessment

Each writing sample was assessed blindly by two raters for an overall view of students‘ reasoning ability displayed via their argumentative pieces The raters gave marks based on their own perception of argumentation and their teaching experiences They were supplied with the 36 essays samples (Appendix A), the rubrics for critical thinking elements (Appendix B) and a rating guide (Appendix C) For further consultation, they also had a holistic rating scale for argumentative writing (Appendix D) provided by the instructor of the writing course The scores both raters give 36 writing samples on both familiar and unfamiliar topics will then be analyzed for inter-rater reliability Then, rater A took exclusive charge of assessing those 36 writing samples for the elements of critical thinking based on the combined model of Ennis-Weir and Stapleton The reason the rater B did not join this assessment process according to him was his self-declared limited knowledge of critical thinking Since rater A has been trained intensively to teach critical thinking skill for tertiary students with years of experiences, her assessments are assumed to be reliable and persuasive enough She provided data of elements of critical thinking appear in both writing samples of familiar and unfamiliar topic They would then be categorized and compared for differences in critical thinking quality displayed in students‘ writings regarding familiarity of the given topics

It should be noted that when dealing with a notion such as critical thinking, which has multiple definitions, concerns arise as to whether the ‗thinking‘ displayed in the writing of the participants truly reflects their real thought processes For example, although participants may write views that oppose their own, and be given credit in the model‘s scoring system for doing so, this does not necessarily mean that they deeply considered these views, as required by true critical thinking Rather, they may be simple following argumentative form and avoiding fallacies in the absence of sincere reflection Although this remains a difficult problem to overcome, the various quantitative and qualitative measures used here sought to detect this ‗weak sense‘ of critical thinking

CHAPTER IV: RESULTS AND DISCUSSIONS

This chapter deals with the results of the study along with discussions related to the findings

4.1 Reliability and Validity of the Assessment

4.1.1 Inter-rater Reliability

Table 1 shows the inter-rater reliability of the assessments of both raters on students‘ overall reasoning quality The inter-rater reliability of both raters‘ assessments on writing samples on familiar topic and unfamiliar topic are displayed in Table 2 and Table 3, respectively

Table 1: Correlation of scores given by rater 1

and rater 2 on all 36 writing samples

Table 2: Correlation of scores given by rater 1 and rater 2 on writing

samples on familiar topic

familiar topic Rater A

familiar topic Rater B

Table 3: Correlation of scores given by rater 1 and rater 2 on writing

samples on unfamiliar topic

unfamiliar topic Rater A

unfamiliar topic Rater B

** Correlation is significant at the 0.01 level (2-tailed)

By using the Statistical Package for Social Sciences software (SPSS), we have Table 1, Table 2 and Table 3 respectively represented the correlation coefficients Pearson r = 712, r

= 729 and r = 717 for scores on the total 36 argumentative writings, for scores on writings

on familiar topic and for scores on writings on unfamiliar topic The correlation coefficients indicated the great significance of the agreement between two raters on assessing overall reasoning quality of all 36 students‘ essays and on assessing both groups

of writings Accordingly, it can be said that the results given by two raters are significantly correlative and the study proved acceptable validity

4.1.2 Validity

Because this case study is not designed to be generalized, we would consider its internal validity, which largely rested on the choice of raters for assessing students‘ critical thought quality Both raters were language teachers deliberately invited based on their qualifications, working experiences and specialized background related to teaching writing skill and critical thinking skill Moreover, two raters had no direct influence on the participants with their isolating assessing process during the time this study was conducted The study, thus, can be regarded as having shown proper validity

4.2 The results of the assessment of critical thinking

4.2.1 The display of critical thinking in students’ argumentative writings

From Table 4 below, generally, the average mark (Median) rater A gave all 36 papers was 4.5 with the most popular mark (Mode) was 3.75, meanwhile, rater B gave an average score (Median) of 5 with the most frequently seen (Mode) mark of 5 Based on the holistic rating scale (Appendix D) giving the score range from 1 to 9 that two raters consulted when giving scores to the students‘ writings, their score set often fell within the middle part of the scale It proves, when being assessed by two different people of different cultures, students‘ writing pieces do display certain grasp of critical thinking with the most popular scores revolved around average scores

Table 4: Average score (Median) and score with highest frequency of appearance (Mode)

of all 36 writings given by each rater

Table 5: Average scores (Median) and scores with highest frequency of appearance

(Mode) of each group of writing pieces regarding topic given

Familiar topic–

Rater A

Familiar topic – Rater B

Unfamiliar topic – Rater A

Unfamiliar topic– Rater B

As the two tables above indicated, there definitely existed critical thoughts in students‘ writing samples considering the whole sampling of 36 essays as well as two separated groups on two different topics Third year EFL students at Hanam Teacher Training College did have an average grasp of critical thinking and reasoning in their written argumentation

4.2.2 The numbers of arguments, evidence, refutations, and fallacies in the 36 students’

argumentative writings

Table 6 below provides the total numbers and average number of each element of critical thinking tallied by rater A, who possesses more extensively trained background of and authentic experiences on critical thinking instructions

Table 6: Total number and average number of each element of critical thinking

or refutation, as well as had 1.2 fallacies These raw results, as observed, only reveal the amount of output from each student, that is, the number of critical thinking elements, without revealing its quality We are now having separate looks at the two groups of argumentative writings on two topics given

4.2.3 The differences of critical thinking elements found in two groups of writings

Table 7: Total Numbers and Differences of Elements of Critical Thinking in Writing Samples on Familiar and Unfamiliar Topics

Variable

Familiar Topic

Unfamiliar Topic

Difference (in positive number)

However, it is interesting to observe that one of the listed elements of critical thinking was found in the writings on unfamiliar topic (2 refutations) while there was none in the ones

on unfamiliar topic Besides, according to Table 7, participants writing on familiar topic made fewer fallacious arguments than the ones writing on unfamiliar topic with the difference of 26 fallacies Although the statistics appear to suggest that familiarity with a topic enhances the number of arguments, evidence pieces and fallacies, it says little about the participants‘ ability to recognize other viewpoints and refute them

4.2.4 Types of arguments, evidence and fallacies found in each group of argumentative writings

4.2.4.1 Types of Argument

Appendix B1 shows the total of arguments for both familiar and unfamiliar topics In

responding to the familiar topic statement, ―Tobacco companies should compensate

smokers who have become ill as a result of smoking”, 41 arguments come under eight

distinct types with two provided by those who agreed with the statement and six by those

disagreed By agreeing with the topic, the writers emphasized the (a) The danger of

smoking and (b) The irresponsibility of tobacco companies with three further subdivided

arguments The rest of arguments, a remarkable large amount, shows the disagreement

with the statement and the embedded prompt “smokers who have become ill as a result of

smoking” The writers disagree with the statement because of six main reasons: (1) Individual choice accompanies individual responsibility, (2) The dangerous effects of smoking are well known, (3) Tobacco industry is permitted by law, (4) Tobacco companies bring benefits to countries, (5) Tobacco companies warn smokers about the danger of smoking, and (6) The rule of demand and supply

On the other hand, in responding to the unfamiliar topic statement, ―“Prostitution should

be legalized because it brings a lot of benefits” there is only one argument counted as

showing the agreement, that mentioned the financial benefits of legalization of prostitution with two further subdivided pieces of reasoning The remained arguments fall within three

different opinions against legalization of prostitution because it (1) Degrade the human

society, (b) Threaten the women‟s lives, and (c) Cause other social evils and serious diseases According to those data, not only the total number of arguments in writings on

familiar topic but also the number of argument types is higher than those in the ones on unfamiliar topic

4.2.4.2 Types of Evidence

Appendix B2 lists the pieces of evidence for both groups of writings on familiar and unfamiliar topic writings According to the statistics, both sets of writings encompassed a wide range of evidence types; it appears that the depth and variety of actual pieces of evidence in the familiar topic writing is more notable The writings on familiar topic provide 32 distinct pieces of evidence with six different types, whereas the unfamiliar topic ones supply 22 with only three types of evidence Participants writing on familiar topic were able to include three forms of evidence – experience, citation and explanation, that were absent from the unfamiliar topic The evidences found in unfamiliar topic writings largely rely on statistics and research studies, which mainly indicate the unsafe and full-of-criminal society with legalized prostitution The focus of those writings were generally put

on the ‗safety needs‘ and ‗social needs‘, concerning people‘ s desires to avoid pain and injury and to seek for companionship and love The evidences found in familiar topic writings, meanwhile, cover more aspects from ‗safety needs‘, ‗social needs‘ and even reach to the level of ‗esteem‘ involving responsibility and self-respect with higher density

of repetition

4.2.4.3 Opposition Recognition and Refutation

Appendix B3 presents the opposition viewpoints and refutations taken from writings on both familiar and unfamiliar topics For unfamiliar topic writings, there are two opposite viewpoints recognized and refuted, while there is none in familiar topic writings Although the number is quite small, the participants writing on unfamiliar topic did attempt to give the counterarguments about the benefits of legalized prostitution and refute them, showing their prominence in argumentation over the others who worked on familiar topic without

no recognition or refutation attempts at all Since both writing tasks just asked participants

to respond to a short statement with a little prompt embedded under time pressure, it is not expected that the opposite viewpoints recognition and refutations would be of any large amount However, as mentioned above, participants writing on the unfamiliar topic still managed to present two opposition viewpoints then refuted them to support their arguments, whereas the number of refutations in familiar topic writing is zero (Appendix B3) It can be assumed that almost no participants of both topics paid proper attention on recognizing and refuting opposite viewpoints, which put their argumentation quality at

average level Nevertheless, as the statistics showed, the participants writing on unfamiliar topic seemed to have better awareness of this element of critical thinking

4.2.4.4 Types of Fallacies

Appendix B4 enumerated the fallacious arguments in both sets of writings As noted above, the fallacies found in the unfamiliar topic writings were four times higher than those in the familiar topic ones (34 against 8) In addition, the variety of fallacies in unfamiliar topic writings is also greater than in the familiar topics Generally, the prevalent fallacies related to flaws in the relationship between the claim and the reasons or evidence

in an argument, the logos ones Both groups of writings had fallacies categorized as

slippery slope and irrelevance While the fallacious arguments from writings on familiar

topic were only confined to those two types, participants of the other writing topic

committed four more types, which are false analogy, hasty generalization, straw man and

overgeneralization It seems that content familiarity of the given topic played some role in

enabling participants avoid making argumentative errors

Many of the errors of reasoning from legalization of prostitution topic fell in the

“oversimplication” and “false analogy”, categories, which may indicate that participants,

in their zeal to support ideas, failed to recognize that they had gone beyond a point of reasonableness They might refer to some sources for their argumentation, but their

reasoning were still too simple, as in ‗Prostitution detrimentally affect society in the form

of broken marriages, divorce rate will increase because men are usually flirtatious so it is easy for women in prostitution industry to seduce them‟ Sometimes the reasons linked to

irrational comparison due to writers‘ simplified knowledge of the issue, one of the fallacies

of this kind are ‗Like all other professions, prostitution also brought diseases and bad

consequences to human bodies‟

Writings on both topics saw the presence of irrelevance fallacy, which presents details or facts that are off the point and do not support the thesis For example, in familiar topic

writing, the student wrote ‗The tobacco companies not only make big profit but also

produce important item on the market It is useful in festivals, weddings, funerals, etc‘ It

did not support the given opinion about why the tobacco company should not compensate the smokers The same errors found in writings on unfamiliar topic, one of those several

students committed was the idea that legalized prostitution „is a way to attract many

workers in rural areas where average income is quite low This led to labor shortages in

agriculture It also attracted many workers who don‟t want to move their hands and brain They are so lazy and dependent on others.‟ when trying to argue against its benefits

Instead of giving the counter evidence, the students ran on with the facts about lazy or dependent people with low income and from rural areas The presence of this type of fallacy, as the critical thinking instructor rater observed, might result from participants‘ carelessness in making argumentation or superficial background of the problem

Another common type of fallacy participants of both topics generate was slippery slope To

understand this kind of fallacy, suppose someone claims that a first step (in a chain of causes and effects, or a chain of reasoning) will probably lead to a second step that in turn will probably lead to another step and so on until a final step ends in trouble If the likelihood of the trouble occurring is exaggerated, the slippery slope fallacy is committed

In case of the familiar topic, many participants favor the argument ‗If smokers can get

compensation from tobacco companies, what comes next? Lawsuits against drinks manufactures over cirrhosis of the liver?/An action against the dairy industry by heart- disease sufferers?‟ According to the rater, ‗it is a slippery slope fallacy because forcing tobacco manufacturers to compensate does not necessarily lead to wine or dairy manufacturers to compensate their customers, too‘ In writings on unfamiliar topic,

participants also made this type of error with about twice as many numbers of slippery slope fallacy listed It could be said that the facts were either too exaggerated or the writers

were too innocent to produce their assumption A good example is ‗Legalization of

prostitution could make our families broken because some people (either wife or husband) couldn‟t bear a fact that their spouse has another person They would be very sad and do negative things It could make our families broken because some people (either wife or husband) couldn‟t bear a fact that their spouse has another person They would be very sad and do negative things When the families are broken, the children lose their father or mother and lack the care from their parents They are too hurt to develop normally as other children They would lack the confidence and not believe in good things in their life‟

The writers certainly cannot be sure how many percent of probability the first thing, legalized prostitution, might cause the last in chain, the lack-of-confidence generation

Two other types of fallacies found exclusively in writings on unfamiliar topic were hasty

generalization and straw man If the former reveals participants‘ insufficient or

unrepresentative evidence, the latter let us know those who work on this topic does not

really understand the problem they were discussing and tried hard to attacked their

distorted understanding For example of hasty generalization, a participant writes „We

don‟t have any reason to consider it illegal, both sides agree with what they trade‟, with

the unprovided resources the information is taken, it was unreliable to jump to such

absolute conclusion For example of straw man, when writing ‗Pimps and traffickers are

transformed into third party business and legitimate sexual entrepreneurs‟, instead of

proving the argument that legalization of prostitution brings less benefits than harm, the student just focused on how the pimps and traffickers could benefit from and failed to gain the rater‘s approval

4.3 Discussions

4.3.1 Types of arguments and evidence and the reflection of the influence of familiarity

on the quality of students’ critical thought

Although the data appear to indicate that the participants had a grasp of critical thinking at some level, the analysis below provides specific insights that arose from the study One

such insight concerns levels of argumentation or the depth of knowledge, which is “an

important factor in determining a learner‟s ability to display innovative thought since it is directly linked to critical thinking‘, Browne & Keeley (1994, cited Stapleton, 2001, p.544)

The levels of argumentation are perhaps best illustrated via reference to the writing on

legalization of prostitution Although it may appear obvious that arguments such as “cause

other social evils and serious diseases”, “degrade the human society”, and “threaten the women‟s lives” occupy three different levels, some model is needed to determine in what

way their depth varies As Stapleton (2001) argued, ―although depth of thoughts cannot be

objectively measured, if arguments can be classified into hierarchical levels of abstraction some comparatively conclusions might be reach‖ (p.525) Then he presented the famous

model of Maslow‘s hierarchy of needs which depicts the way in which humans behave

with respect to their needs The model has five levels, beginning with ―physical needs‖ at the base and advancing up through four more stages of needs: ―safety,‖ ―social,‖ ―esteem,‖ and ―self-realization.‖ ―Physical needs‖ include food and other immediate survival basics,

―safety needs‖ include avoiding pain and injury, ―social needs‖ refer to companionship and love, ―esteem needs‖ encompass attributes such as responsibility and self-respect, and

―realization needs‖ include such abstract notions as independence, creativity, and

self-expression According to Maslow, each more basic need must first be satisfied before reaching the needs higher up This model allows us to assess depth of thoughts In the case

of the three examples given above, “cause other social evils and serious diseases”,

“degrade the human society”, and “threaten the women‟s lives”, the first clearly fits into

Maslow‘s second level, safety needs, the second and the last arguments best match the

social needs level

The writings on the familiar topic of compensation for smokers can also be analyzed using Maslow‘s model Concerns about the harm of smoking to the first-hand and second-hand smokers appear to be related to safety as well as social needs Other arguments that appeared in this set of writings referred to notions of legal right of the tobacco companies and individual smoker‘s responsibility, which appear to most closely matched with social and esteem needs, respectively

Results from the unfamiliar topic writings show similarities to the pattern in the familiar topic essays Participants supported their opinions with reasons and evidence, although there were significantly fewer arguments and less evidence There were also large differences in the relative quality of the arguments Specifically, the variety of responses and the presented knowledge in the legalization of prostitution samples were shallower than in the ones on compensation for smokers Those who disagreed with the prompt wrote almost exclusively about how prostitution facilitates social evils or diseases, whereas the few who agreed spoke largely of the financial benefits Both arguments fall into Maslow‘s second lowest level in the hierarchy of needs model

With those remarks on the depth of knowledge in both sets of writings, it can still be seen clearly that the writing samples did demonstrate a fundamental understanding that opinions require support Participants working on both topics included a variety of types of evidence, showing an understanding that reasons need to be backed up with proof of some sort The broad variety of evidence used in the familiar topic essays, including experience, explanation, facts, consequences, citations, and research studies They are all acceptable evidence types in critical thinking texts, suggests some intuitive understanding of what constitutes justifiable proof Again, the volume and variety of different pieces of evidence used both to agree and to disagree with the prompt (Appendix B2) indicate that participants were not simply summoning up well-rehearsed responses that they had used in the past or had heard used by others For unfamiliar topic, participants also supplied several types of

evidence, similar to the essays written for the familiar topic, although most of this evidence was focused at the safety needs level, that is, concern that legalization of prostitution causes more social evils and diseases

It is true that some of the arguments and evidence used are widely discussed in schools and the media However, the diversity of levels of reasoning from practical to abstract again suggests an element, which articulate the awareness of argumentation among the participants, that is, ―a voice that individuates a writer from all other writers‖ (Elbow,

as cited in Stapleton, 2001) This is especially so in a writing assignment for which there is

a limit on the number of words and put under time pressure Although this remains a concern, the results point in a different direction When writing about legalization of prostitution, participants identified two distinct opposing viewpoints, while participants working on the other topic failed to identify any Refutations, although limited to only few

of the participants, sometimes displayed considerable sophistication or inside knowledge

4.3.3 Types of fallacies and the reflection of the influence of familiarity on the quality of students’ critical thought

Fallacies of several different types were found in roughly one third of the participants‘ essays on both topics All of these could be categorized into the conventional types such as

irrelevance, hasty generalization, oversimplication, slippery slope, false analogy and straw man Again, the existence of these fallacies in recognizable forms suggests some similarity

with what might be expected from first language students The greater numbers of fallacies

in the unfamiliar topic writings with the typical types may be explained by the insufficient background knowledge they have due to the topic unfamiliarity

Although a large amount of fallacies was found in the participants‘ writings of both topics

in this study, it is important to note that that they are also common in the writing of native speaking students Raimes and Zamel (1997, p.80, as cited in Stapleton, 2001, p.528)

asked, ―who are these L1 students who have a relatively easier time in writing classes?

They are certainly not the students who populate the composition courses at public, urban institutes in the U.S where we teach‖ In short, it can be said that not only Vietnamese or

Asian students, but many other second language learners also have those problems in generating and formulating ideas into sound, cogent arguments

4.3.4 Other remarks

While supplying arguments to support one‘s reasoning is relatively easy, finding solid supporting evidence requires research of proper resources, especially if one is writing an argument at tertiary level Considering the pattern established by the number, variety, and depth of arguments, the evidence supplied for the familiar topic was considerably greater than that of the unfamiliar It was explained that, by being more frequently exposed to the mass media with many discussions raised by both Vietnamese and non-Vietnamese, and being close to the existing facts and figures, participants were able to supply evidence from

a wider variety of sources for the discussed issue of compensation for smokers Thus they could cover many problematic aspects with fewer fallacies, from the personal danger to social suffer, from the individual responsibility to the economic rule of supply and demand, etc, all of those they have read, heard, and even deeply discussed with friends or classmates They were well aware of the dangerous impacts of smoking, the stand they took when arguing and had better understanding of many aspects related to the discussing issue Remarkably, a majority of students disagree with the compensation This can be said to reflect the common Vietnamese traditional attitude and practice Unlike the Western consumerism attitude concerned about consumers‘ rights and benefits, most Vietnamese still hesitated when it comes to ask producers or sellers for compensation However, some individual did go further than the old way of thinking to address the consumerism attitude and articulate their rights by agreeing with the statement

On the other hand, participants writing on the unfamiliar topic focus on few social impacts

of legalization of prostitution with repetitive or fallacious reasoning All the arguments

about financial benefits or the widespread of social evils and diseases they provided apparently extracted from foreign information sources of online articles or websites, which supplied the reasoning with third-party analysis When they did not understand it clearly and deeply, their reproduced arguments will be weak or shallow As the rater further

pointed out, ―some writers failed to write a topic sentence that adequately convey an

argument Others, instead, started another point in the same paragraph without even having finished the point given in the topic sentence, and then attempted to finish it‟