SLIDE PRESENTATION EXPERIMENTAL RESEARCH

Method of the research EXPERIMENTAL RESEARCH Supervisor Ms Nguyễn Thị Minh Huệ Group 2 K18C Dương Thị Tâm Nguyễn Thị Tâm Trần Thị Thu Trang (1986) Outline of the presentation Overview of Experimental Research Analysis of an Experimental Research sample Conclusion 1 Overview of Experimental Research 1 1 What is Experimental Research? 1 1 2 Types of Experimental Research 2 1 3 Components of an Experimental Research 3 1 4 Some noticeable statistical concepts and terms 4 1 Overview of Experimental R.

EXPERIMENTAL RESEARCH

Supervisor : Ms Nguyễn Thị Minh Huệ

Group 2 K18C:

Dương Thị Tâm Nguyễn Thị Tâm Trần Thị Thu Trang (1986)

Outline of the presentation

1 Overview of Experimental Research

2 Analysis of an Experimental Research

sample

3 Conclusion

1 Overview of Experimental Research

1.1 What is Experimental Research?

1 Overview of Experimental Research

1.1 What is experimental research?

 An experimental research is:

 a controlled look at nature

(McDonough J.& McDonough S., 1997)

 an attempt to maintain over all factors that may affect the

result of an experiment (James P., 1997)

 the effects of specified and controlled treatments given to

subjects usually formed into groups.

(Selinger & Shohamy, 2000)

 Experimental research is carried out to explore the strength of

relationships between variables (Nunan, 1992)

1.2 Types of Experimental Research

Mean:

- The average of a set of scores.

- The central tendency of the scores.

 Standard deviation (SD) (Nunan: 1992)

- The most important measure of dispersion

- The extend to which a set of the scores varies in relation to mean

- Calculating :

1) Deducting the mean from each individual score

2) Squaring the resulting figures to get rid of minus signs

3) Adding these together and dividing by the total scores minus one → Variance

4) Obtaining the square root of the figure → SD

1.4 Some noticeable statistical concepts

and terms

Score Score - Mean Squares of

Variance = 10 / (5 – 1)

=2.5

= 10

SD = √2.5 = 1.58

 Standard error:

- The standard deviation of sample means: placing a single

sample mean in relation to the population mean.

- Calculating: Dividing SD of a single sample by the square root of the number of observations in the sample (N)

SE = SD : √ N

 T- test: the test to compare two means (2 groups).

 Control group: the group under normal situation

 Experimental group: the group under experimental

condition.

1.4 Some noticeable statistical concepts

and terms

Outline of the presentation

1 Overview of Experimental Research

2 Analysis of an Experimental Research

sample

3 Conclusion

Research report sample

• Title: A Comparison of the Effects of two

Vocabulary Teaching Techniques

• Author: Saeed Mehrpour, an assistant professor

of TEFL at the Department of Foreign

Languages and Linguistics of Shiraz University, Iran

• Source: The Asian EFL Journal Quarterly, June

2008, Volume 10, Issue 2

2 Analysis of the research report sample

2.1 Introduction

- Is the framework for the study clear?

Yes It follows the theoretical framework as the author mentioned varied studies of other researchers on the learning/teaching

vocabulary techniques

Literature review

* Can you tell me where the study fits in?

1 Is the background/ rationale provided?

(for odd-numbered group )

2 Is the relationship to previous research

clear? (for even-numbered group )

Is the background/ rationale provided?

Yes.

 Point out the importance of vocabulary learning/ teaching by mentioning research studies of other researchers => one of the reasons why the researcher is interested in the topic.

 Mention 3 approaches to learning/ teaching vocabulary and studies dealing with the effect of direct and indirect approaches on learning/ teaching vocabulary.

 No consensus of opinions : whether the direct approach is

better than the indirect approach or not

 The necessity of further research in this area

Is the relationship to previous research

clear?

Yes.

 Mention literature review of many researchers which are related to the study.

 A good demonstration of how previous research related to the study.

 The majority of the research findings mentioned in the literature review, e.g., Laufer and Shmueli (1997), Qian (1996), Prince (1996), Critchley (1998), and Hulstijn (2001) support the superiority of decontextualizing vocabulary learning techniques (e.g., using bilingual word lists) over contextualized techniques (e.g., learning words in context)

 Only in one study, (Khuwaileh, 1995), the results supported the use of contextualizing techniques for vocabulary learning.

Statement of purpose

1 What is the purpose of the study?

2 What is research question?

What is the purpose of the study?

To compare the impacts of two vocabulary teaching

techniques (sentence-making practice, i.e., a type of

speaking and writing practice on vocabulary items

(a contextualizing technique) and rote

memorization of word lists (a decontextualizing

technique) ) on vocabulary learning of a low

proficiency group of Iranian learners of English as a

foreign language.

What is research question?

Is sentence-making practice (as a contextualizing

technique) superior to rote memorization of

word lists (as a decontextualizing technique)?

2 Analysis of the research report sample

2.2 Method of the research

Method of

the research

Subjects Materials Procedures Analyses

1 What is the description of the

participants in the study?

2 Is the description of the participants

adequate?

3 What is the method of selection?

4 Is the method of selection clear?

What is the description of the participants

Is the description of the participants

adequate?

• In general, the researcher provides us with

enough information about the participants

• Participants are divided into 2 groups:

control group (24) and experimental group (26)

• However, there is no reason for choosing

only male participants although it is Iran language Institute

What is the method of selection?

• 2 classes were randomly chosen from 11 classes

• The author did not mention the random sampling procedure (simple or

convenient)

1 What materials are used in the research?

2 Is there a description of the materials?

3 What are dependent variables and

independent variables?Do the variables

represent the definitions of underlying

constructs?

4 Is there any description of any equipment?

Description of materials

Vocabulary

test 1

-Pretest – to see if the participants

were homogenous enough or not

- Posttest – to measure the difference

between the achievement of two

group

Including vocabulary items in the first ten units of the textbook

Comments on materials

• Reliability is estimated by KR - 21 method

• Validity is criterion-related validity (correlation between researcher’s subjective score and posttest score)

What are dependent variables and

independent variables?

• Dependent variables : score of student’s vocabulary tests

• Independent variables : two techniques

of teaching/learning vocabulary – rote

memorization (decontextualizing) and

sentence-making (contextualizing)

Do the variables represent the

definitions of underlying constructs ?

• Underlying constructs: cannot be

observed (Intelligence, motivation,

proficiency, competence).

• In the article: Vocabulary competence

 Yes (Scores reflects learners’

vocabulary competence)

Is there any description of any

equipment?

No equipment is utilized.

Is there a description of the preparation of

materials, administration and scoring?

Is there a description of the preparation

of materials?

• The researcher prepares two tests by

selecting representative samples of

vocabulary items extracted from the first

10 units.

Is there a description of administration

and scoring?

• Administration:

Test 1 was administered to both groups at the beginning (in second session) and the end of the experiment

Test 2 was delivered to both groups in

session 9 (3 sessions before midterm exam)

• Scoring: sentence-making test was scored by

two independent raters  increase reliability

1 Is there a description of the

arrangement and grouping of the data?

2 Are the statistical tests listed in

order of use?

A description of the arrangement and

grouping of the data

• The arrangement of the data was in line with the pretest and posttest

• The data was grouped into 2

categories: the results of each test in

groups in each test

Are the statistical tests listed in order of

use?

• The statistical tests are listed in order

Pretest  test 2  posttest

Descriptive statistics

Independent T-test of comparing between 2

groups

Is the study replicable?

• Replicability refers to research procedure

• In this study, we can see that the researcher

stated unclearly about the sampling procedure, duration and the tests…

 The replicability of the study is not very

convenient for other researchers to redo

2 Analysis of the research report sample

1 Are all the statistical tests previously

listed presented as results?

2 Is there a prose explanation?

Are all the statistical tests previously listed

presented as results?

• The results of all these tests were clearly presented in

the results and discussion part

“With respect to the results obtained from the analysis

of data pertaining to the pre and post-test, one can

conclude that although the difference between the

means of the treatment group and the control group was not statistically significant, the treatment given to the treatment group had affected this group to some extent ”

Is there a prose explanation?

• Yes

• The author explained, interpreted and analysed the Tables by words.

2 Analysis of the research report sample

Discussion/ conclusion

1 Is the original research question answered?

2 Is there an explanation of why the results

were as they were?

3 If the conclusion is based on the previous

research, is it well-supported and reasoned?

4 Are the suggestions for further research

provided?

Is the original research question

answered?

• Yes

Is sentence-making practice

(as a contextualizing technique)

superior to rote memorization of

word lists (as a

learners at the beginning levels

of language instruction.

Is there an explanation of why the results

were as they were?

• Yes There are some reasons to explain for the lower performance of the control group

 They could not recall the meanings of the words which is a basic requirement for making sentences using such words.

 Students usually copy sentences from dictionaries when they are assigned to make sentences based on vocabulary items

 They do not actually make sentences to have real practice in sentence-making (crowdedness of class and teacher’s

ignorance)

If the conclusion is based on the previous

research, is it well-supported and reasoned?

• The conclusion is well-supported and reasoned Firstly, the

researcher mentioned some research findings in the literature review part Then, in the conclusion part, he proved his

research results in agreement with some of them

“The results of this study are in line with that of Qian (1996) who suggests that for some language learners

decontextualized L2 vocabulary learning with feedback is

more effective than contextualized vocabulary learning

without feedback ”

Are the suggestions for further research

provided?

There is no suggestion for further study

2 Analysis of the research report sample

References, notes, footnotes, appendices

1 Are all the references cited in the text

Are all the references cited in the text

• Are any pertinent references missing ?

Outline of the presentation

1 Overview of Experimental Research

2 Analysis of an Experimental

Research sample

3 Conclusion

3 Conclusion

• The true- experiment and its variants, the pre- experiment and quasi- experiment are

important tools in language study, and they

have added significantly to our knowledge of language learning, teaching and use

• Experiments are designed to collect data in

such way that threats to the reliability and

validity of the research are minimised

3 Conclusion

With totorial 2, we could:

- Interpret the Experimental research (subjects, materials, procedures, critical points…)

- Analyze the Research sample

- For further purpose, conduct an experimental research

• McDonough, J., & McDonough, S (1997) Research methods for English

language teachers London: Arnold

• Nunan (1992/CUP/), Research Methods in Language Learning, p24-50,

Cambridge

• Saeed M., A Comparison of the Effects of two Vocabulary Teaching

Techniques, The Asian EFL Journal Quarterly, June 2008, Volume 10, Issue

2

• Selinger, H.W., Shohamy, E (1989) Second Language Research Methods,

Oxford University Press.