cnm-handbook-for-educational-research

Table of ContentsPurpose of this Handbook...1 IRB and Educational Research...1 How the IRB Came to Be...1 The Role of the IRB...2 Research Involving Human Subjects...2 IRB Exemptions...3

CNM HANDBOOK FOR EDUCATIONAL

RESEARCH

Central New Mexico Community College

Ursula Waln, Director of Student Learning Assessment, and Fred Ream, Mathematics Instructor

2014

Table of Contents

Purpose of this Handbook 1

IRB and Educational Research 1

How the IRB Came to Be 1

The Role of the IRB 2

Research Involving Human Subjects 2

IRB Exemptions 3

IRB Criteria 4

Informed Consent 5

Research Design 6

Quantitative and Qualitative Options 6

Sampling Methods 9

Statistics 10

Data Types 10

Analytic Approaches 10

Figure 1: Statistical Test Decision-Making Flowchart 11

Refresher of Basic Concepts 12

Figure 2: Normal Distribution 14

Figure 3: Illustration of Outliers 15

Analytic Software Programs 16

Preliminary Data Analysis Tips 16

Glossary 17

References 19

Appendix A: Recommendations from CNM’s IRB 20

Appendix B: Preview of CNM IRB Application Form 21

PURPOSE OF THIS HANDBOOK

This handbook was developed to foster depth of intentionality and critical analysis in CNM student outcomes

assessment It builds on the information presented in the CNM Handbook for Outcomes Assessment and is based

on the premises that 1) student outcomes assessment is a form of educational research, and 2) considering one’sassessment efforts as research may suggest more creative, purposeful, and meaningful approaches Thefollowing pages offer information and tips for those who wish to develop a greater degree of sophistication intheir assessment approach and/or have their assessment efforts render more useful information

IRB AND EDUCATIONAL RESEARCH

If you are interested in conducting research that is not purely operational in nature, is experimental, is for adissertation or master’s thesis, or is going to be used outside of CNM in any way, your project requires priorapproval by the CNM Institutional Review Board (IRB) If you think there is any possibility at all that you willever want to publish your findings, use the data for other publishable research, or do anything else with yourfindings outside of CNM, please get IRB approval before beginning any data collection The question ofwhether your particular study must have IRB approval hinges on whether or not it meets the definition of

‘research involving human subjects.’ However, for any study that is not operational in nature and going to be

used only within CNM, the IRB, not the researcher, must make this determination, based on an application If

you have any doubt at all, submit an application The process itself can help you organize your research plans,

so the time won’t be wasted, and you’ll know you covered your bases

A requirement for IRB approval need not deter faculty from pursuing educational research projects Indeed,structured research into student learning not only has great potential for informing internal improvement efforts,but also can produce insights with the potential to impact the broader profession and/or instructionalcommunity The application form for submitting a project for IRB approval is shown in Appendix B and can beaccessed at http://www.cnm.edu/depts/planning/instres/irb

If you are absolutely certain your project does not require IRB approval and you already know all you need toknow about the IRB, you may want to skip ahead to the research design section, page 11

How the IRB Came to Be

The existence of IRBs for certification of research conducted at colleges and university is required under TheNational Research Act of 1974 Although certainly not the first example of a societal effort to regulateexperimentation involving humans, this federal statute came about in response to medical research that violatedethical principles deemed fundamental to American values and basic human rights Salient examples include themedical experiments conducted in Nazi concentration camps between 1939 and 1945, (which led to theestablishment of the first international code of research ethics, the Nuremburg Code) and the U.S Public HealthService’s Tuskegee Syphilis Study of 1932-1972, in which 400 African American men known to have syphilishad information and treatment withheld so that researchers could observe the progression of the disease(Schneider)

In 1974, Congress established the National Commission for the Protection of Human Subjects of Biomedicaland Behavioral Research and tasked it with identifying ethical principles and developing guidelines for theconduct of research involving human subjects In 1979, the Commission published the Belmont Report, whichsummarized three basic principles:

1 Respect for persons

o Individuals should be treated as autonomous agents

o Persons with diminished autonomy are entitled to additional protections

The Belmont Report principles and guidelines were codified in the Title 45 Code of Federal Regulations, Part

46, and oversight was given to the U.S Department of Health, Education and Welfare (which later split into theDepartment of Health and Human Services and the Department of Education)

In 1991, The U.S Department of Education became one of fifteen federal departments and agencies to codify inseparate regulations the “Common Rule,” including in its 34 CFR Part 97 language identical to 45 CFR 46,Subpart A (NIH) Today, seventeen federal agencies fall under the Common Rule, and oversight for all isprovided by the Office for Human Research Protections, a division of the U.S Department of Health andHuman Services

The Role of the IRB

At the institutional level, “The IRB is an administrative body established to protect the rights and welfare ofhuman research subjects recruited to participate in research activities conducted under the auspices of the

institution with which it is affiliated” (IRB Guidebook, p 1) IRBs determine “the acceptability of proposed

research in terms of institutional commitments and regulations, applicable law, and standards of professionalconduct and practice” (45 CFR 46.107)

The major roles of IRBs in the oversight of research are:

1 Evaluating the respect for persons, beneficence, and justice of all research activities proposed under theauspices of the institution and providing approval or disapproval accordingly

2 Ensuring that the process proposed to collect informed consent meets regulatory requirements

3 Overseeing the progress and protocols of ongoing research studies

NIH, p 84

Research Involving Human Subjects

The CFR provides the following definitions:

…(d) Research means a systematic investigation, including research development, testing and evaluation,

designed to develop or contribute to generalizable knowledge Activities which meet this definitionconstitute research for purposes of this policy, whether or not they are conducted or supported under aprogram which is considered research for other purposes For example, some demonstration and serviceprograms may include research activities

(e) Research subject to regulation, and similar terms are intended to encompass those research activities

for which a federal department or agency has specific responsibility for regulating as a research

Administration) It does not include research activities which are incidentally regulated by a federaldepartment or agency solely as part of the department's or agency's broader responsibility to regulatecertain types of activities whether research or non-research in nature (for example, Wage and Hourrequirements administered by the Department of Labor).

(f) Human subject means a living individual about whom an investigator (whether professional or student)

conducting research obtains

(1) Data through intervention or interaction with the individual, or

(2) Identifiable private information

Intervention includes both physical procedures by which data are gathered (for example,

venipuncture) and manipulations of the subject or the subject's environment that are performedfor research purposes

Interaction includes communication or interpersonal contact between investigator and subject Private information includes information about behavior that occurs in a context in which an

individual can reasonably expect that no observation or recording is taking place, andinformation which has been provided for specific purposes by an individual and which theindividual can reasonably expect will not be made public (for example, a medical record).Private information must be individually identifiable (i.e., the identity of the subject is or mayreadily be ascertained by the investigator or associated with the information) in order forobtaining the information to constitute research involving human subjects…

45 CFR 46.102

IRB Exemptions

The CNM IRB, with endorsement from the VPAA, Deans’ Council, Chairs’ Council, and Faculty Senate,documented its Recommendations for Research in 2012, stating that “Some types of research are exempt fromIRB review: student projects for class, data collection (e.g., surveys) by faculty or staff, assessment datacollection, student course evaluations, and research by the college that is not designed for publication”(Appendix A)

Whether or not an investigation is subject to IRB approval, the CNM IRB (on its homepage) recommendsfollowing the basic rules listed in its document Recommendations for Research In the same vein, it is advisablefor anyone planning educational research at CNM to be familiar with and follow the IRB criteria and informedconsent guidelines in the following sections of this handbook, whether or not the research requires IRBapproval

Forming the basis for the CNM statements of exemption are the following excerpts from 45 CFR Part 46, andthe corresponding educational regulations in 34 CFR Part 97 (formatting enhanced for readability)

…(b) Unless otherwise required by department or agency heads, research activities in which the only

involvement of human subjects will be in one or more of the following categories are exempt from thispolicy:

(1) Research conducted in established or commonly accepted educational settings, involving normaleducational practices, such as

(i) research on regular and special education instructional strategies, or

(ii) research on the effectiveness of or the comparison among instructional techniques, curricula, orclassroom management methods

(2) Research involving the use of educational tests (cognitive, diagnostic, aptitude, achievement),survey procedures, interview procedures or observation of public behavior, unless:

(i) information obtained is recorded in such a manner that human subjects can be identified,directly or through identifiers linked to the subjects; and

(ii) any disclosure of the human subjects' responses outside the research could reasonably place thesubjects at risk of criminal or civil liability or be damaging to the subjects' financial standing,employability, or reputation

(3) Research involving the use of educational tests (cognitive, diagnostic, aptitude, achievement),survey procedures, interview procedures, or observation of public behavior that is not exempt underparagraph (b)(2) of this section, if:

(i) the human subjects are elected or appointed public officials or candidates for public office; or(ii) federal statute(s) require(s) without exception that the confidentiality of the personallyidentifiable information will be maintained throughout the research and thereafter

(4) Research involving the collection or study of existing data, documents, records, pathologicalspecimens, or diagnostic specimens, if these sources are publicly available or if the information isrecorded by the investigator in such a manner that subjects cannot be identified, directly or throughidentifiers linked to the subjects…

For the CNM IRB Recommendations for Research, see Appendix A or go to

https://www.cnm.edu/depts/planning/instres/irb/documents/RECOMMENDATIONS-FOR-RESEARCH.pdf

IRB Criteria

The following CFR excerpts hit the highlights of the federal criteria for IRB approval:

1. Risks to human subjects are minimized…

2. Risks to human subjects are reasonable in relation to anticipated benefits, if any, to human subjects andthe importance of the knowledge that may reasonably be expected to result…

3. Selection of human subjects is equitable…

4. Informed consent will be sought from each prospective research participant or the prospective researchparticipant’s legally authorized representative in accordance with and to the extent required by §46.116

5. Informed consent will be appropriately documented in accordance with and to the extent required by

And, the CNM IRB web site and links to documents can be found at

http://www.cnm.edu/depts/planning/instres/irb

Informed Consent

The principle of respect for persons requires that investigators conducting research on human subjects obtain

informed consent in the form of “[a] legally-effective, voluntary agreement that is given by a prospectiveresearch participant following comprehension and consideration of all relevant information pertinent to thedecision to participate in a study” (NIH, p 117)

Autonomous adults can provide their own informed consent (The age of majority in New Mexico is eighteen.)However, research involving children or other vulnerable populations requires informed consent from not onlythe subject (whenever the individual is reasonably capable of providing or denying consent), but also thesubject’s legally authorized representative In some cases (e.g., specific cultural settings), community or familyconsent may also be required

Informed consent consists of voluntariness, comprehension, and disclosure:

Voluntariness: Consent must be given free of coercion, and the perceived value of any inducements

must be small enough to avoid undue influence on the decision to participate Care must be taken toensure that subjects are not manipulated to ignore the risks of participation either through fear ofrepercussions for refusal or through motivation by excessive or inappropriate rewards Offering bonuspoints toward a final course grade would be an example of an inappropriate incentive However, in somecases, minimal compensation for participants’ time and effort may be appropriate

Comprehension: To provide informed consent, subjects must first understand the information presented

to them sufficiently to make a reasoned, informed decision Subjects must therefore have the cognitivecapacity to understand (or else be represented by legally authorized individuals), and the informationmust be presented using language and methods of delivery appropriate to the people making thedecision

Disclosure: Information that must be disclosed for participants to be able to provide informed consent

include the purpose, risks, and benefits of the research; alternatives to the research protocol; the extent towhich confidentiality can be protected; compensation in case of injury; contact information for questions

or concerns; and conditions of participation, including the right to refuse or withdraw participationwithout penalty (Compensation may be terminated proportionately upon withdrawal, but any offer ofcompensation for participation prior to the time of withdrawal must be honored Telling prospectivesubjects they must complete the study or forfeit all compensation would be an example of a penalty.)Waivers regarding the disclosure of specific information may be considered by the IRB when the research poses

no more than minimal risk to the participants, withholding the information will not adversely affect the rights orwelfare of the participants, and the research could not practicably be carried out otherwise When information iswithheld, it should be disclosed following the conclusion of research unless doing so might cause harm

“No informed consent, whether oral or written, may include any exculpatory language through which thesubject or the representative is made to waive or appear to waive any of the subject's legal rights, or releases orappears to release the investigator, the sponsor, the institution or its agents from liability for negligence” (45CFR 46.116)

Finally, informed consent should be viewed as an ongoing process, not just a one-time event Participantsshould be kept informed and given opportunities to ask questions as the research is carried out

RESEARCH DESIGN

For the purposes of CNM student learning assessment, the primary consideration in designing your researchshould be what you want to know related to the development of your program competencies (student learningoutcome statements) Using that as your starting point, you can employ ‘reverse design’ principles to map outthe steps you will take toward gaining the insights you seek Depending on the type of information you want tocollect and what you want to do with it, your research design may or may not include the following:

 A written statement of the issue to be investigated

 Identification of the population to be studied

 A description of relevant background information

o Procedures for administration

o Data analysis methods

 A timeline

Quantitative and Qualitative Options

Research approaches are often described as quantitative or qualitative, though the two descriptors more

accurately represent ends of a continuum than distinct categories Both extremes seek to answer questions usingsystematic procedures, collecting evidence, and examining findings However, where a research project falls onthe continuum is determined by its objective(s), the types of research questions posed, the methodologiesemployed, and the types of data produced Expectations for researchers differ as well

Quantitative research tends to have as an objective obtaining generalizable findings Questions posed tend tofocus on associations between variables, methodologies employed tend to involve studying specific variables,and data produced tends to be numerical Quantitative investigators are typically expected to adhere toscientifically validated research models and techniques and to remain objective

Qualitative research, on the other hand, tends to have as an objective exploring the dynamics of a specificcontext Questions posed tend to focus on understanding factors that influence outcomes, methodologiesemployed tend to be open-ended and exploratory, and data produced tends to be textual (recorded observations,participant comments, etc.) Qualitative investigators are typically considered free to adapt their researchmodels to fit the changing circumstances of their investigations and to render subjective interpretations

Randomized controlled trials represent the quantitative end of the spectrum In a nutshell, such investigations

provide an intervention with a randomly selected experimental group and compare the results to those of acontrol group (a group that does not receive the intervention) Here are some grouping models commonly used

in randomized controlled trials:

 Parallel grouping: Each student is randomly assigned to

an experimental or control group; the experimental

group receives the intervention and the control group

does not – but may receive a dummy/placebo

intervention, to minimize the potential for participant

expectations to influence the outcomes (In educational

research, a placebo may be, for example, an

instructional method that has already been shown to not

produce the effect being researched.)

 Cluster grouping: Pre-existing groups of students (such

as course sections) are randomly selected to receive or

not receive the intervention

 Crossover grouping: Each student receives or does not

receive the intervention in a random sequence

 Factorial grouping: Used when more than one

intervention is tested simultaneously to determine

whether there is a compound effect; each student is

randomly assigned a group that receives a particular

combination of interventions (AA, AB, AC, & BC,

where A = no intervention, B = intervention 1, and C =

Intervention 2)

A common crossover approach is to switch the treatment and

non-treatment groups at the midpoint of the study Crossover

grouping minimizes the potential for group differences to

influence the findings In educational research, the effectiveness

of a cross-over approach depends in part upon whether the

intervention being studied is influenced by timing

For example, let’s say a foreign language instructor randomly selects half of her students to participate in anemersion program during the first half of the semester only and then has the other half of the studentsparticipate in the emersion program during the second half of the semester The timing of the emersion programcould turn out to be a significant factor Students who develop basic vocabulary and syntactic awarenessthrough classroom instruction during the first eight weeks may have a decoding advantage when they enter theemersion program On the other hand, students who experience the emersion program during the first eightweeks may develop greater awareness of cadence, pronunciation, and speech patterns, and applying these,advance more effectively through the conventional instruction during the second eight weeks Anticipating thesepossibilities, the instructor may decide to include in the design a third group, comprised of students who do notparticipate in the emersion program at all

A subset of randomized controlled trials, called randomized double-blind placebo-controlled crossover studies,

is generally considered the ‘gold standard’ in quantitative scientific research (particularly pharmaceuticalresearch) And, while it need not be the goal of every educational researcher to conduct such a study, anunderstanding of the rationale supporting the design will help inform design decisions

Double-blind means neither the researchers in contact with the subjects nor the subjects know who is receiving

the ‘treatment’ and who is not This minimizes the potential for either the researchers or the subjects toinfluence outcomes based on their expectations In educational research, conducing a double-blind study wouldrequire that the educational intervention be provided by someone other than the investigator

A Source of Some Confusion

The word quantitative may sometimes seem a

little slippery because people use it differently depending on whether they are describing data, measurement methods, or research design

Quantitative data is information derived from

direct numerical measurement Height, weight, area, pounds per square inch, counts of students, proportions of responses, and GPA are examples Representing something as a number when it is not directly, numerically measurable (such as pain rated on a scale of 1 to 10 or level of effort on a scale of 0 to 4) does not make it quantitative data.

It is a common error to refer to qualitative data that has been numerically represented as

quantitative measurement methods are those that

involve direct, numerical measures However, common usage lumps together all measurement methods that produce numerical data (especially when conducted in systematic ways) as

quantitative measurement methods As a result,

Likert-scale questionnaire items and rubric-based evaluations are commonly referred to as quantitative methods even though the data they comprise is qualitative data.

At the research design level, quantitative

essentially means the researchers intend to use inferential statistics to make generalizations

For example, let’s say a researcher wants to find out whether students approached outside of class by a peer andasked to explain core course concepts perform significantly better on subsequent exams than students notquestioned in this way To make this a double-blind study, the students must have no inkling that beingquestioned by a peer is an intervention, and the researcher must remain ignorant regarding which students arequestioned until after the data collection is complete.

At the other end of the spectrum, qualitative research usually seeks to answer a question related to the

perspectives of an involved population Qualitative research can be particularly useful in studying attitudes,values, opinions, perceptions, beliefs, behaviors, emotions, relationships, social situations, experiences, andother aspects of individual, community, or cultural contexts Although insights gained may have the potentialfor extension to similar populations or situations, qualitative research usually focuses more on delving into thefactors that influence outcomes within the specific study population than on being able to generalize thefindings to other populations Qualitative research explores and interprets how people experience the complexreality of given issues and/or situations

For example, imagine CNM researchers want to better understand why area employers within a specific sector

do not hire as many qualified CNM alumni as might be expected on the basis of job openings The objective oftheir investigation will be to identify factors influencing the decisions of the local employers Their findingsmight ultimately reveal something about employer perceptions that could be useful, by extension, to other CNMprograms or other colleges with similar programs However, their primary goal is not finding something thatcan help other programs, but rather, better understanding what their students’ prospective employers want sothat the program faculty might better address the preparation of candidates

Research within the field of education is often driven by context-specific questions posed by faculty andtherefore tends toward qualitative design Indeed, large-scale quantitative educational studies are relatively rare,and generalizations made on their basis tend to be subject to dismissal because of the difficulties inherent inapplying scientific models to studies of humans that take place in natural settings, especially when theconclusions contradict strongly-held convictions about teaching and learning (For an illustrative example,consider Project Follow Through, “the largest, most expensive educational experiment ever conducted”[Adams, 1996].)

Because investigators conducting qualitative research typically care more about the usefulness of theinformation gathered in supporting interpretation and insight than about the reproducibility of the study,qualitative research tends to be characterized by adjustability and/or fluidity The design needs to facilitatecollection of the desired information, not stand up to scientific scrutiny Stated another way, validity of thefindings is important in qualitative research, but reliability is not

Observation, interviews, and focus groups are methods commonly used in qualitative research The dataproduced may be a mix of direct and indirect measurements and/or textual information with no associatednumeric values (such as recorded dialogs, comments, field notes, responses to open-ended questions, etc.) Inputusing categorization, holistic ratings, Likert scales, rubrics, and/or forced-choice responses may be numericallyrepresented to facilitate statistical analysis

Note that the use of statistics is not the exclusive domain of quantitative research; however, while quantitativeresearch tends to employ inferential statistics, qualitative research tends to incorporate descriptive statistics.These categories are explored further in the Statistics section of this handbook

Sampling Methods

A population is all of a group of interest (in New Mexico, the whole enchilada) A sample is a subset of the

population being studied (in New Mexico, some of the enchilada) Sampling can make it feasible to implementresearch methods that would otherwise be impossible or too time consuming to consider

Sampling methods fall into two broad categories: those that offer the potential to support inferences about theentire population (probability methods) and those that do not (non-probability methods) Probability methodsare typically used with quantitative research designs and inferential statistics; whereas, non-probability methodsare more often (but not always) used with qualitative designs and descriptive statistics

If you want to be relatively sure that the results you obtain from your sampled students are generalizable to thepopulation of students in your course and/or program, consider using a probability method, such as simple,systematic, stratified, or clustered random sampling Below are some examples:

 Simple random sample: Drawing names from a hat (For the sample to be truly random, all of the

possible names must be in the hat to begin with If you systematically leave out any group of people[e.g., students who are failing the course], then your results will not be generalizable to that group.)

 Systematic random sample: Counting off every 5th student who comes through the door

 Stratified sample: Randomly selecting 10 first-year students, 10 second-year students, and 10 recent

graduates

 Cluster sample: Randomly selecting 6 students from each of five sections of the same course

To be confident in your inferences, you will want to select a sample that is big enough to be representative ofthe population you are studying Consider using an online sample-size calculator, such as the one provided byCreative Research Systems at http://www.surveysystem.com/sscalc.htm, to find out how many students youneed to include in your sample to achieve the level of statistical reliability you want Then, you can use anonline random integer generator, such as the one at http://www.random.org/integers/, to get numbers that youcan apply to a numbered list of all students in the research ‘population’ to create your random sample

On the other hand, if you don’t care about generalizing your sample’s results to a larger population, you mightprefer to use non-probability methods such as convenience, purposive, snowball, quota, or theoretical sampling.Just keep in mind that if you choose any of these, the statistics you obtain will not support claims that yourfindings can be generalized to the entire population Following are some examples:

 Convenience sample: Using your own class as a sample

 Purposive sample: Asking all of the African American students in your program to participate in a focus

group to provide feedback regarding their perceptions of cultural inclusiveness within the program

 Snowball sample: Interviewing a few people and asking them to refer their friends to you for interviews

 Quota sampling: Selecting 200 students in such a way that your sample has essentially the same

proportionate representation of ethnic groups seen in your program

 Theoretical sample: Questioning a few students about something you’re considering studying further

Data Types

Although as a branch of mathematics statistics involves numbers, many types of information can used forpurposes of statistical analysis Before selecting a method of analysis, it is important to understand what type ofdata you have And, considering the method of analysis prior to conducting a study can help to ensure that the

data collected will provide the desired functionality There are four main types of data: nominal, ordinal, interval, and ratio:

 Nominal: Numbers or words are used to label, classify, or categorize For example, each student may be

represented by a student ID number Or, maybe students are grouped according to a characteristic, such

as gender, with 1 representing male and 2 representing female The numbers or words arerepresentations, not measures, and they have no inherent order (1 could just as easily represent femaleand 2 male), so they cannot be used in mathematical calculations However, nominal data can be used toindicate similarities and differences

 Ordinal: Numbers or words are used to categorize information on the basis of order Observations are

rank-ordered on the basis of some criterion, but the intervals between the rankings are not necessarilyequal For example, a postgraduate degree is higher than a bachelor’s degree, which is higher than anassociate’s degree, which is higher than a high school diploma Likert-item responses, letter grades, and

rubric scores are other examples of ordinal data Ordinal data is used to indicate more than or less than

relationships

 Interval: Numbers represent equal intervals between points on a scale, but the scale lacks an absolute

zero IQ scores, calendar years (e.g 1983 and 2012), and temperatures are examples

 Ratio: Numbers represent equal intervals between points on a scale, and the scale has an absolute zero,

which allows measures of ratio to be taken We can say that one item is two-thirds the size of another orthat one student took twice as long as another to complete a task Height, weight, time intervals, andcounts are additional examples

Analytic Approaches

Statistical analyses are often classified as parametric or nonparametric:

Parametric analyses, associated primarily with inferential statistics, can be used with interval or ratio data

that are normally distributed when the variance within the two groups being compared is the same orsimilar Parametric tests include t-tests and analysis of variance (ANOVA) Longitudinal and cross-sectionalstudies typically employ parametric analyses

Nonparametric analyses, associated more with descriptive statistics, can be used with nominal and ordinal

data They are also useful for interval and ratio data that are not normally distributed and/or do not havevariances similar to those of the comparison populations Chi square tests are non-parametric

The decision chart below can be used to identify the statistical tests most applicable to your research project

Figure 1: Statistical Test Decision-Making Flowchart

Adapted from Ruth, D., Practical Statistics for Educators, p 227

You may have noticed that the flowchart above does not include any guidance for the use of ordinal data This isbecause statistical applications using ordinal data must take into account the degree to which the intervalsbetween can be assumed to be equal In some cases, such as the example of educational attainment given in theprevious section, ordinal data is essentially nominal data with some order to it While one might assign numbers

to progressive educational awards (e.g., 1 for High School, 2 for Associate’s, 3 for Bachelor’s, and 4 for grad), it would make no sense to calculate a mean on the data (or to conduct mean comparisons using anindependent samples t-test) However, it might make sense to calculate the mode or to calculate median incomeassociated with each educational level

Post-In other cases, when the ranking strongly suggests a degree of continuity across intervals, ordinal data may betreated more like interval data For example, it is common practice, though not all agree it should be, tocalculate mean responses from Likert items One sees this more in the social sciences and in qualitative researchthan in the fields of math and science or in quantitative research.

Correlation studies are commonly used in educational research because they are very helpful in determiningwhether a relationship exists and is worth exploring further Fortunately, or unfortunately depending on howmuch you love statistics, a variety of correlation coefficients are available for computation The Pearsoncorrelation coefficient (a.k.a the Pearson product-moment correlation, or just Pearson for short) is most widelyused, and indeed some analytic software programs offer only the Pearson Generally speaking, however, it is thetype of data that determines which correlation study is most appropriate Because the differences could beimportant if you intend to publish your research, the following differentiations are provided:

Pearson is a nonparametric test and can be used when the data is not normally distributed and/or when you

want to see if there is a relationship between two interval or ratio variables As indicated above, however,Pearson is commonly used with normally distributed data as well

Spearman’s rho (Spearman for short) is a parametric test, intended for use with normally distributed data.

You can use it to compute the correlation between two ordinal variables However, the Spearman is mostappropriately used with ordinal data for which the ranking is highly detailed and/or strongly suggests adegree of continuity across intervals In other words, Spearman is good for analyzing ordinal data thatclosely resembles interval data

For ordinal data that is more like nominal data, having five or fewer rankings with some order to them

and/or having intervals that are not really approximately equal (such as Likert-item responses), a Gamma

correlation may be more appropriate

Point-biserial correlation studies can be used to analyze dichotomous data Note, however, that only one of

the variables can be dichotomous Some data is naturally dichotomous, such as gender or Yes/No responses.And, sometimes people artificially dichotomize data, particularly ordinal data (beware however that somepeople frown on this practice) A common example of artificial dichotomization is taking Likert-items andputting the “Agree” and “Strongly Agree” responses into one category while putting the “Disagree” and

“Strongly Disagree” responses in another Another example would be putting all of the students who passedwith grades of C or better into one category and all the students who got D’s and F’s into another

Refresher of Basic Concepts

The following list highlights some key statistical concepts Additional definitions are provided in the glossary.Far more extensive glossaries of statistical terminology can be accessed online at sites such as statistics.com and

stat.berkeley.edu

Alternative

Hypothesis: Typically a statement made prior to a study asserting (hypothesizing) that there is or will be an

observable effect or difference (Contrast with Null Hypothesis.)

Averages: 3 main measures of central tendency (though there are others)

Mean: The arithmetic calculation of the sum of the numbers divided by the count of the

numbers This is what most people mean (no pun intended) when they say “average.”

It is the balance point of the data (A deviation is the difference between a data value

and the mean With the mean as the balance point, the sum of the deviations above themean will cancel the sum of the deviations below the mean.)