Mixed Methods in Educational Research -The Third Research Paradigm Handout

Quantitative ResearchExamines relationships among variables using numerical data under standardized conditions; tests theories Mixed Method Research Combines methods and capitalizes on t

The Third Research Paradigm:

Mixed Methods in Educational Research

ELC 2019 Belleview, WA

Session Overview

OBJECTIVES:

1 Examine the unique contributions and value of mixed

methods research designs in physical therapy education

4 Discuss how to establish a mixed method research team

5 Explore resources within the APTE, APTA, and ACAPT to

support educational researchers

Why Mixed Methods Research?

& intervention for public health problems

 Genetic / physiologic causes

 Behavioral determinants

 Societal / environmental factors

 Quant – body structure/function & activity limitations

 Qual – participation, social & environmental contexts

Quantitative Research

Examines relationships among variables using numerical data under standardized conditions; tests theories

Mixed Method Research

Combines methods and capitalizes on the

strengths of each; provides better

understanding of research problem

Best Practice Guidelines

in the Health Sciences

 NIH – OBSSR (2011); Creswell, Plano Clark, Klassen, & Smith

_research/index.aspx

research into their applications for NIH research, fellowship, career & training grants

research applications

5

Designing Mixed Methods Studies

 Present rationale for combining qualitative and quantitative strategies

 Distinct vs integrated aims

Box 2 Table of Strategy, Sample, Goals, and Analysis Gallo, J.J (2003-2007)

primary care settings

Assess depressive symptom patterns and correlates

Multivariate regression models

Semi-structured

interviews

Purposive: 50 African American and

50 white adults from Spectrum sample (who may or may not be depressed)

Identify an explanatory model for depression

Grounded theory

Free listing

and pile sorts

First 25 African American and 25 white adults selected above for semi-structured interviews for free listing Second 25 African American and 25 white adults selected above for pile sorts

Identify the domain

of depression and its characteristics

Cultural consensus analysis

Identify social meaning

Discourse analysis

Gallo, J.J (2003-2007) The sociocultural context of depression in late life Research grants funded by the National Institute of Mental Health (R01MH67077, R01MH62210, and R01MH62210-01S1).

Qualitative Data

Merge Concurrent data

Results Quantitative data

Qualitatve data Quantitative data Results

Quantitative

Qualitative data

Results

Connect Sequential data (exploratory/explanatory)

Design & Integration of Quantitative & Qualitative Data

Adapted from Creswell & Plano Clark, 2007

Summary for the Review Process

best option

technique

 Integrate!!

each method

10

Examples from the Literature

11

Palermo C, King O, Brock T, Brown T, Crampton P, et al (2019) Setting priorities for health education

research: A mixed methods study, Medical Teacher, 41:9, 1029-1038, DOI: 10.1080/0142159X.2019.1612520

13

Research Question

Results

16

Research Question

18

Research Question

Results

20

Developing and Educational Research Agenda

building lines of inquiry that will result in

original contributions to the profession

scholarship, scholarly activities and

anticipated accomplishments with a timeline

show some level of consistency

21

Developing an Educational Research Agenda

to teaching and learning

 Recommendation 13 (from PTE-21): Create learning environments in academic and clinical settings that provide opportunities for situated

learning experiences that are intentional, sequenced, and occur

frequently across the curriculum.

the question through research/reading

22

Developing an Educational Research Agenda

 Quantitative Component?

 Quantitative Component?

or another piece of the "question"

23

Developing an Educational Research Agenda

24

Reading and Research

•Study 3

Design/IRB

•Study 2

Data Collection

•Study 2

Data Analysis

•Study 1

Writing

•Study 1

Teamwork: Forming the Mixed Methods Research Team

The nature of the research question shapes the nature and experience levels of the research team compiled to address them: questions driving the research initiative

determine the expertise required

Rosenfield’s (1992) Taxonomy of Three Levels of Collaboration

Level Three

Teamwork: Forming the Mixed Methods Research Team

■ Successful mixed methods teams include breadth and depth of expertise:

Address prioritization early

 Requires planning & integration of both methodological and disciplinary expertise

 Breadth of disciplines relevant to key questions; conceptual & analytic depth to

answer them.

■ A mixed methods team is more than a team of individual researchers with

distinct methodological traditions and skills:

 Should transcend distinct methodological & epistemological differences  create processes for integration throughout; ideally engaging multiple members of the

research team

■ The relationship of each member of the research team to mixed methods

research: It is not necessary for everyone to hold expertise in all methods employed.

■ The capacity of a mixed methods research team takes time to develop:

Perspective and approach adopted by core team members; evolves over time

■ A team that has a history of successful collaboration is in a strong position to demonstrate capacity for mixed methods research.

Leading and Guiding a Mixed Methods Research Team

■ The skills and outlook of the PI or research leader are critical:

 A broad perspective on the utility of different methodologies

 An ability to support and acknowledge different team member contributions

 An effort to maintain continual dialogue about issues in working together

 A sensitivity to workloads of team members that may pose challenges to working on the project

 Support for educating team members in different methodologies, when needed

■ An effective leader should be experienced and interested in qualitative, quantitative, and mixed methods research, as well as a person who is able to bridge differences and create synergy

 Will integrate team members who hold distinct methodological positions / expertise and those who cross methodological / disciplinary boundaries

■ A shared vision and defined roles within the mixed methods team is critical

■ A mixed methods team can be structured a variety of ways: a linear relationship, a spoked wheel

■ Different team perspectives need to be recognized and honored

Building Infrastructure and Resources

■ All research initiatives benefit from being conducted in a

well-resourced and dynamic research environment:

 Mixed methods research likely to require wider array of computer software needs than a single method study

 Mixed methods teams will benefit from being able to call upon other experienced and informed colleagues at their own institutions.

 Effective mixed methods research teams require collective as well as individualized workspace.

 Flexibility for staffing support

■ Team members can profit from mixed methods training and

workshops

■ Mixed methods teams may benefit from the ability to work together virtually

Training: Mixed Methods Research Team

■ Engaging in mixed methods research can serve as a learning opportunity for team members:

 Collaborative educational opportunities → build methodological capacity throughout team

 Productivity and effective collaborations largely defined by explicit, shared understanding of each member’s expectations and goals

■ Mixed methods research likely to require additional training of some / all of the research team

■ Appropriate and effective training for mixed methods research need not occur in

a traditional classroom setting

■ Mixed methods analysis benefits from team members’ comfort with multiple

types of data, analysis methods & approaches

■ Mixed methods research can benefit from adopting training and materials from other scholarly disciplines

■ Building mixed methods expertise should explicitly prioritize collective capacity.

Strategies to Develop Partnerships

Within and Across Institutions

possess specific skill sets that differ from

your strengths

 i.e Grit Study

have similar interests; reach out

30

Resources to Support Educational

33

Almalki, S (2016) Integrating Quantitative and Qualitative Data in Mixed Methods Research—Challenges and Benefits Journal of

Education and Learning, 5(3) doi:10.5539/jel.v5n3p288

Creswell JW Research Design: Qualitative, Quantitative, and Mixed Methods Approaches 2ndEdition Sage Publications; 2003.

Creswell J, Plano-Clark V Designing and Conducting Mixed Methods Research Thousand Oaks, CA: Sage Publications; 2007.

Creswell JW, Carroll Klassen A, Plano Clark VL, Clegg Smith K Best Practices for Mixed Methods Research in the Health Sciences Office of Behavioral and Social Sciences Research Available at https://obssr.od.nih.gov/wp-content/uploads/2018/01/Best-Practices-for- Mixed-Methods-Research-in-the-Health-Sciences-2018-01-25.pdf Accessed 9/30/19

Curry L.A, O’Cathain,A, Plano Clark VL, Aroni, R, Fetters M, Berg D The role of group dynamics in mixed methods health sciences

research teams Journal of Mixed Methods Research August 29, 2011; pp 5–20.

DeJonckheere M, Lindquist-Grantz R, Toraman S, Haddad, K, Vaughn LM Intersection of Mixed Methods and Community-Based

Participatory Research: A Methodological Review Journal of Mixed Methods Research 2018.1-22

https://doi.org/10.1177/1558689818778469

Jensen GM, Hack LM, Nordstrom T, Gwyer J, Mostrom E National Study of Excellence and Innovation in Physical Therapist Education:

Part 2—A Call to Reform Physical Therapy 2017 97(9):875–888 https://doi.org/10.1093/ptj/pzx062

Jensen GM, Nordstrom T, Segal RL, McCallum C, Graham C, Greenfield B Education Research in Physical Therapy: Visions of the

Possible, Physical Therapy, Volume 96, Issue 12, 1 December 2016, Pages 1874–1884, https://doi.org/10.2522/ptj.20160159

Mayoh J, Onwuegbuzie AJ Toward a Conceptualization of Mixed Methods Phenomenological Research Journal of Mixed Methods

Research 2015 9;1:91–107 https://doi.org/10.1177/1558689813505358

Mertens DM Publishing Mixed Methods Research Journal of Mixed Methods Research 2011;5(1):3–6

Moseholm E, Fetters MD Conceptual models to guide integration during analysis in convergent mixed methods studies Methodological

Innovations 2017 10(2) 1–11 https://doi.org/10.1177/2059799117703118

Rosenfield P The potential of transdisiplinary research for sustaining and extending linkages between the health and social sciences

Social Science and Medicine 1992;35(11):1343-1357.

Saint Arnault D, Fetters MD RO1 Funding for Mixed Methods Research: Lessons Learned From the “Mixed- Method Analysis of Japanese

Depression” Project Journal of Mixed Methods Research 2011;5(4):309–329.

Sendall MC, McCosker LK, Brodie A, Hill M, Crane P Participatory action research, mixed methods, and research teams: learning from

philosophically juxtaposed methodologies for optimal research outcomes BMC Medical Research Methodology 2018;18(1):167

https://doi.org/10.1186/s12874-018-0636-1

Shulha LM, Wilson RJ Collaborative mixed methods research In A Tashakkori & C Teddlie (Eds.), Handbook of mixed methods in social

and behavioral research (pp 639-669) Thousand Oaks: Sage Publications, 2003.

1 Address the shortage of qualified faculty and of academic leadership Recommendation 5: Develop shared leadership models that facilitate innovation and excellence

Recommendation 6: Foster innovation and risk taking to drive the shared vision

2 Create strong, equal academ- ic-practice partnerships that foster excellence

Recommendation 7: Build fair, creative, and responsible partnerships between academic and clinical faculty Recommendation 8: Make clinical faculty full partners with the academic program.Recommendation 10: Require early authentic clinical experiences that provide for teaching and learning in the context of practice

3 Infuse the learning sciences into the preparation of academic, clinical, residency, and

fellowship faculty

Recommendation 12: Implement faculty development programs grounded in the learning sciences that facilitate an understanding of, and competence in, the pedagogy of learning for and through practice Recommendation 13: Create learning environments in academic and clinical settings that provide oppor- tunities for situated learning experiences that are intentional, sequenced, and occur frequently across the curriculum

Recommendation 16: Foster opportunities for physical therapist educational researchers to generate an evidence-based approach to teaching and learning

4 Develop a continuum of profession- al performance expectations that are grounded in key

competencies and support excellence in learner development

Recommendation 14: Require academic programs to participate in residency education to enable reciprocal teaching and learning between professional and postprofessional learners in communities of practice Recommendation 15: Establish a comprehensive, longitudinal approach for standardization of perfor- mance-based learning outcomes grounded in foundational domains

of professional competence Recommendation 25: Create a national data set that includes essential metrics of performance outcomes, structures, and processes to guide future evidence-based change

5 Focus curriculum content on soci- etal needs for physical therapist practice

Recommendation 2: Demonstrate learner-centered teaching and patient-centered care in all settings Recommendation 9: Make the profession’s signature pedagogy, the human body as teacher, visible in all environments and available for further investigation

Recommendation 11: Develop consistent, shared language about the multifactorial movement system that can be used across academic and clinical settings

Recommendation 17: Develop a comprehensive, longitudinal approach to teaching, learning and assessment of clinical reasoning abilities

6 Devote significant resources to enhance the diversity in the pro- fession

Recommendation 22: Academic institutions must take a leadership role to create more diverse and inclusive learning and practice environments in order for the profession to have a positive impact on addressing the social determinants of health

7 Educate students to become moral agents as health care practitioners

Recommendation 18: Develop a strong sense of the moral foundations that underpin and are inseparable from practice in all physical therapists so that they develop the moral courage and ability to meet patients’ needs There must be faculty who have expertise in the moral foundations of practice and who collaborate with other faculty members to integrate the moral foundation of practice into learning throughout the curriculum

Recommendation 19: Prepare learners who act as moral agents and exhibit moral courage in addressing substandard practices

Recommendation 20: Act on our individual and collective responsibilities to society, or we jeopardize our status as a profession

Recommendation 21: Recognize the unique responsibility as academic programs to partner with the community in developing and implementing programs that place positive health outcomes to the community as their primary focus

8 Achieve control of fiscal resources for physical therapist education

Recommendation 3: Develop leaders who can leverage resources to succeed in a rapidly changing health care system

Recommendation 4: Develop strong leaders with a compelling vision

Recommendation 23: Focus attention across the profession on improved graduates’ outcomes, the need to reduce the cost of education to students and society, and the acquisition of resources to support physical therapist education

Recommendation 24: Stop expending resources to identify a narrow set of specific academic organizational structures or curricular models for physical therapist education; expend resources

to identify the best way to achieve excellence reflected in the findings of this study and recommendations

Recommendation 26: Assure that academic programs have control of their financial resources, and that they develop economic models for revenue generation through multiple means, and move toward larger programs as one means to increase the range and depth of faculty expertise and other necessary resources Recommendation 27: Develop strategies so that academic programs become respected, valued partners within their organizations and have influence over