2.3. Studies on teacher professional development
2.3.1. Studies on teacher professional development in other countries
2.3.1.3. Methods used in research on teacher professional development
A review of the literature on teacher professional development reveals a variety of methods used to examine teachers’ knowledge: questionnaire (e.g. Aminudin, 2012;
Chen et al., 2009; Evan & Broad, 2006;Desimone et al., 2002; Garbin et al., 2015;
Santacroce-Tejedor, 2011; Sari, 2012); interviews (e.g. Aminudin, 2012; Boyardo, 2009; Brody & Hadar, 2015; Evan & Broad, 2006; Garbin et al., 2015; Sari, 2012;
Williams, 2010); classroom observation (e.g. Boyardo, 2009; Brody & Hadar, 2015;
Evan & Broad, 2006; Sari, 2012; Santacroce-Tejedor, 2011); tests for teachers’
knowledge (e.g. Smith & Banilower, 2006); tests for students’ achievement (e.g.
Evan & Broad, 2006); diaries (e.g. Le, 2013; Sari, 2012; Zwart, 2008); narratives (e.g. Barkhuizen, 2008; Clemente & Ramirez, 2008; Crawford, 2009; Liu & Xu, 2011; Sari, 2012; Wolfensberger, Piniel, Canella & Kyburz-Graber, 2009); reflective writing (e.g. Brody & Hadar, 2015).
Results from tests for teachers’ knowledge and tests for students’ achievements may be questionable as factors affecting such results are hard to control. Different assumptions about observation, interviews, and surveys as data collection methods have been made. Wragg (1999) assumes that observation is more unbiased as compared to self-report methods (interviews and surveys) because observation involves external reports. Meanwhile, Wengraf (2004, as cited in Desimone, 2009, p.188) considers observation as the most time-consuming and expensive method. He adds that how bias interviews are depends on the trust relationship between the interviewer and the interviewee, and for surveys, it depends on the way they are implemented. It can be implied from these assumptions that the quality of the instruments and how they are conducted determine the degree of bias; put it in another
33
way, as long as observation, interviews, and surveys are well constructed and appropriately used, they all yield useful data.
Based on various methodology books and articles, Desimone (2009) summarizes the situations in which each of these methods is useful. To be more specific, observation and interviews are more appropriate for the investigation into critical reflection (Putnam &
Borko, 1997; Wilson & Berne, 1999), the quality of discourse and the coherence of instructional presentations (Ball & Cohen, 1999; Burstein et al., 1995), and teacher warmth, humor, and openness (Burstein et al., 1995; Rosenshine, 1979; Solomon &
Kendall, 1976). Interviews and observation appropriately provides narratives, examples, and anecdotes to answer research questions about models of teacher interactions, and complexities of professional development in a specific context, how beliefs and attitudes changes, and the processes through which teachers change their instruction (Merriam, 1988; Wengraf, 2004). Meanwhile, surveys are more appropriate for the investigation into the frequencies and trends (Yoon et al., 2004), and specific features of professional development, and instructional time for specific content and practices (Koziol & Moss, 1983; Newfiled, 1980). Interviews and video-recorded observation are agreed to be powerful in qualitative research (Desimone, 2009), yet reflective writings are currently considered as well-established data source as researchers have more than the writing outcomes (i.e. the content itself) to look into. Even the structure of the text, the sequence of the events, and the timeline of the writings provide researchers with further understanding about the participants and their changes (Zilber, Tuval-Mashiach, &
Lieblich, 2008; Jasper, 2005). Desimone’s (2009) summary indicates the importance of how appropriate the data collection methods for the purpose of a specific study.
Another review of more than 50 studies on various professional development activities reveals that questionnaire and interview are used by many studies, and the focus of those studies is on teachers’ perceptions about changes and/ or impacting factors. Observation is used when the aim is to examine the changes in teaching practice. Meanwhile, narratives are used by fewer studies.
34
Table 2.4: Data collection methods in previous studies
Method No. of
studies Aims
Questionnaire 25
- Teachers’ perception of changes
- Teachers’ perception of their own needs and knowledge - Correlation between the professional development program and changes in teachers’ knowledge
Interview 25 - Teachers’ perception of impacts and/ or changes - Teachers’ description of the changing process Observation 14 - Examination of changes in teachers’ practice Reflective
journal/ diary/
narrative
8
- Teachers’ perception of changes
- Teachers’ description of the changing process - Voice from the below
It is revealed that many of the researchers have just drawn attention to the what- question based on teachers’ perception (Ado, 2012; Cabaroglu, 2014; Hathorn &
Dillion, 2018; Jao, 2013; Lee, 2011; O’connor, Greene, & Anderson, 2006; Pancsofar
& Petroff 2013; to name but a few). Their research question is mostly like “what are the impacts?” or “what changes do teachers gain after their participation?” These researchers did use more than one data source: questionnaire, interview, and classroom observation, or even use pre- and post-participation questionnaires and interviews. However, what they presented was just teachers’ perception of what they could gain from the process. Whereas, it has been agreed to be more and more challenging to measure professional development as its conceptualization has been more complicated (Wayne et al., 2008, Desimone, 2009).
Some other researchers have looked into professional development activities with how-question, besides what-question (Crawford, 2009; Goodnough, 2010; Hine &
Lavery, 2014; Smith & Winn, 2017; Zwart et al., 2008). Interviews, to a certain extent, can serve as a tool to dig into how an impact happens; as such, examples of interview questions can be: “How has participation in the project caused you to change your beliefs about teaching science through inquiry?”, “How has this project
35
helped you gain a greater understanding of your beliefs about student learning?”,
“How is your work being influenced by the context of your school?” (Goodnough, 2010). However, compared to the length of the professional development activity, a 60- or 90-minute interview does not seem to be enough to uncover much of teachers’
development process. That is likely to be a reason why reflective journals and diaries (categorized as written narratives) have been employed in professional development research. These tools are potential to be used at the end of a several-month professional development intervention (e.g. Hine & Lavery, 2014) or in a longitudinal study in which the participant is required to write a reflective journal or a diary after each implementation phase (e.g. Smith & Winn, 2017; Zwart et al., 2008). They are believed to allow teachers to mention their learning activities as well as the sequence of those activities that leads to their learning outcomes, not only teachers’ perception about changes (Zwart et al., 2008, p.986). However, the number of studies using narratives as a data collection method has been limited.