School - University Partnerships for Science and Mathematics Reform,

Horizontal and vertical integration of science and mathematics was enhanced in the second year of establishing a district-wide learning community of teachers of junior high science and m

Final Report (submitted to Arizona Board of Regents in August 2004, updated slightly in May 2012):

School - University Partnerships for Science and Mathematics Reform,

Year 2.

A grant for $48,900 funded in February 2003

by the Arizona Board of Regents: Improving Teacher Quality Program

Written by P.I Jane Jackson, Faculty Associate, Dept of Physics & Astronomy, ASU

480-965-8438, Jane.Jackson@asu.edu

The Co-P.I was David Hestenes, Research Professor of Physics & Astronomy, ASU

Summary:

School - University Partnerships for Science and Mathematics Reform was the second year

of a pilot project to establish the Arizona Science and Technology Education Partnership (AzSTEP) as a statewide program to provide K-12 schools with the resources they need for high-quality, systemic reform in science, mathematics and technology education

Horizontal and vertical integration of science and mathematics was enhanced in the second year of establishing a district-wide learning community of teachers of junior high science and mathematics in Mesa Public Schools (MPS) Twenty-three teacher participants, 2 apprentice leaders, and 6 mentors from the previous year's workshop participated in a 3-week summer workshop in physical science with mathematical modeling The workshop included thematic strands in scientific modeling, structure of matter, energy, and use of calculators and computers

as scientific tools Mathematics instruction was coupled to this thread through an emphasis on mathematical modeling MPS science and mathematics specialists coordinated recruitment Workshop leaders led follow-up meetings Increased content knowledge and better instructional strategies of teachers correlated with measured improved learning of students

I Overview of the Modeling Workshop: project design and procedures

Physical science and mathematics teachers in 11 of Mesa's 13 junior high schools were selected by Mesa's science and mathematics specialists, Rick Vanosdall and Sandra Nagy, for their interest, ability and experience Most selected teachers were of grades 8 and 9, because the workshop curriculum is considered to be too hard for 7th graders, and the 7th grade curriculum is mostly life science About two-thirds of the 23 participants teach science, and one-third teach math Two participants teach industrial technology, which use math and science for technical applications Three of the 23 participants repeated the workshop, having previously attended in summer 2002

School-level teams of a math and a science teacher were sought, because research shows that

the best follow-up is daily interaction of teachers about the reform Few teams could be found.

However, some interaction was possible, because 23 teachers had participated in the previous summer from most of the same schools

Support of principals was enlisted by Rick Vanosdall and Sandra Nagy Some principals were already aware and supportive The specialists organized a morning workshop in March for principals, led by workshop leaders Larry Dukerich and Stella Ollarsaba, to acquaint them with the project design, involve them in a modeling activity, and solicit their support They were

reminded of the value to their school because improved coordination between science and math

teachers will lead to better performance on the AIMS test, since students will have opportunity to

learn math concepts in a physical context

Workshop leaders Larry Dukerich and Stella Ollarsaba invited 8 workshop participants from last year to serve as apprentice leaders or mentors, to establish continuity and enhance the reform Summer activities started the week of June 2 with re-design of the previous summer's workshop materials and evaluation instruments (PSCI and MCI) by the two apprentice leaders and Larry and Stella to align with Mesa's standards and state standards in mathematics

The workshop itself was held from June 9 – 27, 2003 at Dobson High School in Mesa Apprentice leaders conducted parts of some sessions under guidance of Larry and Stella Teachers met daily for 4 hours; included were mentors on a few days

Most participants, including a few teachers in the previous year's workshop, gave the revised

Physical Science Concepts Inventory (PSCI) or Math Concepts Inventory (MCI) to a class this

year We wanted to learn if their implementation improved during their second year, because we find that, in physics, implementation usually improves for 3 or 4 years

Unfortunately for the project, Rick Vanosdall left the district in the summer for another job

He had been enthusiastic about this project, and he had planned to assume leadership in the fall His guidance was sorely missed because he and Sandy Nagy were best positioned (1) to coax teachers in each school to collaborate, and (2) to work with principals to implement short inservices

During the academic year, teachers met informally in small groups in their schools and in a large group on two Saturdays with Larry and Stella to deepen the learning Teachers questioned and debated, and shared materials, methods, and reflections on progress They evaluated effectiveness of instructional materials, and modified and re-designed curriculum materials for future use The 23 participants from the summer 2002 Modeling Workshop were invited, and most attended

Teachers were encouraged to request classroom visitations and informal advice from apprentice leaders and mentors Larry and Stella reported on the commitment of participants in implementing their workshop learning, and they provided informed feedback and support to participants

For long-term professional development, teachers were subscribed to a local junior high school modeling listserve managed by Modeling Instruction staff

Contact time was 72 hours, plus individual work (readings, written reflections, learning technology, adapting instructional materials for their courses), totaling about 135 hours Teachers earned 3 semester hours of graduate work in physics or in math education - their choice

II Recommendations for Improved Modeling Workshop implementation

We recommend that the 40 Mesa Public Schools teachers who have learned Modeling Instruction have additional follow-up and support, to retain and deepen their learning Most of the following actions can be spearheaded by district math and science specialists

• Mentors should be given specific training Larry Dukerich gave a vision in his e-mail of June 4, 2003 He wrote me,

"We met with three of the mentors (Carl Harms, Jacob Davis and Tony Garibay) yesterday and hope to see the others today They seem very intrigued by the expanded role they will play You see, for the longest time in Mesa, you either put

on a workshop or attended one There was no one encouraging you to actively work with your colleagues to continue the work started in the workshop It was like

planting a seed and walking away Here we are watering it and nurturing it This is very different and will require encouragement on the part of you and Stella and me

to remind these mentors (and leaders) that they can and should take an active role in bringing about reform

Speaking of that, I would like to give each of the participants a copy of "Before It's Too Late", the final report of the Glenn Commission I found that it clearly articulated most of the arguments that teachers would like to make (but find themselves unable to) when talking with administration about what needs to change You should have heard Jacob - that boy is a ball of fire and really wants to see change take place He suggested it would be a good idea to let the math and science teachers break away during a late-start day to review what was learned in the workshop - pretty neat that he came up with the same idea we proposed, no? Well, the steps outlined in the Glenn Commission report are the kind of ammunition teachers need when trying to convince administration to change."

• Teachers should re-design curriculum together, with guidance from Modeling Instruction Project faculty at Arizona State University

• Teacher networks are the most effective means for consolidating and sustaining reform (Adams, 2000) Thus teams of math and science teachers in each school should meet to plan some lessons together, discuss how they can use modeling strategies, and observe each other's teaching; they could report to science and math coordinators in writing on this Also, they should study samples of student work MPS science and math specialists can coax teachers to collaborate

• Workshop leaders (Larry and Stella) should be given release time and substitutes to visit selected classes of workshop participants, or vice-versa, to help improve implementation Admittedly, this is difficult, given the present organization of a teacher’s day; but it is important

• 'Best practice' videos of modelers should be viewed by teachers Two such 20-minute videos

of Larry Dukerich's teaching were made in 2003 at ASU in Ann Igoe's and James Middleton's NSF Preparing Tomorrow's Teachers in Technology-PT3 grant James.Middleton@asu.edu Streaming format: http://vimeo.com/channels/modelingphysics

• Explicit instruction should be given to under-prepared teachers on conservation of weight (or mass), conservation of volume, proportional reasoning and other rational number concepts (see section below on measurement of teacher learning) All teachers should be given instruction on how to teach thinking skills such as conservation of weight and volume, proportional reasoning Some resources for this are by "Thinking Science" developer Adey (1999) and Adey (2003)

• We suggest that you buy each science teacher a copy of Anton "Tony" Lawson's book,

Science Teaching and the Development of Thinking (Wadsworth, 1995 & 2002; paperback &

hardback) The revised edition is expected to appear in 2005 The book is the text for BIO480, "Methods of Teaching Biology", which is a senior level course but could perhaps be cross-listed as a graduate course and a special section be organized for inservice teachers:

<Anton.Lawson@asu.edu> You could form study groups with that book as focus

• Principals should be asked to try to provide common planning time for teams

• Principals should be asked to target some monthly school late-start days for an inservice for all math/science teachers, led by workshop participants Additionally, workshop participants could lead school-wide inservices on whiteboarding and discourse

• Principals should be made aware that implementation of modeling instructional materials in the 8th grade science course will provide students with skills necessary to succeed in physics

if the district eventually moves to a physics-first sequence (i.e., mostly physics in grade 9, mostly chemistry in grade 10, biology and advanced courses in sciences in grades 11 and 12)

III Measurement of teacher learning

A Physical Science Concepts Inventory

All 23 teachers took the Physical Science Concepts Inventory (PSCI) pretest (version 7) on the first workshop day and the posttest on the last workshop day Teachers' pretest mean was 76%, posttest mean was 83% (pretest median was 80%, posttest median was 88%) These scores are higher than the previous year's group, when the Mesa teachers’ pretest mean was 69% and posttest mean was 80% An earlier version of the PSCI had been used, but most questions were the same

Appendix A is a graph of each teacher’s scores Seven teachers were initially low-performing Two of these teachers improved tremendously; however, the other 5 did not improve (their scores remained at 50% to 65%) This is troubling, for they lack basic reasoning skills that are crucial for math and science, as we see in the following analysis

The content subscales of PSCI on which teachers (specifically, this group of lowest-performing teachers) did badly were these:

This is shocking, for most 10-year old children have mastered this thinking skill, according

to Arizona State University Professor Anton "Tony" Lawson, whose research field is scientific thinking skills (private communication; also, Lawson and Bealer, 1984) (Tony Lawson developed and teaches the Learning Cycle Modeling Instruction is a refinement of the Learning Cycle; it is the Learning Cycle structured around scientific and mathematical models.)

This is 1/4 of the class; a very disturbing situation, because most 11 to 12-year old

children have mastered this thinking skill! (Anton Lawson, private communication, June 2003.)

posttest This is one-quarter of the class; a disturbing situation because proportional

reasoning is crucial in math and in physical science!

• Energy & states of matter (4 questions): 14 teachers (60% of the class) still had

misconceptions at the end of the workshop This content was the last focus of the workshop,

so the poor results aren't surprising because teachers hadn't had time to reflect on their new learning, which was novel Larry wrote me, "Stella told me that I was a slave driver, as I pushed the teachers to complete the materials, knowing that the energy treatment would be novel for most of them."

Regarding the 25% of the class who failed to do proportional reasoning: in December 2002, after I related similar results from the previous workshop to Larry Dukerich, he wrote to me,

"The questions borrowed from Lawson's Classroom Test of Formal Reasoning (items 1-8 on test

v5) have been banging around for quite some time The majority of the teachers already knew how to do items 5-8 Those who did not, showed only slight gains primarily because there was little instruction on proportional reasoning in the workshop."

Other researchers have found similar gaps in middle school teachers' understanding For example, referring to teachers' understanding of rational number concepts, Behr, Harel, Post and Lesh (1992) reported, "In our recent survey of over 200 intermediate-level teachers in Minnesota and Illinois, one-quarter to one-third did not appear to understand the mathematics they were teaching (Post, Harel, Behr, & Lesh, 1988)."

Clearly, explicit instruction on conservation reasoning, proportional reasoning and other rational number concepts must be included in future workshops As Behr et al say, "Although there is no indication that teachers cannot learn these concepts, large-scale in-service in these areas becomes a logical necessity." (ibid) We offer a companion course, "Integrated math and science for middle school", that focuses on these thinking skills

Most teachers achieved well on subscales of graphing skills/motion, geometrical & physical properties of matter, and atomic structure of matter In fact, their achievement on atomic model

of matter was MUCH better than the previous group's achievement Larry Dukerich was aware that he needed to improve this component Apparently he did; congratulations to Larry!

B Math Concepts Inventory

All 23 teachers took the MCI pretest (version 4) on the first workshop day and the posttest on the last workshop day Teachers' pretest mean was 79%, posttest mean was 84% (pretest median was 76%, posttest median was 84%) Appendix A is a graph of each teacher’s scores

The first 10 questions on both instruments (PSCI and MCI) were the same (they are questions

#1 to 10 of Anton Lawson's Classroom Test of Formal Reasoning) Teachers answered these 10

questions the same way on each test, a sign that they took the tests seriously

We find it alarming that five teachers scored low on both pretests and both posttests (their scores remained at 50% to 65%) Two of these 5 teach science, two teach math, and one teaches

both subjects From the test scores, it appears that these 5 teachers don't understand the math

and the scientific reasoning skills that they are teaching The problem is apparently not in the

workshop design, for other teachers with low pretest scores improved dramatically on the posttests

Current versions of the PSCI (vs 8) and MCI (vs 7) (which don't include Lawson's questions

#7 and 8 on proportional reasoning, for they are deemed too hard for junior high students) are at

http://modeling.asu.edu/MNS/MNS.html The password to open them can be obtained from Jane.Jackson@asu.edu

IV Follow-up meetings and implementation

On Nov 15, the first of two Saturday morning meetings was led by workshop leaders Larry

Dukerich and Stella Ollarsaba Amazingly, 29 teachers participated! Among them were several

of the 2002 Modeling Workshop participants

The agenda was:

* break into subject groups (math & science), discuss progress, fill out whiteboards to report to

main body

*reconvene as large group – show and tell lessons we’ve tried or plan to try

* discuss how to effectively work together in the building – how to increase communication –

how to modify the workshop content to make it more meaningful to junior high teachers Let’s be positive!

* divide into groups to review use of Graphical Analysis software or graphing calculator

Teachers were positive and enthusiastic, reported Larry Dukerich

The second Saturday meeting was held on Feb 21 The group discussed implications of the new draft science standard Teachers and workshop leaders thought that it would be useful to prepare a document representing the views of a significant fraction of Mesa junior high teachers

Prior to the November 15 meeting, 21 teachers e-mailed to me a report on their implementation A summary of their responses follows Keep in mind these facts:

1) 7th and 8th graders have only 1 semester of science

2) 7th grade science was mostly life science, with a couple of weeks on phases of matter

3) Motion was taught only in 8th grade, and it was taught after mid-November, in only 3 weeks, including instruction on Newton's laws Energy and atomic nature of matter weren’t in the 8th grade curriculum

4) In 9th grade science, about 3/4 of the year focused on chemistry, including atomic nature of matter

5) Energy wasn’t taught in-depth in grades 7, 8, or 9 A serious omission!

6) District math and science curricula were jam-packed full of required objectives

7) 9th grade alternative algebra is double period, for low achievers in math

Thus implementation of teaching methods is expected to be more evident than that of content.

-SUMMARY OF TEACHERS' REPORTS ON IMPLEMENTATION

• Median # years experience teaching junior high was 6 Range was from 1 to 31 years

• 13 teachers gave the PSCI to students, most to grade 8 Three teachers who gave it to grade 9 students were in their 2nd year of implementation; one teacher of grade 8 was in 2nd year of implementation

• 8 teachers gave the MCI, most in grade 8 Two gave it in grade 9 alternative algebra (double period)

Answer on scale of 1 to 5 (1= not at all or insignificant, 3 = somewhat, 5 = fully or substantial)

CONTENT: To what extent have you already implemented the units this semester?

• Unit 1: models of measurement Half of the teachers wrote 3 (somewhat) No difference

between math and science teachers Range 1 to 5

• Unit 2: motion Most teachers wrote 1 (not at all) 3 math teachers & 1 science teacher

wrote 3 Range 1 - 3

• Unit 3: structure of matter/energy 9 th grade science teachers wrote 3 or 5 Most others

wrote 1

METHODS:To what extent have you used this semester:

• whiteboarding? 50% of the teachers wrote 4 or 5 40% wrote 3.

• Socratic questioning? 45% of the teachers wrote 4 or 5 45% wrote 3.

• circle whiteboarding? 20% of the teachers wrote 4 or 5 Half wrote 3.

• cooperative groups? 60% of the teachers wrote 4 or 5 25% wrote 3.

COORDINATION OF MATH AND SCIENCE: To what extent are you coordinating your math

or science course with your colleague(s)? (so that the courses enhance each other, and thus

students learn more) 75% wrote 1 (not at all) Range 1 - 3.

SUMMARY: Overall, to what extent has the Modeling Workshop enhanced your teaching?

25% of the teachers wrote 4 or 5 50% wrote 3.

-Teachers voluntarily made extensive comments about their implementation Most comments were similar to those expressed by the previous year's workshop group (see that ABOR report)

We repeat them here because they are informative and typical

* "The workshop brought about a change in my teaching style This is shown in the increase in

my students’ enthusiasm for science They hardly ever work alone now They are much more

engaged My students’ test scores also show improvement."

* "Big whiteboards really promote interaction between the students While one is writing, the others are helping and checking." "While working in groups, the students are debating and

critiquing each other Oral and written communication skills are being improved."

* A math teacher was so enthused that she and her principal made sure that every one of the math teachers in her school had a classroom set of whiteboards

* They all felt that they knew enough about whiteboarding that they could implement this aspect

of modeling in their classrooms

* "The workshop experience changed the way I look at science in general I find myself looking for the models in the rest of the 9th grade science curriculum It helps me realize the way students learn so that I am a more effective teacher."

* Several teachers reported that they are “converting” colleagues who didn't participate

* The level of the activities was too high for 7th grade students

* Everyone expressed the concern that what was expected of them (either preparing for

district tests or conforming to the existing curriculum) hindered their ability to implement

as much of the materials as they would have liked This was the greatest roadblock to implementation!

* Several teachers commented that it would have been much better if a team of math and science teachers from their school had participated in the workshop

The Nov report included several more questions Teachers responded as follows (in both years):

• groups of students especially well served by Modeling Instruction are lower ability kids -concrete learners; shy, bright kids, ELL (visual representations; input from other kids; whiteboarding helps a lot with vocabulary!); gifted students (they can use software without supervision); special needs students special ed (graphing skills increase their self-esteem), and girls (they enjoy science more)

• Obstacles to implementation are: modeling takes time; too much hassle to go to the computer lab to use MathWorlds or Graphical Analysis software

• One teacher used SimCalc MathWorlds software for multiple periods Most didn't use it

• Desires for future professional development are: review strategies in modeling, questioning, and cooperative groups; review use of TI-83+ graphing calculator; review SimCalc MathWorlds software; how to use modeling instruction in reading, social studies, and civics;

take a few days in summer to develop lessons for one's classes and share them; specific modeling strategies in mathematics; strategies for low-level math students

• All said that it would be very valuable if they had an hour scheduled regularly (every other

week? once a month?) to coordinate their math and science courses with colleagues in their school

• Technology desires: several have small classrooms (too small for desktop computers) and would love to have a classroom set of TI-83+SE graphing calculators (one said that calculators are preferable to computers because graphing calculators force students to work, whereas students can be lazy if they're in a group that's working on a computer) Two teachers said that they wanted 10 computers in their classroom, and I was able to provide StRUT-donated computers for them Larry Dukerich gave his classroom set of older computers to a teacher

Summary of teachers’ implementation:

• Large increase in use of whiteboards

• Some increase in student discourse (Socratic questioning)

• Less lecturing

• Less use of a standard textbook

• The workshop moderately enhanced their teaching (3 on a scale of 1 to 5)

• Little coordination of courses with colleagues

• No improvement in classroom technology Typically teachers have 1 to 3 computers in their classroom Some have access to a computer lab (often with difficulty in scheduling) Most teachers don't have TI-83+, but rather older models for which SimCalc can't be used

• The degree of encouragement from administration to implement the workshop learning was vastly different in different schools It tended to be either very low or very high

Listserv for participants:

I subscribed all teachers to a listserv to which 150 Phoenix area junior high Modeling Workshop participants, workshop leaders, and district coordinators subscribe Hardly anyone ever posts, but this is typical for a teacher listserv of this size, in my experience I posted occasionally on resources: how to borrow classroom sets of TI-83s from Texas Instruments, how to write a $500 grant to the Wells Fargo Teacher Partner program, recent research on 'not giving the answer' to middle school students, exemplary middle school science programs, and the like

V Measurement of student learning

I preface this section by quoting Larry Dukerich (e-mail to me, June 2003):

"I am hesitant to draw far-reaching conclusions about the merits of individual questions (or categories of questions) based on the scores of some teachers in Mesa who frankly admitted that they were unable to implement the curriculum in any meaningful way I cannot tell you how dismayed I was to learn (in our follow-up sessions) that the teachers could only do this or that activity here or there because they felt compelled to cover all the topics required of them in order to prepare their students for the end-of-year tests Having spent a significant chunk of a year devising

a set of activities designed to carefully build skills and a coherent view of the atomic

model of matter and energy, I felt as if it had been a real waste of time Some of these teachers may as well be a control group!

So, I would do my best to compare gains to the teachers' self-rating of the degree

to which they were able to implement the curriculum they learned - just as we did with the Modeling Instruction in High School Physics workshops It seems reasonable to me that one would only find gains for the students in classes in which the teachers were able to implement the curriculum in a coherent way."

Although 21 teachers administered the PSCI or MCI (see above), the usual difficulties in matching students' pretest and posttest scores occurred, with the result that our staff could analyze significant numbers of matched pre- and posttest student data only for 9 science teachers and 3 math teachers Results follow

A Results from the previous year (2002 – 2003):

As background and for comparison: in classes of teachers in the previous modeling workshop (in summer 2002), the highest student gain in Mesa was 9th grade math (7% gain; i.e., 7 percentage points gained); most students tested were in alternative algebra, a 2-period course for

low achievers (representative of the group most in need of improved instruction!) Specific gains

in scores for the 135 9th grade students who took BOTH the MCI pretest and the posttest were: 7% overall gain

8% conservation of weight & volume

7% proportional reasoning

9% graphing skills and motion

6% geometrical and physical properties of matter

Actual MCI scores for that Mesa group are recorded in last year's ABOR final report

Appendix B is a tabulation of last year's MCI and PSCI scores for grades 7, 8, and 9 in Mesa, and for grades 7, 8, and 9 in 5 Phoenix-area districts: Glendale Elementary USD, Washington Elementary USD, Glendale Union High School District, Peoria Unified, and Mesa All data are matched pre- and posttest Numbers in parentheses are the number of students in the sample

B This year's results (2003 – 2004):

Appendix C is a spreadsheet of (mostly) grade 8 and 9 student pretest and posttest mean scores in the MCI and the PSCI in 2003-04 All data are matched, pre- and posttest Numbers in parentheses are the number of students in the sample, i.e., the number who took both the pretest and posttest We don't have time nor staff to analyze data by gender, and the numbers of disadvantaged minority students in the samples are too small to yield good statistics on race/ethnicity

Math Concepts Inventory (vs 7)

This year (2003 – 2004), we don't have enough matched MCI data points, i.e., not enough students took both the pretest and posttest, to split it into grade levels The 68 students for whom

we have matched pre- and posttest MCI scores were in grades 8 and 9 (of 3 teachers: 9th grade alternative algebra, 8th grade pre-algebra, and industrial technology) Their overall gain was 4

percentage points (posttest average of 43%) Gains in subcategories were:

9% conservation of volume

5% graphing skills

11% measurement and geometry

5% statistics and data analysis

Students had no gain in the new category of equation skills (average of only 25%) I wonder

why, for the questions are conceptual and basic - designed by Mesa teachers to be similar to the

8th grade AIMS test

The new version of the MCI is markedly different from previous versions; hence the new subcategories, which better reflect state standards in math

Physical Science Concepts Inventory (vs 8)

In the PSCI, the 58 9th graders (general science: 3 teachers, all of whom are in their 2nd year of

implementation) gained 6 percentage points (posttest average of 43%) Gains in subcategories

were:

13% conservation of volume

10% control of variables

6% geometric and physical properties of matter