School - University Partnerships for Science and Mathematics Reform

The Arizona Science and Technology Education Partnership AzSTEP began as a statewide partnership between the ASU physics department and high school physics teachers to support the teache

School - University Partnerships for Science and Mathematics Reform

A one-year proposal for $100,000 submitted in Dec 2001

to the Arizona Board of Regents: Eisenhower Math-Science Program

David Hestenes, Research Professor of Physics, ASU

Jane Jackson, Academic Associate, Department of Physics, ASU

480-965-8438, Jane.Jackson@asu.edu

Final Report by Jane Jackson June 2003 (revised and updated in fall 2008, with underlining

to highlight Mesa Public Schools activities and results, since that was our primary partner district)

Introduction and overview:

The idea that K-12 schools need help from universities to upgrade the science/math curriculum has emerged as a major initiative in the National Science Foundation: the Mathematics and Science Partnerships Program It provides substantial federal funding for the

kind of school–university partnership created in this project Such school–university partnerships provide schools with a much-needed mechanism for continuous research-based science/math education reform – – perhaps an essential mechanism

Schools and school districts are ill-equipped to conduct science and math professional

development on their own, because they lack necessary expertise in mathematics, science and technology as well as resources to keep up-to-date with advances in science and math curriculum

materials and pedagogy Those resources reside primarily in the nation’s universities

It is imperative that we learn how to make such partnerships work effectively as soon as possible so that federal money will be well spent and Congress will have justification for continuing this program For this reason we did not wait until the new program took effect last year; we applied immediately for this “proof of concept” project on partnership design and implementation, with high potential to serve as a model for other universities As we have been laying the groundwork for statewide partnerships for a decade, we are well prepared to get Arizona partnerships up and running rapidly

The Arizona Science and Technology Education Partnership (AzSTEP) began as a

statewide partnership between the ASU physics department and high school physics teachers to support the teachers as local leaders of K-12 science and technology education reform In the four years since it started, AzSTEP has engaged half of Arizona high school physics teachers in Modeling Workshops Physics teachers cultivated in AzSTEP provide a cadre of leaders for expansion of AzSTEP to a statewide network of school district - university partnerships

• Immediate Goal: To set up trial school-university partnerships in local public schools and

demonstrate the effectiveness of this mechanism for providing school districts with access to university resources to drive science/math/technology education reform

• Long-range Goal: To establish the Arizona Science and Technology Education Partnership

(AzSTEP) to broker school-university partnerships statewide and secure their funding to support continuous upgrades in science/math/technology education

Although ultimately science and mathematics reform must encompass the entire twelve years of schooling, this project addressed only one component, albeit a crucial one: the need for

an integrated physical science and mathematics workshop for seventh through ninth grade teachers Among other things, it established a much-needed curricular link between high schools and their feeder schools Physical science courses in middle school and ninth grade must provide students with the conceptual underpinnings needed for success in high school physics and chemistry At the same time, these courses must develop basic scientific literacy for students who will not go on to physics and chemistry

As AzSTEP's goal is to establish a permanent mechanism for statewide reform affecting all students, we targeted districts with the greatest likelihood of success in this “proof of concept” project Participating school districts were Mesa Public Schools, Peoria Unified, and Glendale Union High School District (GUHSD) with its two feeder districts, Washington (WESD) and Glendale Elementary (GESD) They were selected on the basis of administrative commitment and teacher qualifications Administrative commitment is, of course, essential for district-wide reform But the most important requirement is the existence of a cadre of teachers capable of leading implementation of the program Each of the selected districts already has such

a cadre, composed of high school teachers who have been active in the Modeling Program for years and have shown that they are fully attuned to the aims and methods of the project In fact, their reputation for outstanding teaching was the most important factor in gaining administrative support from their districts Thus two expert high school teachers, one in physics and one in math, were selected to lead the appropriate district-wide workshop In Glendale, two workshops were held, one including high schools to which GESD feeds and the other similarly with WESD

Each district pledged to contribute $10,000 or more for teacher stipends In fact, Mesa and GUHSD contributed considerably more than this, whereas WESD contributed considerably less Each district applied for and was awarded an Arizona K-12 Center "Fast Track" grant of

$10,000, since AzSTEP was a pre-approved provider of this Modeling Workshop

Evidence of school district needs:

Mesa Public Schools: Mesa is the third-largest city in Arizona It has experienced explosive growth in recent years With 74,100 students in more than 90 schools and alternative K-12 programs, Mesa Public Schools is the largest district in the state of Arizona It is now an urban district, with sections of the city inhabited almost exclusively by ethnic, mostly disadvantaged families Hispanic and American Indian youths constitute almost a third of the student population, and 45% of Mesa’s students qualify for free or reduced lunch subsidies

District data indicate that ethnic children in Mesa’s economically disadvantaged regions perform poorer on measures of educational achievement As of September 2002, 18 of Mesa’s 35 Title I K-12 schools failed to make Annual Yearly Progress (AYP) for two years or more,

subject to state intervention according to new regulations in President Bush’s No Child Left Behind legislation These data underscore the profound increase in diversity, and the

accompanying needs, of many students and schools in the district If unchecked, this situation will result in significant numbers of students failing to meet minimum competencies, in turn qualifying Mesa as a “failing” district according to the new law Thus Mesa’s school community

is motivated to find ways to help the district’s lowest-achieving students

Similar data and needs exist in WESD, GUHSD, and especially in GESD, where 65% of students are on the free lunch program

Our telephone interviews in spring 2002 with science and mathematics teachers in all

thirteen Mesa junior high schools revealed that few mathematics and science teachers were coordinating their courses Moreover, junior high schools have been experiencing rapid turnover of science teachers, with the result that a preponderance of teachers are inexperienced

in use of technology in instruction

GUHSD: Our district-wide survey of mathematics teachers in August 2001 revealed that

only one-third of 34 ninth grade math teachers took physics in college For the most part, articulation in mathematics and science was lacking between GUHSD, GESD and WESD.

GESD and WESD: In spring 2001, only 3% of Glendale Elementary District eighth graders passed the mathematics section of the AIMS test Graphing calculators were not used in

most middle schools, thus depriving students of an important representational tool for learning mathematics

Peoria USD: In surveys returned to us by 42 science teachers in grades 6, 7, and 8 in Aug 2001, the median reported number of hours per year of school inservices related to science

or math content was zero! Only half of the teachers have taken a college-level course in physics

or earth science, and only one-fifth had astronomy Most high school teachers of physical science majored in life science, and some have only a minor in any science Fewer than one-third

of ninth grade science teachers have had a calculus course

Development of workshop:

Larry Dukerich, the Mesa Modeling Workshop leader, described development of the Modeling Workshop in an e-mail to Jeremy Roschelle of SRI International (Jan 4, 2003) Dr Roschelle is

a collaborator with Professor James Kaput in the SimCalc mathematics project at the University

of Massachusetts - Dartmouth

"The course development for Models of Physical Science originated years ago when physics teachers in Modeling Workshops attempted to prepare a set of remedial materials for under-prepared physics students patterned after the efforts of Jim Minstrell and Arnold Arons Teachers found these to be of use for their ninth grade physical science courses, where, as you likely

know, the textbooks are abysmal David Hestenes challenged Jeff Hengesbach (now at the Webb School in Claremont, CA) and me to flesh out the course so that it would weave the threads of modeling tools, structure of matter, and energy into a coherent course that could be used to prepare students for a physics-first course in ninth grade The design specifications also called for greater integration with the math curriculum for eighth and ninth grades David strongly recommended that we work Sim-Calc materials into the course design We met with you up at NAU a year ago and we went to work.

The task was simply too large for two teachers working part-time to complete, but we finished the first three units well enough (we think) that the materials could be used for the Physical Science-Math Modeling Workshops that you and Jane discussed Since our primary goal was to develop a physical science course, we chose those Sim-Calc materials that seemed most relevant

to our needs in the first unit."

Recruitment of teachers:

In spring 2002, each of the five school districts recruited teachers using a two-page application form Leaders were sought: stated criteria were, for example (GUHSD): "Preference will be given to applicants showing the greatest potential for improvement of student learning, implementation of technology and modeling instruction on a regular basis, and leadership at the local school for implementation of technology."

Pledged commitments were, for example (Peoria):

* Are you willing to give a 1/2 hour standardized test next year as a pre-test and post-test?

* Are you willing to serve as a resource in your school for science & math teachers, both on an individual basis and in conducting training sessions on classroom technology for groups?

* Are you willing to be actively involved in a learning community of physical science and math teachers?

GUHSD asked principals to choose a team of a math and a science teacher from among applicants, and to try to provide common planning period Other districts recruited too late for this

Summer modeling workshops:

Ninety-two teachers participated in four three-week Modeling Workshops in June at:

* Independence High School: 11 GESD teachers, 10 GUHSD teachers

* Washington High School: 12 WESD teachers, 11 GUHSD teachers

* Dobson High School: 23 Mesa teachers, 1 from Desert Eagle charter school, 1 from Gilbert HS

* Peoria High School: 20 Peoria teachers, 1 from Higley HS, Morenci HS, AZ School for Arts

All teach grades 7, 8 or 9 except for six lower-grades GESD teachers and a few sixth grade Peoria teachers Half of the 92 teachers teach science, 35 teach mathematics, and the rest teach both or all subjects Forty-two schools were represented in the five districts, and almost two-thirds of the schools had two or more participants Six teachers at Cholla Middle School in WESD participated, including almost the entire science department

All workshops except Dobson High School met for six hours daily, with a total of 90 contact hours in 13 to 15 days Dobson had four hours each afternoon for 15 days, for most Mesa

math teachers were committed to teach in mornings The half-day workshop design was best: teachers worked effectively and were still enthusiastically learning in the third week, whereas teachers in the three all-day workshops were tiring and losing focus.

Since optional ASU graduate credit in physical science or mathematics was available, teachers were assigned homework such that the total time devoted to the course by a participant was at least 135 hours as required by the Arizona Board of Regents for a three-semester hour course

Workshop leaders were expert high school teachers in the respective districts, one each in physics and chemistry (listed first) and mathematics

* Independence High School: Hal Eastin and Cheryl Bryant

* Washington High School: Sheila Ringhiser (with Dawn Harman) and Veronica Carlson

* Dobson High School: Larry Dukerich and Stella Ollarsaba

* Peoria High School: David Hill and Curt Aylward

Workshop leaders communicated regularly via e-mail, sharing successes, problems, and strategies

The workshops provided teachers with education in standards-based content and instructional strategies ASU is well-qualified for this: in 2001 an Expert Panel of the U.S Department of Education designated the Modeling Instruction Program as one of two Exemplary K-12 science programs in the nation, out of 27 projects reviewed Ratings were based on these criteria: (l) Quality of Program, (2) Educational Significance, (3) Evidence of Effectiveness, and (4) Usefulness to Others In 2000 a different Expert Panel of the U.S Department of Education designated Modeling Instruction as one of the seven best K-12 educational technology programs

in the nation (out of 134 applicants!) Links to panel reports are at http://modeling.asu.edu

Participants were introduced to the Modeling Method as a systematic approach to the design

of curriculum and instruction The name Modeling Instruction expresses an emphasis on making and using conceptual models of physical phenomena as central to learning and doing science Adoption of "models and modeling" as a unifying theme for science and mathematics education

is recommended by both NSES and NCTM Standards as well as AAAS Project 2061 However,

to our knowledge, no other program has implemented it so thoroughly Modeling workshops meet or exceed many Arizona professional teaching standards, and grade eight and high school performance objectives in the science and mathematics standards

The essence of Modeling Instruction is that content is reorganized around basic models to increase its structural coherence Student activities are organized into modeling cycles, which engage students systematically in all aspects of modeling The teacher guides students unobtrusively through each modeling cycle, with an eye to improving the quality of student discourse by insisting on accurate use of scientific terms, on clarity and cogency of expressed ideas and arguments After a few cycles, students know how to proceed with an investigation without prompting from the teacher The main job of the teacher is then to supply them with more powerful modeling tools Lecturing is restricted to scaffolding new concepts and principles

on a need basis Details of the modeling cycle are at http://modeling.asu.edu/modeling-HS.html

In Modeling Workshops, participants are supplied with a complete set of course materials (resources) for a semester, and they work through activities alternately in the roles of student or teacher

Workshop goals for teachers were to:

* improve their instructional pedagogy by incorporating the modeling cycle, inquiry methods, critical and creative thinking, cooperative learning, and effective use of technology in instruction,

* understand content in structure/properties of matter, motion, energy, scientific thinking skills, and related skills in each of the six Arizona Mathematics Standards (Strands),

* strengthen coordination between mathematics and physical science

Anticipated student outcomes included improved understanding in geometrical and physical properties of matter, motion, energy, graphing, and related mathematics and reasoning skills such

as measurement, proportional reasoning, and relation between graphs and equations

Thematic strands woven into this course included scientific modeling, structure of matter, energy, and use of calculators and computers as scientific tools Mathematics instruction was integrated seamlessly throughout the entire course by an emphasis on mathematical modeling The course included these models and modeling activities:

1 Modeling geometric properties of matter: length, area and volume

2 Modeling physical properties of matter: mass and density

3 Model of a point particle with constant velocity

4 An atomic model of solids, liquids and gases

5 Energy and the states of matter

Larry Dukerich reflected on his workshop experience in his e-mail to Jeremy Roschelle

of SRI International (Jan 4, 2003):

" The three-week summer workshop was an eye-opening experience for the workshop leaders as well as the participants Participants began to realize that it was possible for junior high math and science teachers to use common vocabulary - that the math teachers could do activities and the science teachers could use TI-83+ calculators (it was as if some taboos had been lifted) The workshop leaders became more painfully aware of how fragmented our math and science curricula were Due to the pressures placed on math teachers for their students to perform well on the state-mandated test, it's as if blinders have been placed on the majority of math teachers For many, their strategy has been to prepare for the test rather than to try an approach that stressed a conceptual understanding rather than drill-and-kill The workshop

leaders (and the math teachers who got it) tried to convince the rest that more iterations of what hadn't worked in the past wasn't going to lead to success Here's a sample exchange: Traditional teacher, "How are we going to do this [Sim-Calc approach to slope] AND what we usually do?" Reformed teacher, "No, you're going to do this INSTEAD of what you've been doing."

Unsolicited comments by teachers, soon after the workshop ended (via e-mail):

• I really enjoyed the Math & Science Modeling Workshop Hal and Cheryl did an excellent job!! I learned so much I have so many new and exciting modeling lessons and materials to share with my students this upcoming school year.

• Had to let you know how much I enjoyed the class Larry and Stella did a fabulous job in every way Thanks for getting me involved in this venture my fear of physics has been greatly reduced and I intend to continue with the program and your suggestions.

Teachers' workshop evaluations, measurement of teacher learning by site:

Workshop rating (10=superb): Peoria 9.7,WESD/GUHSD 9.3, Mesa 9, GESD/GUHSD 6.2 GESD presented difficult challenges They had to select six teachers of grades four to six because 7th & 8th grade teachers taught summer school This slowed the pace of the workshop, and the classroom needs of the lower grades teachers weren't met because the level of activities was too high for them On the other hand, although the content and level were appropriate for the ninth grade teachers, the pace was too slow for them Also, one workshop leader, a novice leader, worked diligently and tirelessly but was less than successful because of a leadership style that is more conducive to working alone rather than with a co-leader

Physical Science Concepts Inventory (PSCI) teacher pretest & posttest mean scores:

All teachers were pre-tested and post-tested on the Physical Science Concepts Inventory (PSCI) and the Math Concepts Inventory (MCI) These instruments for students were developed specifically to align with this course The first eight questions are in pairs, from the Classroom Test of Scientific Reasoning, a research-based test developed in the 1970’s by Anton Lawson, on

which extensive student data internationally are available (some are cited below) Most others are released NAEP and TIMSS questions, and questions similar to the NCLB-required Arizona high-stakes math test for eighth and tenth grades, the AIMS test Considerable overlap exists between the two inventories, for it is intended that the PSCI be given to science students and the MCI to mathematics students The PSCI and MCI were last revised in June 2003, as a result of these workshops; they are in pdf at http://modeling.asu.edu/MNS/MNS.html The password to open them can be obtained by e-mail to Jane.Jackson@asu.edu (The most important revision is that the harder pair of the four proportional reasoning questions, #7 and 8, was deleted.)

Teachers began the workshop with poor understanding of matter and energy; and teachers with lowest pretest scores improved most, in general In Mesa, the pretest mean was 69%,

posttest mean was 80%; WESD/GUHSD: pretest 73%, posttest 81%; GESD/GUHSD: pretest 69%, posttest 77%, Peoria: pretest 66%, posttest 83% All teachers at each site except Peoria took the posttest on the last workshop day The Peoria posttest was given at the Saturday

follow-up session in January; only six Peoria teachers' names could be matched pre- and posttest (Appendix A shows each teacher’s scores; it is available upon request to Jane.Jackson@asu.edu) Content subscales of PSCI (but not determined for Peoria, since the sample was small) are:

* Conservation of mass (questions 1 & 2): everyone got these right on the posttest.

* Conservation of volume (questions 3 & 4): Eight out of 65 (12%) got #3 and #4 wrong on the

posttest Five of the eight were from Mesa

* Proportional reasoning (questions 5-8): at each site, the mean pretest score was about 75%;

mean scores improved to about 85% on posttest Most teachers answered all four questions right

on both tests, but several teachers answered all questions wrong on the pretest and improved to

one right, indicating that they do not understand even the basics of proportional reasoning.

* Graphing skills/motion (four questions); at each site, pretest was about 90%, posttest about

95%

* Geometrical & physical properties of matter (four questions): Mesa started lowest, at 70%;

WESD/GUHSD started highest, at 90% Mesa and GESD/GUHSD ended at about 85%

* Atomic model of matter (six questions): at each site, pretest and posttest mean scores were

about 65% However, actual teachers' scores were diverse, from none right to all right, with little improvement by anyone

* Energy and states of matter (four questions): Tests showed tremendous improvement in most

Mesa teachers' scores; moderate or little improvement in individual teachers' scores at the two

Glendale sites At each site, the mean pretest score was 40% Mesa teachers improved by far the most, to 75%; WESD/GUHSD to 50% GESD/GUHSD didn’t include this content because of

slower pace due to six elementary school teachers; their mean posttest average was 45% Teachers' scores were diverse in the pretest at each site, from none right to three right

In December 2002, after I related the above results to Larry Dukerich, he wrote to me,

"I would like to caution you against reading too much into the test results from the Physical Science Concepts Inventory Unlike the FCI [Force Concept Inventory], which was developed over years based on interviews with students to come up with plausible distracters, this test is an amalgam of items selected ad hoc from a variety of sources with different goals in mind This is not to say that the individual questions are not good, but that this collection of loosely related items does not constitute a coherent test design.

Drawing conclusions about the relative effectiveness of the instruction in the four workshops may be inappropriate given the test limitations Here's my analysis of the sub-groupings in the questions.

1 The questions borrowed from Lawson's Classroom Test of Scientific Reasoning (items 1-8) have been banging around for quite some time The majority of the teachers already knew how to do items 5-8 Those who did not, showed only slight gains primarily because there was little instruction on proportional reasoning in the workshop

2 Items 9 - 12 [graphing/motion] were too easy for teachers of 8th and 9th grade - hence they fail to discriminate between teachers who already understood the concepts and those who learned as a result of the workshop.

3 On the positive side, I believe that questions 13-15 and 26 (geometric properties of matter) could be used to test how much of an effect the workshop had on teachers as this concept was explicitly covered in the workshop.

4 The items on the atomic model of matter are questions that probe deep misconceptions about the atomic model in much the same way as does the FCI I am not surprised that, given the short duration of the workshop, that there was little or no gain This should tell us more about how we have to structure the workshop in the future.

5 The relative success of the Dobson [Mesa] workshop teachers was due, in great part to the fact that we got through all three units, whereas teachers in the other workshops barely got

to energy, if at all Stella told me that I was a slave driver, as I pushed the teachers to complete

the materials, knowing that the energy treatment would be novel for most of them I have confidence in these items simply because I know how long and hard Gregg Swackhamer worked

on them.

So I urge caution in using these test results to draw conclusions about the workshops."

Follow-up meetings, and reflections by workshop leaders:

Each district planned and funded its own follow-up events, subject to AzSTEP's request of at least two follow-up events of significant depth, preferably totaling ten hours

GESD:

In August, most GESD teachers met for six hours on a Saturday with Hal Eastin, a workshop leader, to develop lessons for their classes In spring 2003, Renee Kopcha, the GESD curriculum coordinator, organized three afternoons of classroom teaching demonstrations followed by discussions The workshop co-leader, Cheryl Bryant, reported that these model teaching meetings were valuable, professionally and personally She wrote, "Some GESD teachers do amazing things with their students! Very good structure, policy, and procedures! My students can't fool me any more, when they tell me they weren't taught well at GESD Unfortunately, GESD has huge turnovers of students, teachers, and principals; this is not conducive to student learning!" Cheryl said she's convinced that a key to improved teaching is for teachers to observe other teachers, followed by reflection and sharing of ideas A great need exists for this in GESD, she said, for teachers are isolated, unlike at GUHSD, where a mentoring program is in place at each of the nine high schools

GUHSD:

Debi Plum, GUHSD mathematics coordinator, organized a full day of follow-up in October, including pay for substitutes for 18 teachers and five workshop leaders Since ninth grade science is primarily earth/space science, not physical science, workshop co-leader Dawn Harman presented modeling activities in astronomy Teachers worked on the following technology and modeling instructional techniques:

Implementation discussion – what’s going well? What are our concerns?

Moon and daylight graphing activities

Downloading data to Graphical Analysis software

SimCalc MathWorlds software refresher

Parachute activity (modeling motion using CBR, presented by co-leader Veronica Carlson.)

GUHSD teacher teams were asked to plan two lessons together, observe their team member's teaching, and return lesson planning/observation report forms to the district office

Mesa:

On Nov 23, Mesa held the first of two Saturday morning meetings with workshop leaders Larry Dukerich and Stella Ollarsaba; ten teachers participated Agenda:

* Midcourse correction: testimonials, difficulties

* Math and energy activities

* Expressions of desired future activities Teachers proposed these: "Get together as a math/science team and focus on a concept (e.g., proportional relationships) in which you can discuss the vocabulary, topic and ways that each subject matter can augment the other."

"Develop parallel worksheets from the curriculum that can be used in science/math."

Representative comments at this meeting by teachers were compiled by Larry Dukerich.

* "The workshop brought about a change in my teaching style This is shown in the increase in

my students’ enthusiasm for science They hardly ever work alone now They are much more

engaged My students’ test scores also show improvement."

* "Big whiteboards really promote interaction between the students While one is writing, the others are helping and checking." "While working in groups, the students are debating and

critiquing each other Oral and written communication skills are being improved."

* A math teacher was so enthused that she and her principal made sure that every one of the math teachers in her school had a classroom set of whiteboards

* They all felt that they knew enough about whiteboarding that they could implement this aspect

of modeling in their classrooms

* "The workshop experience changed the way I look at science in general I find myself looking for the models in the rest of the ninth grade science curriculum It helps me realize the way students learn so that I am a more effective teacher."

* Several teachers reported that they are “converting” colleagues who didn't participate

* The level of the activities was too high for seventh grade students

* Everyone expressed the concern that what was expected of them (either preparing for district tests or conforming to the existing curriculum) hindered their ability to implement as much of the

materials as they would have liked

* Several teachers commented that it would have been much better if a team of math and science teachers from their school had participated in the workshop

Peoria:

Peoria funded 10 hours follow-up at $22/hour, including an eSchool using Blackboard.com and a four-hour Saturday meeting in January with workshop leaders David Hill and Curtis Aylward (David Hill is the science facilitator for Peoria USD, in addition to teaching physics full time.) David reported that the in-person meeting, which was attended by seven participants, was more successful than eSchool, in part because eSchool is a new modality for teachers, so it was hard to get them to use it to ask questions of the instructors and to share ideas The agenda for the meeting included issues of teachers, using technology, and changes in activities and sharing of experiences Teachers expressed frustration at the district's slowness in installing Graphical Analysis on networked computers, and science teachers were frustrated at not being given commitment for time to do science activities because of focus on the math AIMS test On the other hand, math teachers found creative ways to use Modeling Workshop activities to teach Arizona math standards, not by rote, but by science applications

David Hill noted that the chief value of the Modeling Workshop was in process skills, rather than content Teachers reported to him that their classroom has become more student-centered Rather than students just doing a lab, filling out a worksheet, and turning it in, they engage in whiteboarding and Socratic dialogue teachers ask students more questions Rather than the teacher giving a definition and doing a demonstration, followed by student verification, children develop concepts through experimentation and discourse/reflection (This is not quite model development, he noted.) He said that it was hard for 7th and 8th grade science teachers to directly implement most Modeling Workshop activities because 7th grade is earth science and 8th grade is life science Sixth and 9th grade are physical science, so these teachers had no problem, he said David recommended that, since fewer than half of the teachers were able to attend the well-publicized Saturday meeting, early-release time or two after-school meetings would be better WESD:

In early August, WESD had two full days of follow-up with workshop leaders Sheila Ringhiser and Veronica Carlson to integrate workshop learning into their curriculum Teachers wanted more follow-up sessions to strengthen their skills, but in mid-year the curriculum coordinator, Jennifer Cruz, took another job in the district Thus follow-up sessions did not occur, to the disappointment of some teachers

Listserv for participants:

I started a listserv for all 92 participants, workshop leaders, and district coordinators Hardly anyone ever posted I posted occasionally on resources: how to borrow classroom sets of TI-83s from Texas Instruments, how to write a $500 grant to the Wells Fargo Teacher Partner program, recent research on 'not giving the answer' to middle school students, exemplary middle school science programs, error-laden middle school science textbooks, and the like

Implementation: unsolicited comments by teachers (via e-mail):

• "I introduced whiteboarding and the TI-83 into my classroom for the first time this year I cannot express strongly enough what a positive impact this has had on my classroom! Wow!

My students love sharing their work through whiteboarding and it has been an effective method for me to encourage more risk-free participation in cooperative groups In fact, I

keep spreading the word about how well it works and have even got my best friend whiteboarding in her English class over at Metro Tech! In addition, the TI-83 has completely captured my students' imagination."

• (Received in May 2003) "I would have sworn that I didn't use much of the material I learned last summer, but then I watched my kids take that test I had many copies so I let the kids work on the test[sheets] They were on task during the test (I was in shock!) and then when I looked over the tests, I realized that each child had attempted every problem and that they had a method of approach for each problem They extended graphs, drew pictures and wrote notes to themselves I was floored:)" [Note: this experienced mathematics teacher reported

little implementation but had a great deal of enthusiasm for the workshop Her students showed little gain on the Math Concepts Inventory.]

Measurement of student learning, by school district:

I preface this section by quoting Larry Dukerich (e-mail to me, June 2003):

"I am hesitant to draw far-reaching conclusions about the merits of individual questions (or categories of questions) based on the scores of some teachers in Mesa who frankly admitted that they were unable to implement the curriculum in any meaningful way I cannot tell you how dismayed I was to learn (in our follow-up sessions) that the teachers could only do this or that activity here or there because they felt compelled to cover all the topics required of them in order to prepare their students for the end-of-year tests Having spent a significant chunk of a year devising a set of activities designed to carefully build skills and a coherent view of the atomic model of matter and energy, I felt as if it had been a real waste of time Some of these teachers may as well be a control group!

So, I would do my best to compare gains to the teachers' self-rating of the degree to which they were able to implement the curriculum they learned - just as we did with the Modeling Instruction in High School Physics workshops It seems reasonable to me that one would only find gains for students in classes in which teachers were able to implement the curriculum in a coherent way."