Development of a Student Writing Skills Programme for First Year UG Students - Final Report

26 I had received a first on my [previous semester’s] essay … and believed the portfolio would be a waste of time … However after the first five weeks my opinion began to change … [My pe

Report for English Subject Centre and Keele Innovations Fund

Susan Bruce,Keele University,

29th September, 2003

Write Through The Semester Report for English Subject Centre and Keele Innovations Fund

Organisation is another notion which this portfolio has

lead [sic] me to discover For example the portfolio

instructs that week 6 requires “a plan of your assessed

essay” My jaw fell in utter disgust along with the rest of

the student nation on receiving such a blow Surely

week 6 is intended for far more testing activities in

creation such as constructing skyscrapers out of beer

cans as opposed to a sterile essay plan, my heart sank

However, the truth of it is that as a result of the portfolio

my essay began in week 6 and was completed in plenty

of time (at least not the night before!) Such planning

ahead is now firmly embedded within my student moral

principles (along with never refusing a free drink and

always using a tea bag at least twice) Another

advantage in this early bird approach was plenty of time

in which to do other work which I didn’t start in week 6

(and in retrospect perhaps should have) (47)1

I have a complete lack of knowledge about many of the

later plays, but it is all right because I know how to

evaluate a website … it is a futile way of making me

work The Education system has constantly

disappointed me, by forcing me into restrictive ways of

performing tasks This … has destroyed my creativity

and stifled any talent that may have been growing It is

only recently that I have gained the confidence to shake

off the shackles of an oppressive education and stand

up for what I think These methods of working that are

forced upon us students in the form of this ‘portfolio’ just

by the students The numbers in brackets after them are the numbers assigned to the essays by the contract researcher who analysed the batch of essays.

serve to aid in the corruption of free-flowing thought …

This method … has caused my level of competency of

essay writing to regress somewhat … I have probably

reduced the number of marks available to me by not

supporting the course, but I do feel much better for

venting my annoyance (26)

I had received a first on my [previous semester’s] essay

… and believed the portfolio would be a waste of time …

However after the first five weeks my opinion began to

change … [My peer group was not working well so] in

order to … receive some response to my essay I

exchanged work with my friend who is also on the

course, but not in my group … [WTTS] has made me

appreciate that at this level, we are all able to contribute

some level of original thought to the work that we do (18)

Table of Contents.

1 Description of the Project 4

2 Description of the Evaluation of the Project

4 Conclusions of Pilot Year Evaluation and Amendments to

the project in its second year

Appendix Two: Reflective Essay Evaluation: Detailed Results and

detailed student comments

21

Appendix Three: Proformas for Self/ Peer Assessment

26

Write Through The Semester Report for English Subject Centre and Keele Innovations Fund

1 Description of the Project.

Write Through the Semester is an Introductory Writing Course which aims tohelp students improve the literary skills so many of them currently lack on entry to HE, and thereby allow academics to spend more time on content and less on form in their responses to their students' writing Research suggests that student writing improves most markedly when writing tasks arelinked directly to the students' discipline(s), and that students engage most effectively with individual tasks when the relation of those tasks to an end product is clear We tried, therefore, to develop a model which would be embedded in a specific discipline, piloted in a module in English, yet

adaptable to the requirements of other disciplines inside and outside our own institution There were, however, a number of difficulties which needed to be taken into account in planning the course Self-evidently, improvement in students' writing demands that they write regularly, often, and reflectively But there are only very limited extra resources available to invest in the teaching of such skills, and, currently, none to be invested in any extra

marking of student work Similarly, student time is also limited and

consideration must be paid to the danger of producing for the students an excessive workload The challenge then was to formulate a writing course which would:

 be directly related to the material the student is studying in a given

module;

 get students writing weekly;

 encourage students critically to reflect on the act of writing on an ongoingbasis;

 encourage students to evaluate their writing and the writing of others on

an ongoing basis;

 avoid an excessive workload for the students taking the course;

 bear an obvious relation to the end product of that module (the assessed essay);

 be rewarded at a level commensurate with the effort expended on the assignments by the students;

 operate insofar as is possible with minimal weekly intervention from

tutors;

 result in a comparable marking load to that generated by more

conventional modes of assessment;

 avoid, insofar as is possible, inflation or deflation of average grades for the module as a whole (this is also important for the assessment of any change in the quality of student writing)

Our solution to these problems was to devise a series of short assignments, which have intrinsic and collateral value in themselves, but which also build up progressively to the submission of a first draft of an assessed essay

at the beginning of the second half of the module, and then, by a series of peer- and self-evaluations, to a revision of the assessed essay The final assignment in the initial year of the project was a short reflective essay, which evaluated the usefulness of the project from the student's point of view Throughout the semester, students were required to exchange their writing, every week, with two (or in some cases three) of their peers, and to deposit copies of their writing in a portfolio which they eventually submitted totheir tutor The mark scheme for the module was devised to ensure that students could not pass the module without submitting the complete portfoliobut were also rewarded for the work they did in completing it The mark scheme devised ensured that tutors awarded marks only for qualities they already felt comfortable in judging (in this case, for example, hard work; and the argumentative essay); and that any change to the average grade of students taking the course would reflect change in the quality of their writing

2 Description of the Evaluation of the Project.

The project was extensively evaluated in the first year We tried to maximise the objectivity of the evaluation by embedding in the project’s plan a clear distinction between the person responsible for devising and instituting WTTS (Susan Bruce) and the person responsible for evaluating its success (Monica McLean) In addition, we hired contract researchers to undertake substantial statistical analysis of the students’ responses to WTTS, as well as to conduct

focus groups and interviews with those who were teaching on the course in 2002.

The evaluations undertaken were as follows:

1 A mid-semester questionnaire, distributed to all students in class

around week 6, which aimed to ascertain: the use the students had made of the Handbook; their level of confidence about their writing prior to taking WTTS, and after it; what they liked and did not like about WTTS; whether any problems were occurring with its delivery (and if so, what they were)

2 An obligatory reflective essay as the final course assignment,

entitled, ‘”Writing this Portfolio was a Complete Waste of Time”:

Discuss’ All these essays were read by Susan Bruce She produced

a chart detailing what she thought were the most common remarks, with the help of which a contract researcher analysed all the essays and drew the information together into statistical conclusions

3 Focus Groups were conducted (by our contract researchers) with all

of the tutorial groups involved with WTTS in its pilot year

4 Interviews with all tutors teaching on Elizabeth Tragedy (2 f/t staff

members; 2 postgraduate teaching assistants) were conducted by one of the contract researchers, using questions devised by Monica McLean

3 Conclusions of Evaluation of Pilot Year

3.1 Mid-semester Questionnaire.

Detailed results of the answers the students gave to the questionnaire are available in Appendix one The most notable results were in the area of the students’ perception of their confidence in their writing skills; in the students’ engagement with the course handbook; and in their reaction to the optional workshops

Confidence The students’ confidence in their writing skills appeared to be

significantly improved by having taken the course Thus whilst 54% of the respondents said that they felt ‘confident’ or ‘very confident’ about their

writing skills on entering HE, and 57% said that they felt ‘confident’ or ‘very confident’ after completing the English Department’s Introductory course, 85% of the students said that they thought that they would feel ‘confident’ or

‘very confident’ after completing WTTS Just as marked were the relative percentages of students who felt that their anxieties about writing had been substantially alleviated by WTTS 23% of students felt ‘anxious’ or ‘very anxious’ about their writing on entering HE This rose to 28% of the cohort after they had taken the Introductory course But only 2% of the students felt that they would feel ‘anxious’ after completing WTTS, and none replied to thisquestion with ‘very anxious’ It would seem then, that the most substantial benefit in terms of confidence in their ability to write was afforded to those students who were anxious about their writing skills (the numbers of those who said they ‘didn’t really think about it’ remained pretty constant between semester one and semester two)

The Handbook A markedly high percentage of the students (93%) stated

that they had read through the entire handbook (and thus would at least haveread material on punctuation; on how to present an essay correctly; on

assessment of the quality of their own and others’ essays and so forth) Of the 7% who said that they had not read through the handbook, most had read the relevant sections to each assignment, and some had skimmed the handbook

The Workshops Together with Claire Slater-Mamlouk of the University’s

Skills Centre we had scheduled two optional workshops for the students

Take-up for these was very disappointing, so poor that we were in the end only able to run one workshop (on thesis sentences; only 7 students turned

up to this) According to the results of the mid-semester questionnaire, however, only 15% of the students had actively decided not to go Most of the rest either intended to go, or were prohibited from doing so through time-table clashes, part-time jobs, or poor time management (Those who did attend the workshop, incidentally, all said on their evaluation forms that it hadbeen useful to them)

3.2 Final Reflective Essay.

The most extensive statistical analysis came from the analysis of the

reflective essays The students were in this assignment free to mention or not mention anything they wanted to within the terms of the essay and thus the percentages mentioning a particular aspect of the course should be understood differently than they would be had a student specifically been asked to comment on a particular aspect of WTTS (as in a questionnaire, for instance) Many students, for example, chose to structure their essays

around the series of assignments which they had been given Thus 93% mentioned the website reviews (which accounted for the first batch of

assignments), whilst only 40% specifically mentioned having to revisit their Yr1 first semester essay (which was only a part of one assignment) 83% of students did not mention the reflective essay itself So, for instance, the fact that 29% of the students requested more tutor involvement in the course does not imply that 71% of the students did not want more tutor involvement, but rather that 29% of the students actively requested more involvement without any prompting

Once again, we have given more detailed extracts from the reflective essays

in an appendix (Appendix 2), and confined ourselves here to the main

conclusions

 A clear majority of the students thought that overall, writing the portfolio had been beneficial to some degree 72% of the essays disagreed to varying degrees with the proposition that writing the portfolio had been

a complete waste of time About 14% of the students thought that it had

been a complete waste of time, and about the same percentage were unsure But of those who said it had been a complete waste of time, a

considerable number were contradictory (for example: ‘it was when

studying my essay more critically that I realised I had devoted too much space to ‘”Hamlet” in both the main body of the essay and the conclusion, and I then corrected this In conclusion, although doing the portfolio this semester did have it’s [sic] advantages such as improving my critical abilities, it didn’t really improve my writing skills’ (15))

 There was huge approval of the idea of working in peer groups 68% said

that they liked this idea, as against 15% who said that they did not But a clear majority thought that peer group work entailed significant problems: 56% thought that the peer group work had not worked, as against 27% who thought that it had

 There was very significant approval of having to start – being forced to start – their assessed essay very early in the semester 64% of the

students independently noted this aspect of the course was something of which they approved; while 10% said they did not like this aspect of the course

 Almost all those who mentioned it said that they wanted the assignments

to be more directly related to the texts 32% of students independently suggested this as a possible improvement to the course

 A significant proportion of students wanted more tutor involvement in the course (29% of the essays made this plea)

 A significant proportion of students mentioned the workload entailed – 46% (not always negatively, however, and often in conjunction with what they believe to be a heavy reading load for the module (one or

occasionally two plays per week))

Statistical information needs to be read in conjunction with the things the students actually say Because we undertook such extensive evaluation of WTTS we have reproduced only a relatively small sample of student

comments here A more extensive selection of comments is reproduced in the appendices

From mid-term evaluations

 ‘Have started essay already – really pleased’

 ‘I am more critical about my writing, especially my grammar and

vocabulary’ (N.B neither of these were singled out in the course as

special points of concern)

 'The website reviews seemed quite pointless to begin with but helped in the long run.'

 'Essay grading was really really good as you find yourself picking up on things that you wouldn’t in your own You look at your own more

objectively.'

 'I didn’t enjoy the Internet Detective it was quite tedious and didn’t feel as

if I gained much from it.'

 'I think more about the structure of my essay, the paragraphs and how theessay flows I also pay more attention to my introduction and thesis I think more about the formation of an argument.'

 'It is a useful way of getting students to develop their own writing skills The assignments give it pace, so it is not left until the last minute.'

 'It is a lot of work when put together with all the reading and note-taking

we have to do each week, and I feel that the relevant writing skills can be gained through tutors comments on assessment work and their guidelinesfor improvement in the future.'

 'Commenting on others’ essays highlighted to me what not to do.'

 'Drain on time due to the mind-numbing pointlessness of the tasks which caused me to waste more time attempting to find ways of avoiding the work.'

From reflective essays

 'The portfolio suffers from [two] weaknesses, problems with the order of assignments and a lack of motivation in the students.' (8)

 'Move the portfolio to the first semester.' (8)

 'At first I felt uncomfortable writing comments and criticisms on other people’s work I felt patronising passing comment when my own work was not of a high standard However I think in retrospect, this has given

me more confidence in my own writing, a skill which I can use throughout university.' (10)

 'For the first time ever since I have been writing essays of a higher

standard, my misconception that the essay needed to be written in order

of introduction, main body and conclusion was shattered I actually wrote

my conclusion just after writing my plan, which was then followed by the introduction and I finally took paragraphs from the main body of the text

and wrote these separately before linking them all together I felt

confident for the first time that I could take my essay to pieces and then put it all back together again.' (11)

 'To conclude, I think that this course is generally a success and that all first year English students should take part in it '(11)

 '[The 1st assignments] seemed to have little, or even no relevance to the Renaissance Tragedy course.' (12)

 'The [brevity of the assignments] … helps me to be more concise … My writing is therefore easier to understand and is better structured It

enables me to keep to the specific argument.'

 'Overall I have enjoyed working on the portfolio I feel that the aims of thecourse have worked in my favour and that my writing has improved I am more aware of the quality of writing rather than quantity.' (16)

 'I do believe … that everyone has a different writing style and to try to standardise this is futile '(21)

 'Prior to the internet exercises, I have always been somewhat

embarrassed to ask tutors about punctuation and other points of writing, assuming that they expect their pupils [sic] to already be aware of essay writing skills.' (24)

 ['of nutsandbolts.washcoll.edu] I have revisited this site numerous times now.' (25)

 'My essay is finished a week before it needs to be handed in.'(27)

 'I … found the fact that the portfolio was worth so many marks quite strange, because this also means that you cannot leave it until the end, because it carries 40% of the marks I think this is a little too much '(37)

 'The second website review did give occasion to re-reading last

semesters English essay that was helpful and something I would not havedone otherwise.' (38)

 'Production of a portfolio means constantly writing, something I have found lacking in the English department where only one or two major essays are required each semester This results in most English studentsonly producing written work, or even thinking about their writing style a few times every year What I like about the portfolio is that it encourages the regular review of self, as well as peer, of how and why we write as we

do and what can be improved through this self-assessment Saying this, however, I haven’t found this portfolio particularly encouraging in the growth of my own writing style In the following paragraphs I hope to explain why I think this portfolio has failed for me personally, and the structural problems that I believe are in inherent in the system as peer

reviews [But conclusion says it has been useful, although not to the degree he had hoped] '(41)

3.3 Focus Groups.

We asked our contract researchers to employ the Nominal Group

Technique to ascertain from all of the tutorial groups involved in the

project what they thought that the main strengths and weaknesses of WTTS had been The results were as follows:

Strengths.

 Regular writing very beneficial and an improvement on semester 1, wherethere is no writing [sic; in fact, there is] until the final essay

 WTTS made students more aware of the processes of writing

 Group work was a useful way of encouraging discussion and

collaboration; and also encourages closer friendly relations

 Having to mark essays is useful because it makes students see essays from a tutor’s perspective

 Many students’ time management was vastly improved

 Internet sites gave useful direction to useful sources

 Smaller assignments on a weekly basis prevented work overload

 Smaller assignments forced students to think about how to write

concisely

 Writing the essay was not done in a rush, and students had time to

develop and think about ideas

 The Handbook

 Final product was rewarding

 Made people work harder

 Having to produce plans and essay drafts made students realise how useful they are

Weaknesses/ Suggestions for improvement.

 Some assignments were ‘pointless’ (e.g Internet tasks) Assignments should be more linked to the module’s material

 Students felt that they were not qualified to comment on their peers’ work;

or that they were too embarrassed to make suggestions Some

suggested anonymous marking

 Many found the workload a strain

 Instructions about the process need to be clearer, especially at the end of the course

 Peer groups could be replaced by pairs of students

 Make the assignments fortnightly and longer so that students can practicestructuring essays

 More input from tutors

 Some students do not generally write a first draft and did not like having to

The responses of all three2 tutors teaching on the course were remarkably similar in mentioning:

 That there is a clear need to address the problems students have with writing; and that it is a virtue to attempt to do this systematically in the firstyear

 That the workload for tutors is roughly the same as that generated by a conventional twenty per cent exercise

 That students were concentrating much more on their writing than they would have had they not had to take WTTS

 That students were organising their workloads better as a result of the course

 That peer groups are a good idea and are a desirable way of encouragingstudents to collaborate with one another, but that there are difficulties in making them work properly

concerning the more practical measures of instituting the course (the questions were devised

by Monica and not communicated to Susan in advance).

 There were some anxieties on the part of tutors that students would read less because they were working on the portfolio; but also recognition that this is hard to judge, given that many students read minimally anyway.

Other things that were mentioned were:

 The students think that their writing has improved, though it is difficult to judge from a small sample

 Some students appeared to have made up the draft of the essay (i.e theyhad written their assessed essay out first and had then cobbled together something to make it look like a draft of the essay)

 The assignments should be more tied in with the tutorial work

 The determining grade ought not perhaps to rest on the reflective essay

 It might be better to spread the course over the whole year

 Peer group work should not be anonymised

 Although the workload was comparable to a normal 20% exercise, WTTS did generate more emails than normal from students querying aspects of the course’s organisation

 Move the course to the first semester

 Increase the number of formal exercises the students are asked to do

 Increase the amount of involvement from tutors

4 Conclusions of Pilot Year Evaluation and Amendments to the Project in its second year.

A number of comments and suggestions for improvement emanated from all the various forms of evaluation of the project that we carried out

4.1 Tutor Involvement.

Quite understandably, students wanted more involvement from tutors, both interms of the week to week organisation of the course (that is, in terms of the governing and oversight of the students’ exchange of material) and in terms

of them reading more of the material that the students were producing We were entirely in sympathy with the latter desire: clearly, in an ideal world, it

would be greatly to the students’ benefit to have their tutor read and

comment on their written work more frequently Sympathetic as we were to that desire, however, it remained the case that one of the principles behind the construction of WTTS had been the proviso that tutors could not afford any more time on teaching –related duties, including marking and feedback

So it was impossible to give the students exactly what was requested here

In response to their desire for more tutor involvement, however, I changed the structure of the assignments slightly I dropped the final reflective essay, asking the students instead to write some brief notes about what they found useful or not useful about the process of writing the portfolio (this was

important for two reasons: to preserve a degree of embedded feedback, and

to encourage the students to reflect consciously on what they had learnt or

failed to learn from the project and why) And instead of the essay draft in

week six, I asked the students to write a short version of the final assessed

essay (of around 1,200 words; the final assessed essay is 2,000) At the

bottom of this short version, the students had to include a brief list of 3 things which they thought they needed to improve or work upon further This short draft was handed in to their tutors who marked it, commented on the three points for improvement, and assigned it a grade The grade, however, was withheld from the students until the conclusion of the course, in an attempt to get away from the concentration on grades so prevalent in the current

educational climate, and cultivate instead the perception that all writing can

be improved, however good it is

This amendment to the original assignment structure seemed towork very well when we ran the course for the second time in 2003 The students who took the course this year were fulsome in their appreciation of the usefulness of having detailed tutor’s comments on a draft of their essay (something which does not otherwise happen regularly in the English

department here); and the inclusion of a tutor marked assignment in the middle of the course appeared to allay their anxieties regarding the lack of participation from the tutors, at least to some degree (there were fewer such pleas under this new assignment regime in 2003)

There was some evidence that this new assignment improved the writing of some of the students: comparisons between the draft essay grades of many of my tutees with their final grade for the finished assessed essay showed that a number of students had improved on their performance

to the tune of 5-10 marks (in other words, a class difference, and in a couple

of instances, two class differences) I suspect that the fact that the students were going to go on to write the essay proper shortly afterwards meant that the likelihood of them actually reading what I had written (rather than briefly

skimming the end comment and looking at the grade) was substantially greater than it usually is The other reason why this assignment appeared togenerate better grades, was probably because it was so clearly connected tothe final assessed essay Most available evidence suggests that student motivation is intrinsically connected to the degree to which they perceive a given task to be related to the final substantial piece of assessed work This relation is not always clear: I suspect, for instance, that many of our students

at Keele would be hard-pressed to define the relation between, say a short exercise that counts for 20% and the final assessed essay for the module The students' focus tends to be on the assessed essay which counts for the bulk of the marks ( a focus which is only likely to increase in the future, as education becomes increasingly mechanistic, and increasingly driven by the demands of market forces; and the students’ motivation for what they are doing increasingly driven not by ‘love of their subject,’ which we will have to work ever-harder to cultivate, but by other, less immediately intellectual desires and aspirations)

This assignment change brought other benefits too, beyond thequestion of improvement in student writing Marked writing assignments have

in many institutions become almost entirely summative, and marking

summative assignments is rarely anything other than a chore for most

academics, not least because it furthers a kind of hierarchical relationship between academic and student that many of us would wish to contest most

of us want to talk to our students in an ongoing dialogue, not correct their mistakes This experience was very different: much more rewarding for both parties, I suspect, and for the right reasons, beyond the bettering of the students’ writing skills So from at least one tutor’s point of view (mine), the marking of these short drafts was actually pleasurable (!) as tutor and

student were engaged in a kind of dialogue which has become increasingly difficult to generate in the assessment regimes of contemporary British HEIs

I found myself writing comments such as: ‘do you think you might want to change this?’; ‘this is really good, how might you take this further?’; ‘I agree you need to look at this aspect of the draft, but I disagree with you about this other thing, which I think has real potential’ – but for the first time in years with a real sense that the student might actively and immediately respond tosuch comments

So a better approximation of a more dialogic model of education (however necessarily limited it is these days by the inevitable constraints of ever-increasing student numbers and ever decreasing

academic staff time) was one unanticipated benefit of this new assignment

structure I thought the assignment worked very well, for a lot of different reasons.

4.2 Peer group operation

The students also wanted more week-to-week tutor involvement

in the over-seeing of the peer groups This request was much more difficult

to accommodate, and points also to intrinsic difficulties with peer group

models which I suspect were exacerbated by the pilot nature of this project Many students, in both years, requested pair working rather than peer group working, but for a variety of reasons, this seemed inadvisable.3 There were, undoubtedly, some problems with many groups On the other hand, almost all managed to submit a complete portfolio with at least some work from at least one peer in it Clearly , then, in the majority of cases, people had managed to find ways of working together A number remarked (again in both years) that they had constructed closer relations with people in their groups, as well as, sometimes, with people in other groups; rather

disturbingly, some students in both years expressed the feeling that working

in peer groups would leave them vulnerable to plagiarism by their friends (a depressing model of intellectual interaction, and one which I think we should actively dissuade)

So: there were problems with the working of peer groups; on the other hand there were also benefits I think that in order for peer groups

to work to their optimum potential, more effort needs to be expended on making collaborative working between students a norm of an expectation rather than an experiment (The project, in both years, has been explicitly presented to students as experimental.)4

becomes progressively more difficult to work with, in terms of the numbers of copies of individual assignments that students had to exchange with one another (as well as the cost, for the students, of photo-copying) Pairs presents other problems: if one student fails to turn up, his or her partner cannot then exchanging work Three thus seemed to be the optimum number

rigourously One obvious one is electronic – to deliver it via a VLE such as Blackboard But again, available evidence suggests that useful as they are from the point of view of students, VLEs demand very significant investment of staff time – much more than WTTS currently entails Such investment, with present and foreseeable understaffing in institutions such as Keele, is untenable A possible improvement to the organisation of the course, were it to run

in future years, might be to put students in groups of 3, but ask them to exchange individual assignments with one of the three (a gives to b, b to c, c to a: if only two turn up to a tutorial,