Technical Report: "Using "think aloud" methods to investigate the processes secondary school students use to respond to their teachers' comments on their written work." (1)



Joe Belanger, Ph.D., The University of British Columbia

Philip V. Allingham, Ph.D., Lakehead University



Table of contents

1. Aims and Methodology ............................................................................. 2

2. Findings ........................................................................................................ 2

a. Finding 1: The power of the grade ........................................................... 3

b. Finding 2: (Mis)understanding teachers' comments ............................. 4

c. Finding 3: The importance of early papers in a term ............................. 4

d. Finding 4: Teachers' revisions of students' sentences ........................ 4

e. Finding 5: Editorial usage ......................................................................... 5

f. Finding 6: Assistance in interpreting comments ................................... 6

g. Finding 7: The role of peers ..................................................................... 7

h. Finding 8: Specific teacher comments .................................................... 8

i. Finding 9: Use of previous teacher comments ....................................... 8

j. Finding 10: Praise .............................................................................. ........ 8

3. Conclusions ............................................................................................... 9

4. Epilogue .................................................................................................... 10

References .................................................................................................... 11

Appendices

A. Research Proposal to School Board ................................................... 13

B. Student Questionnaire .......................................................................... 15

C. Revisions Following Pilot ..................................................................... 21

D. Interview Schedule ............................................................................... 23

E. Questionnaire Statistical Data ............................................................. 24

F. The Power of the Grade ........................................................................ 26

G. Barriers Created by Low Grades ......................................................... 31

H. Misconstruing Teachers' Comments ................................................ 34

I. Teachers' Revisions of Sentences ...................................................... 44

J. Teachers' Personal Preferences ........................................................... 46

K. Teachers' Marking Symbols and Abbreviations ............................. 46

L. Peers as Audiences .............................................................................. 47

M. Responses to Specific Comments ..................................................... 48



.

1. Aims and Methodology

The objective of the research reported below was to discover how students responded to the grades, comments, and criticisms which their teachers placed on their written compositions. (Appendix A: Research Proposal to School Board). To do this, we (2) asked teachers to give us compositions they had marked but not returned to students. We then photocopied these papers and returned them to the students individually, observing the students while they read their teachers' comments. We asked them to "think aloud" as they read their teachers' comments, and when they were through, we asked them questions about their feelings and their understanding of specific comments. Fifty-three students were interviewed in this way; interviews averaged 15 minutes in length. We also gave a questionnaire to students in seven classes (just over 200 students) not participating in the observational phase, but problems with obtaining parental permission [a large number of students filled out the questionnaire because it was done during class time, but neglected to return the permission form] reduced the sample size to 119 (Appendix B: Student Questionnaire).

Both the interview schedule (a list of suggested questions and procedures for the interviewers to follow) and the questionnaire were piloted and revised prior to undertaking the study (Appendix C: Revisions Following Pilot). An interview schedule was developed (Appendix D: Interview Schedule). With the students' permission (each student agreed), each interview was tape recorded and then transcribed by the researchers. Questionnaire answers were entered into an SPSS data base and t-tests were used to judge the significance of differences between the responses of boys and girls; native speakers of English and ESL students; those reporting positive attitudes towards English classes and schooling and those reporting negative attitudes. A preliminary report on the findings was circulated to teachers, students, and parents in the final month of the school year, with an invitation to raise questions or share observations. There were no responses outside of "thank you" calls from two teachers.



2. Findings

The think-aloud interviews with students offered insights both into students' affective responses to their teachers' annotations and judgments and into their strategies for improving their writing abilities. The key concept in the interviews was "interest." Students were very interested in the grades placed on their papers, but the grades both captured their interest and distracted them from learning much about becoming better writers. A far less universal interest, but a significant one for those who cared, was teachers' responses to particular ideas in the papers that students found intriguing. Students who had pursued an idea that they found fresh waited for their teachers to take a conversational turn on the topic, frequently in vain. On the other hand, there was a pervasive lack of interest in matters mechanical except as they might be used to argue for a higher grade. While the questionnaires offered support for many of the insights gained from the interviews, the self-reports frequently appeared to be somewhat inflated, particularly regarding knowledge of standard usage which proved to be shaky at best in the interviews. Ten findings from the study are discussed below. Appendices F to M present excerpts from the transcripts of the interviews.

Finding 1. The interviews and questionnaires suggested that the grade on the paper governed most students' responses completely. The importance of the grade is not news to anyone who has spent much time in classrooms, but what surprised us was that the effects of the grade were so sweeping. We would not have predicted that so many students would find the grade a barrier to further reflection on the paper and on the teacher's comments. Most puzzling was that many students who received the grade they expected did not read any of the teachers' comments, despite the fact that teachers had made a number of suggestions that students could apply to future essays. Their rationale appeared to be "the grade confirms my assessment of the writing; since I got it right, there is no point in reading any farther."

However, this contradicts data gathered in the questionnaires, as shown in Table 1 where students strongly agreed with the statement "I always read comments made by my English teachers thoroughly"; the average response was 4.2 on a five-point scale. Table 1 also reports the statistical significance of the differences on three pairings: boys and girls; native speakers of English and ESL students; and students who reported positive or negative attitudes toward school. On all six variables in Table 1 the scores were higher for girls and students with positive attitudes, but only 4 of the 12 comparisons reached statistical significance. Results on the ESL pairings were mixed with ESL students more inclined to ask a friend to interpret an English teacher's comment and less inclined to ask the teacher about his or her comments. None of these comparisons reached statistical significance. To reduce the number of columns required, means and standard deviations for Gender, First Language, and Attitude are not reported in Table 1 or Table 3. These data are found in Appendix E.

Variable n Mean SD Gender Language Attitude
Check grade first 118 4.6 .89 .21 .61 .35
Read passage as well as comments 117 3.7 1.13 .005* .95 .06
Would normally read teachers' comments 117 4.2 .95 .03* .36 .04*
Read comments made by other teachers 118 4.1 .96 .09 .13 .05*
Ask this teacher about his/her comments 118 3.8 1.22 .07 .37 .15
Ask a friend about English teachers' comments 59 2.9 1.34 .19 .06 .84

TABLE 1: Students' responses to their teachers' written comments as reported in their

answers to questionnaires. Likert choices ranged from "1, strongly disagree" to "5,

strongly agree." Statistical comparisons are reported for differences between the

responses of boys and girls, native speakers of English and ESL students, and students

who reported positive and negative attitudes to English class and to school.

In the interviews, students who received a grade higher than they expected predictably spent some time basking in the warmth of the occasion (Appendix F: The Power of the Grade). However, they did not seem to learn much from the teachers' comments: they generally read only the final comment, often more than once, mostly lingering on any praise they found. The higher-than-expected grade prevented students from thinking about how to revise their papers or even from paying attention to suggestions for improvement. Furthermore, even some excellent grades (in excess of 90 percent) interfered with the students' responses because the students were hoping for something a little bit better--such as a perfect score. Frequently, those who read the teachers' comments saw them as subtractive rather than instructive: "I guess that cost me a mark."

On the other end of the scale, students who received a grade much lower than expected did not read the comments either. They were generally agitated and their focus was on planning to ask the teacher if the paper could be rewritten and re-graded (Appendix G: Barriers Created by Low Grades). This suggests that if the students are allowed to rewrite the papers for credit, they will read the teachers' comments more carefully. However, we found no evidence to suggest that those who do not revise will read the comments at all.

As Table 1 shows, on the questionnaires a substantial number of students agreed that they read the passage in their essays which contained errors marked by their teachers (Likert 3.7). The interviews, however, revealed that very few students in fact read the faulty sentences and attempted to see the errors in context. Even those who read the marginal comments often read only the marking symbol (e.g., R.O.) and reported orally the name of the error rather than its application to the paper.

Finding 2. Observing students attempting to unpack their teachers' comments was a study in misunderstanding. Even the students who received the top grades on the papers were unable to decipher what seemed to be the most rudimentary directions. For example, the Grade 12 girl who received the highest grade in her class puzzled over the following diction error:

"...with which her cousin Bellario has equipped [teacher note: "word choice"] her as a

last resort."

The student's response was

I'm kind of confused about what she meant by word choice. I guess maybe because it gets kind of stopped by the flow. I guess that's what she meant. [pause]. OK. I see what she meant; if like a past tense. "Bellario had equipped"; I should have done that. That should have been a "d" there to figure it out. That would have done it. Unfortunately, as the transcripts in Appendix G (Misconstruing Teachers' Comments) show, this kind of elaborate mis-explanation of teachers' code words was very widespread. It was also more common in grades 11 and 12 where students appeared to think that they needed to be able to explain such things (Appendix H: Misconstruing Teachers' Comments).

Finding 3. Students seemed to respond to comments differently at different stages in the school term. Some of the students interviewed were receiving comments from their teachers for the first time while others had already completed several assignments for their teachers. Those students whom we observed receiving an initial response from their teachers tended to examine comments more thoroughly than students who had received previous responses to other writing assignments. The comments students made suggest that they used initial and early papers to try "to figure out" what the teacher wanted and valued in student writing.

Finding 4. Teachers' suggested revisions to students' sentences were seldom appreciated by students and frequently resented. One time-honored method of helping students learn to write is to demonstrate more effective diction or a better way of writing a sentence or two. Students very rarely volunteered a comment about a teacher's revision and when pressed for responses, students were at best neutral ("I suppose it's a better way of saying it"). However, the bulk of the responses were either resentful ("Well, that's her style but it's not mine") or rebellious ("It really picks me when someone rewrites my sentences, as if I can't write"; Appendix I: Teachers' Revisions of Sentences). Students frequently dismissed these-and a wide variety of other comments-as simply being capricious and idiosyncratic: "teachers' personal preferences" (Appendix J: Teachers' Personal Preferences).

On the other hand, teachers seldom wrote the correct spellings of words they had circled as errors, another marking practice of long standing. Since students reported that they almost never looked these words up in a dictionary, perhaps the kinds of editorial help offered by teachers is misplaced.

Finding 5. Neither the questionnaires nor the interviews suggested that many students benefitted from comments on their papers about editorial or standard usage. As Table 2 shows, on the five-point Likert Scale (1=strongly disagree), most students claimed to "know" the terms "fragment," "run-on," "comma fault," "paragraph," and "awkward," but fewer said they knew "pro agr," "tense," "shift," or "s-v agr.". Most of these terms appeared frequently in teachers' annotations of students' papers, suggesting that students were, indeed, expected to be familiar with them. However, during interviews, most students glossed over these errors saying things such as "Comma fault. Yes. I still make them" without indicating that they knew how to correct them. When pressed, students were generally unable to explain what such problems as "agreement" were or how they would correct the errors.

As would be predicted, girls reported greater knowledge of these terms than boys did (boys reported greater knowledge about only "sentence fragments" and "pronoun" case, but the groups were almost equal as the t-values indicate), and students who reported positive attitudes claimed more knowledge of the terms than their less-positive counterparts. However, only two of the twenty gender or attitude comparisons were statistically significant. Native speakers of English reported greater knowledge of terms relating to complete sentences ("fragments," "run-ons" and "comma faults"), but ESL students reported greater knowledge of matters of "agreement" and "awkwardness." In all cases the differences were small which may account for the fact that they were not obvious during the interviews.

Direct instruction in editorial usage appeared to be an important factor in students' understanding. One teacher, for example, assigned each pair of English 12 students one concept in standard/editorial usage to teach to the rest of the class. This was done over a number of classes and appeared to be effective in helping students to understand usage problems. During the interviews we found that students from this teacher's classes had an above-average ability to explain and to correct the common errors in editorial usage.

One standard method used by teachers of English to help students learn editorial usage is to provide students a key to marking symbols, indicating the page in a textbook or handbook where an explanation of a usage error can be found. Students in this study seldom considered seeking textbook advice on items of usage. On the questionnaires, few students reported that they would look up an unfamiliar term (an average of 1.8 on the Likert Scale, the second-lowest response reported in the study; see Table 3). No student interviewed suggested that he or she would check a book to learn about an unfamiliar term. Perhaps representative of many students' thinking, one student regarded looking up an item of usage in a textbook as a challenge to the teacher's authority: "I wouldn't actually go to a book. You sort of trust your teacher to say this is the right way... [but] ...if I read it and it doesn't make sense, I'll go to him and ask and he'll explain why."

Teachers' marking term or abbreviation n Mean SD Gender First Language Attitude
Sentence fragment or frag 111 3.0 1.5 .94 .19 .08
Run-on sentence or r.o. 108 4.2 1.3 .15 .28 .65
Comma fault or c.f. 100 3.4 1.4 .004* .93 .41
Pronoun Agreement 105 2.3 1.3 .30 .09 .37
Paragraph 106 4.4 1.1 .89 .99 .19
Awkward or awk 102 3.5 1.4 .07 .64 .19
Tense 104 3.0 1.5 .14 .22 .48
Subject-Verb Agreement 106 2.8 1.5 .41 .58 .41
Pronoun Case 104 1.9 1.1 .82 .04* .04*
Shift 102 1.9 1.2 .36 .96 .23

Table 2. Standard terms and abbreviations used by teachers to note student errors. Likert choices ranged from "1, do not understand" to "5, understand clearly." Statistical comparisons are reported for differences between boys and girls (gender), native speakers of English and ESL students (language), and those with negative and positive attitudes toward English (attitude). Of the 119 students who responded to the questionnaire, as many as 19 did not answer specific parts of this question. (See Appendix K "Standard Terms and Abbreviations" for additional statistical data).

Finding 6. As Table 3 shows, students tended to strongly agree they would ask their teachers about an unfamiliar term (Likert 3.8) and rated asking a friend to explain the term slightly less positively (Likert 3.4). As might be expected, similar numbers reported that they would ask teachers to clarify comments and marking symbols. Very few students, however, indicated that
they would ask another teacher in the school for an explanation of a usage error. Small numbers
agreed with the statement that they would voluntarily revise a paper (Likert=1.9) and slightly larger numbers agreed that they would voluntarily make corrections on their papers (Likert=2.4).

Only three of the 24 comparisons for gender, first language or attitude showed significant differences; however, all of the differences favored girls and students reporting positive attitudes. Language differences were mixed with native speakers being more likely to consult with teachers while ESL students reported more reliance on friends and books.

The interviews, too, suggested that students were luke-warm toward discovering the meaning of marking terms and symbols: most students reported that they seldom or never enquired about an unfamiliar term. Their interest in the terms depended on how frequently a given problem appeared, whether they remembered seeing the term on past papers, and whether or not it was emphasized in the final comment. In most cases, when students were asked what they would do about a usage item they did not understand, they replied "I would probably just forget about it." Similarly, few students suggested during the interviews that peers would play an important part in their understanding of editorial usage, perhaps because the peers were not regarded as having the expertise to help effectively



Variable n Mean SD Gen. Lang. Att.
Ask this teacher about his/her comments 118 3.8 1.22 .07 .37 .15
Ask Friend about English teachers' comments 59 2.9 1.34 .19 .06 .84
Revise paper without being told to 117 1.9 1.1 .63 .07 .52
Correct errors noted by teacher 117 2.4 1.1 .09 .26 .09
Ask teacher about the meaning of a marking symbol 117 3.9 1.3 .01* .86 .27
Ask friend about the meaning of a marking symbol 118 3.4 1.3 .02* .02* .06
Look in textbook for the meaning of a marking symbol 118 1.8 1.0 .18 .81 .10
Ask second teacher to interpret first teachers' comments 118 1.7 1.00 .41 .50 .29

TABLE 3: Sharing and questioning arising from teachers' written comments

(questionnaire data). Likert choices ranged from "1, strongly disagree" to "5,

strongly agree." Statistical comparisons are reported for differences between the

responses of boys and girls, native speakers of English and ESL students, and students

who reported positive and negative attitudes to English class and to school.

Finding 7. As Table 4 shows, according to the questionnaires, peers were not significant audiences for students' papers after the papers were returned by teachers. The average student response fell in the neutral range for both reading a paper aloud to a friend and exchanging papers with a friend. The girls in the study and the students reporting positive attitudes showed significantly higher inclinations to share papers than their comparison groups, but there were only minor differences between ESL students and native speakers.

In the interviews, very few students reported sharing papers once they had been marked, but some who did share were very systematic about it (Appendix L: Peers as Audiences). Three grade-eleven girls, for example, shared their teachers' comments as a form of triangulation to see what the teacher really wanted in an assignment. One of these girls commented that since her teacher couldn't write all of the necessary suggestions on one person's paper, she felt it was useful to read the comments of two or three friends to gain a complete picture. However, the grade influenced both girls' and boys' tendencies to share their writing after the teacher had evaluated the composition. Lower grades seemed to inhibit sharing while higher grades appeared to provide students with more confidence about allowing other students to see their papers.



Topic n Mean SD Gender First Language Attitude
Read teachers' comments aloud to a friend 117 2.6 1.3 .001* .58 .005*
Exchange marked papers with a friend 117 2.6 1.4 .001* .69 .001*

TABLE 4: The role friends played in students' responses to their teachers' written

comments (questionnaire data). Likert choices ranged from "1, strongly disagree" to "5,

strongly agree." Statistical comparisons are reported for differences between the

responses of boys and girls (gender), native speakers of English and ESL students (first language), and students who indicated positive and negative attitudes to English class and to school (attitude).

Finding 8. Students looked for specific, focused, individual comments (Appendix M: "Responses to Specific Teacher Comments"). In both the interview and questionnaire data, students sought comments on specific strengths and weaknesses of the substance of papers; error was relegated to a minor role. One grade-eleven student summarized the common view:

The comment tells you why you got the grade that you did. He doesn't take off a lot of marks for errors. He tells you how it can be fixed. He always tells us what is good about our papers. He usually puts down what you need in your paper. If the teacher just writes down "Oh, good paper; I enjoyed reading it" you don't know what he enjoyed about it, but Mr. X always outlines "good transition, good thesis or something." And then you know to try and write the same or do better or use enough. Some just write "Excellent work," but you don't know what the teacher's talking about; what part of it is good because not all of it is good; you can't use that. Like even if you get perfect, you want to know what part the teacher liked.

Finding 9. When students were asked about the disposition of their compositions, most said they saved their papers, but only about half of them claimed they would use these papers for reference during future assignments. Ironically, although about half said that they would reread their papers and the teacher's comments when preparing for the next composition, very few reported that they had read through previous papers prior to the final editing of their current papers. The vast majority of students admitted (sometimes after much prodding) that their marked compositions were "stuffed into the backs of their binders" never to be reread. Many of these students articulated beliefs that revisiting compositions could be beneficial, but most acknowledged that they rarely or never did so.

Finding 10. Most, if not all, students appeared to search for praise in their teachers' comments before reading suggestions for change or corrections. Some students read only the positive comments and ignored any comments which were critical of their writing. Students who received little or no praise in their teachers' responses expressed unhappiness and believed "there must have been something good about it [the composition]."

Most students admitted that criticisms and corrections were necessary to improve their writing, but they called for positive comments to provide balance in their teachers' responses. Students who received some positive comments were more likely to reread their compositions and their teachers' responses while those who received little or no praise spent considerably less time reading comments and expressed little desire to reread their writing.

However, it should be noted that some students were quick to denounce praise which seemed vague, hollow or unmerited. When teachers used the same positive comment on many students' papers, it was perceived by some as being the same kind of "canned" comment they saw on computerized report cards. For these students, praise needed to be specific and personalized or it was viewed as insincere.

3. Conclusions

1. Most of the senior secondary students (grades 8 through 12) in this study seemed to learn little from their teachers' comments, but there were exceptions. The most successful comments or corrections were those which referred specifically to criteria that teachers had taught in class. These were read with more understanding and accepted more readily by the students than corrections or comments on material which had not been taught directly by the teacher. In classrooms where there was peer teaching of editorial usage or detailed teacher explanations of assignment criteria, students seemed to have a better grasp of the meaning teachers' comments. This supports Hillocks' (1986) conclusion that "The available research suggests that teaching by written comment on compositions is generally ineffective," (p. 167) adding that "It may be, however, that when comments are focused and tied to some aspect of instruction...they do increase the quality of writing" (p.168).

The second factor in this learning cycle may be tailoring the comments to individual student's needs and abilities. In this study, very few students were able to understand the plethora of comments on their papers; most appeared to be discouraged with the large number of different errors which were indicated only one time on the paper. One particularly successful teacher-- according to student reviews--appended a cover sheet which indicated that only two or three types of errors were marked on the paper and that the student was to work on these in subsequent papers.

2. "Error" influenced students' responses in a number of ways, mostly counter-productive. Students in this study were frequently distracted by excessive technical errors noted by their teachers. Consequently, for them the task of revision was unattractive because it was seen as merely a cumbersome task of correcting spelling and grammatical errors.

Students had a paradoxical attitude towards error. On the one hand, they frequently dismissed the "errors" their teachers marked as matters of the teachers' stylistic preferences. On the other hand, when asked to discuss how they might improve their papers or how they might write better papers in the future, almost all students limited their suggestions to eliminating errors. Reflecting the kinds of comments they found on their papers, most of these suggestions for error reduction were vague and general. The substance of the essay and its organization seemed to have little place in either the teachers' marginal comments or in the students' speculations on how they might improve their writing .

The focus on error also reinforced the myth that grading is a subtractive processes wherein all papers start with 100 percent and are reduced as marks are deducted for errors. When reading errors noted on their papers, students frequently lamented, "I guess that cost me a mark." One consequence of this is that students who were looking for extra marks read comments not to learn to improve their writing, but to find points on which to quibble with their teachers so that they could argue for a higher mark. If they received an acceptable mark, they frequently dismissed the comments as "not worth quibbling about."

3. The grade overpowered all else. All but three of the 53 students interviewed turned to the grade first, to no one's surprise. What was surprising, however, was the extent to which the grade governed the rest of the response: some students who received the grade they thought they deserved did not read any farther (the grade confirmed what they felt about the paper which made doing anything else superfluous); students who received grades much below what they were hoping for could think of nothing else but raising the grade (one said that, in addition, for the next couple of classes she would sulk); the small number of students who received a higher grade than they thought they deserved basked in a warm feeling but did nothing to try to determine why the grade was better than anticipated.

The second major problem with the grade is that it appeared to put the stamp "finished" on the paper--with the sole exception of small debates about testing its accuracy. Since the grade is a confidential and private judgment rendered by the teacher and recorded for official purposes, it presents the illusion of finality which appears to block further reflection by the student. This sense of completion might also be at least partly responsible for the students' lack of interest in the teachers' comments: they will never write another paper quite like this one, so what is to be gained by dwelling on its shortcomings?

4. Peer response and sharing with peers provides many students--especially girls--both an aid to understanding and a set of criteria on which to judge their own work. One particularly astute grade-eleven girl reported that her teacher could not possibly say everything on one paper that needed to be said. Therefore, she read comments on friends' papers to fill in the gaps in her own. On the other hand, a substantial majority of the students interviewed--including almost all of the boys--admitted they rarely shared their writing with anyone, especially after the teacher had evaluated it. A grade-eleven girl reported feeling guilty about being "selfish" in keeping the paper for herself. She said, however, that she found it easier to share papers in subjects other than English because they were less personal..

5. Students searched for positive comments and praise and lingered on the examples they found. Many of the students interviewed expressed a desire to see positive and encouraging comments on every composition, but they were usually quick to spot unmerited praise as empty, frivolous, or hollow. Comments which were personalized seemed to give students the impression that their teachers cared about them and what they had to say. Personalized comments were those which related current compositions to past writing, shared the teacher's beliefs and experiences, and validated the ideas and hypotheses of students. Criticism, on the other hand, was largely ineffective. Students tended to resent what they saw as extensive criticism, constructive as it might be, or teachers' suggested revisions of their sentences and generally ignored more than a few well-placed, focused, and easily understood suggested changes in writing style or substance.

4. Epilogue

Responding to students' written work is, of course, the most difficult part of the English teacher's job, because of the time commitment if for no other reason. If a teacher devotes ten minutes to responding to each student's essay, he or she can mark six essays per hour. A class of thirty essays requires five hours work on top of teaching, preparation, and extra-curricular activities. And, of course, teachers do not teach just one class.

The suggestions above are not meant to be a panacea for all of the difficulties of grading: if there were an easy solution to the grading problem, teachers would have found it long ago. We hope, though, that our suggestions will make the work that English teachers do pay somewhat higher dividends.



References





Arnold, L. (1964). Writers' cramp and eyestrain--Are they paying off? English Journal, 53(1), 10-15.

Bardine, B. A., Bardine, M. S., & Deegan, E. F. (2000). Beyond the red pen: Clarifying our role in the response process. English Journal, 90(1), 94-101.

Béchervaise, N. E. (1998). Assessment in English. In Wayne Sawyer, Ken Watson, and Eva Gold (eds.). Re-Viewing English. Sydney: St. Clair.

Belanger, J. (1985). Conflict between mentor and judge: Being fair and being helpful in composition evaluation. English Quarterly, 18, 79-92.

Belanger, J. (1986). "Student written errors and teacher marking: A search for patterns." Report to the Educational Research Institute of B.C., June 1986, 120 pp. (ERIC: ED 273960)..

Carifio, J, Jackson, I., & Dagostino, L. (2001). Effects of diagnostic and prescriptive comments on the revising behaviors of community college students. Community College Journal of Research and Practice, 25 (2), 109-22.

Chanock, K. (2000). Comments on essays: Do students understand what tutors write? Teaching in Higher Education. 5(1), 95-105.

Cohen, Andrew. (1987). "Student processing of feedback on their compositions". In A. Wenden and J. Rubin (Eds.) Learner strategies in language learning . Englewood Cliffs: Prentice-Hall.

Connors, R. J., & Lunsford, A. A. "Teachers' rhetorical comments on student papers." College Composition and Communication, 44(2), 200-223.

Diederich, P. B. (1974). Measuring Growth in English. Urbana, IL: National Council of Teachers of English.

Diederich, P. B. (1996). Turning Fords into Lincolns: Reminiscences on teaching and assessing writing. Research in the Teaching of English, 30(3), 352-60.

Faigley, L. and Witte, S. (1981). "Analyzing revision". College Composition and Communication, 32(4), 400-414.

Farrell, E. (1970) Deciding the Future: A forecast of responsibilities of secondary teachers of English, 1970-2000 AD. NCTE Research Report No. 12. Urbana, IL: National Council of Teachers of English.

Ferris, D. R. (1997). The influence of teacher commentary on student revision. TESOL Quarterly, 31(2), 315-35.

Gee. T. C. (1972) Students' responses to teacher comments. Research in the Teaching of English, 6(2), 212-21.

Hillocks, George. (1986). Research on written composition: New directions for teaching. Urbana, IL: NCRE/ERIC Clearinghouse.

Hyland, F. (2003). Focusing on form: Student engagement with teacher feedback. System, 31(2), 217-31.

Hyland, F. (1998). The impact of teacher written feedback on individual writers. Journal of Second Language Writing, 7(3), 255-86.

Hyland, F. & Hyland, K. (2001). Sugaring the pill: Praise and criticism in written feedback. Journal of Second Language Writing, 10(3), 185-212.

Jonnston. B. (1987). Assessing English. Buckingham: Open University Press.

Kouritzin, S., & Vizard, C. (1999). Feedback on feedback: Preservice ESL teachers respond to evaluation practices. TESL Canada Journal, 17 ,(1), 16-39.

Leach, L., Knowles, N., & Duckart, T. (1997). "Living the myth: Mergine student and teacher needs in responding effectively and efficiently to student papers." Paper presented at the Annual Conference of the English Council of California Community and Two Year Colleges, San Francisco, CA, October 1997. (ERIC: ED 420349).

Marzano, R. J., & Arthur, S. (1977). Teacher comments on student essays: It doesn't matter what you say. (ERIC Document Reproduction Service: ED 147 864)

Matsumura, L. D., Patthey-Chavez, G. G., Valdes, R. & Garnier, H. (2002). Students' revision in lower- and higher-achieving urban schools. Elementary School Journal, 103(1), 3-26.

North Carolina State Dept. of Public Instruction. (1998). Evaluating and Improving Student Writing and Writing Instruction: A Systematic Approach. (ERIC: ED414601).

Norton, L. S., & Norton, J. C. W. (2001). Essay feedback: How can it help students to improve their academic writing? (ERIC Document Reproduction Service: ED 454 530)

Owens, C. V. (2001). Teachers' responses to science writing.(ERIC Document Reproduction Service: ED 457 157).

Radecki, P. and Swales, J. (1988). "ESL student reaction to written comments on their written work". System, 16 (3), 355-65.

Sloan, G. (1977). The wacky world of theme marking. College Composition and Communication, 28(4), 370-73.

Sommers, Nancy. (1982). "Responding to student writing". College Composition and Communication, 33 (2), 148-56.

Sprinkle, R. (1998). "Self-assessment for writing instructors: A practical guide to more effective written commentary." Paper presented at the Annual Meeting of the Conference on College Composition and Communication, April 1998. (ERIC: ED 417415)

Straub, R. (1997). Students' reactions to teacher comments: An exploratory study. Research in the Teaching of English, 31(1), 91-119.

Tchudi, S. (Ed.). (1997). Alternatives to Grading Student Writing. Urbana, IL: National Council of Teachers of English.

Thompson, T. (1993). "Personality preferences and student responses to teacher comments." Paper presented at the Annual Meeting of the Conference on College Composition and Communication, San Diego, CA, April 3, 1993. (ERIC: ED 357392).

Wiltse, E. M. (2001). The effects of motivation and anxiety on students' use of instructor comments. (ERIC Document Reproduction Service: ED 458 630).













1. 1 The study reported here was supported by an internal grant from the University of British Columbia Humanities and Social Sciences committee. We would also like to thank Donna VanSant, Kathy Puharich, Grace Klemovich, and Edward Metcalfe, English department heads and central-office administrators who facilitated the project.

2.

1 Andrew Sloan, a doctoral student in Language Education at the time, was a graduate assistant for the project. Mr. Sloan conducted half of the interviews and assisted in analyzing the data and drafting the report. Many of the insights in the report are Mr. Sloan's, but oversights, omissions and misinterpretations are ours.