Doing action research in English language teaching (part 3 of 3)

This is the third of 3 blog posts summarizing a book by Prof. Ann Burns on Action Research (AR) in language teaching. In my previous posts, I summarized the first and second parts of the book, on what action research is and how to design an AR study. In this post, I provide a summary of the last chapters of the book, on how to analyze, reflect on, and present the data collected. There are lots of excellent examples here: systematic analysis protocols, graphic organizers, and plots/charts/diagrams to present to stakeholders!

Burns, A. (2010). Doing action research in English language teaching: A guide for practitioners. Routledge.

Chapter 4: Observe – observing the results of the plan

Burns begins the chapter on data analysis with a real example of a university instructor in Japan who was researching how students in a distance course engaged with the online discussion forum. This person was overwhelmed with the “sheer mountain of data… hundreds of discussion list posts, private emails, phone conversation transcripts, theory memos, and scads of site user statistics to crunch. Finding a common narrative thread was a big challenge” (p. 103).

The key to not being overwhelmed with the amount of data you collect is to look at the data while you are collecting it, not all at the end, and write reflections on what you notice. Burns lists several question prompts (p. 104):

• Do these data answer my [research] questions? If so, how?
• What are the main messages so far? What are the gaps in the message I still need to fill?
• Am I still asking the right questions or are the data telling me that something else is more important? (This is a key one… please consider this seriously)
• Do I need other kinds of data to help me really see what I am looking for? (Another key one)
• To answer my questions, are some pieces of data more important than others? (If so, you can stop collecting/paying attention certain pieces of data so that you can pursue more important or different data sources more intensively)

This requires an open-ended inquiry with no quick rules of thumb. The thing is to be alert for what the data shows, and follow that wherever it may lead. This can be challenging, rewarding, and inspiring (p. 105).

Burns next goes into different ways of analyzing qualitative data.

Categorizing

Categorizing (coding) is the most intuitive form of qualitative research analysis. Burns uses the analogy of a room filled with scattered toys, which you might put away based on type, condition, age of the child, etc. (p. 107). This is called “coding” for “categories” that “emerge” from the data. For a detailed explanation of how to do this, see this post on Grounded Theory. Here, Burns gives an old, nostalgic example from a famous professor, Andy Xuesong Gao, who coded interviews with 14 people about what made a good language teacher (Gao, 2006):

During the process, I searched for all the students’ references to teachers in the data. For instance: “He was a serious teacher in class. But after class, he could be your nice friend. In fact many students like such teachers. (Jing Jing, September 19, 2024)” I highlighted the words serious, nice, and like, because they suggest that an ideal teacher in this particular student’s opinion should be nice and strict. I identified the initial coding categories by carefully reading through one interview transcript, and I refined the coding categories by applying the initial categories in interpreting the other interview transcripts. (p. 107)

Keep in mind that this is a small-scale study and usually we read more than one transcript to come up with initial codes, which we refine on the rest of the data set. The application of pre-determined codes to the data set is called deductive coding. These codes can come from a pre-existing theoretical framework. In contrast, letting codes emerge from the data is called inductive coding. (No study, in my opinion, relies entirely on one or the other—all qualitative research, due to its epistemology, involves inductive coding, but even studies that rely on inductive coding cannot filter out the researcher’s pre-existing views on what is meaningful, based on their readings or professional discourses.)

Gao also examined “what these students wanted teachers to do for them in the learning process… [and] analyzed positive and negative images of teachers to identify particular patterns of students’ expectations” (p.107). In other words, we don’t merely present categories of codes but try to find explanatory links between ideas. This is what some Grounded Theorists call axial coding or conceptual coding, which is one level higher than categorization; see this post for the different levels of codes. Also, don’t think of your categories as one-dimensional. For example, Burns notes that Gao was interested in “the role of the teacher, the behaviour of the teacher, or the skills of the teacher” (p. 110). It is all right if some things belong to multiple categories, or in a miscellaneous pile, or form new categories of their own, or stop being a category, or get moved up/down from category to subcategory… coding is supposed to be dynamic this way.

Analyzing talk

This is my favorite type of qualitative analysis. Qualitative studies that analyzed classroom talk made a big difference in improving by skills as a teacher. Analysis of talk is also helpful for understanding how power dynamics and ideologies play out in subtle ways.

Burns recommends reading a book by Keith Richards called Qualitative Inquiry in TESOL (Richards, 2003) for doing this kind of analysis, and I will recommend a shorter reading by the same author—an article called “‘Being the teacher’: Identity and classroom conversation” (Richards, 2006). At least read that article before you start investigating “identity” in conversation, because you need to understand what identity means in talk: what sociolinguists call “brought about” rather than “brought along” identities (Zimmerman, 2008).

When you audio- or video-record and then transcribe a conversation, you can do a birds-eye view of it for obvious features (what is going on, the stage of the lesson, who is talking the most, etc.). These are important, of course, and then Burns says you can get into structural elements, like how many turns people take, etc. You can also divide the conversation into stages, or several lines of talk into moves such as “question/answer”, or “confusion/repair/confirmation of understanding”. Burns points out three types of teacher talk, which the teacher can shuttle back and forth between (Taylor, 2006, as cited in Burns, 2010, p. 117):

1. Teaching function—help students construct, extend, or activate knowledge and understanding of language.
2. Structuring function—Structure and manage procedures conducive to language learning.
3. Rapport-enhancing function—Create and maintain positive affect through support.

However, expert teachers tend to blend these three so they are not easily separable. There’s a study by incoming AAAL President Mari Haneda that illustrates this in action, which I often assign in my teacher education classes (Haneda, 2004). If you are passionate about this type of research, you can see my series of four posts summarizing a book on analyzing classroom talk by Betsy Rymes, on turn-taking, contextualization, narration, and framing.

Quantitative data

Burns notes that some think action research is purely qualitative, and though a lot of it is, numbers are useful to get a concise overview of the big picture (averages, frequencies, percentages, typicalities, anomalies). You can look at the central tendency, which can be defined as:

• mean (the average, or the sum of all scores divided by the number of people)
• median (if you were to rank the scores from lowest to highest, what would be the middle one or two scores), or
• mode (the most popular or frequent score).

You can also look at the dispersion—the extent to which scores cluster around the mean, or whether they spread out more. There are two main measures of dispersion:

• Range, or let’s say the lowest score on a test is 50 and the highest 80, then the average is 65 and the range 30.
• Standard deviation (SD). Presenting SDs requires a fairly large number of participants, because if the sample is too small it is meaningless. Those who fall within 1 SD of the mean represent the “pack” or the most typical two thirds (68%) of participants. People who fall within 3 SDs of the mean represent the top and bottom 2% of participants. Then there are those well above/below average, who fall within 2 SDs of the mean, but are not extremes. (And there are 4 SDs, a tiny fraction of one percent, but that is likely to occur with very large quantitative samples and not in an action research study.)

Numbers can also be ordinal (ranks/”list in order of preference…”) or interval (such as a survey that asks “1 – very often”, “2 – often”, etc.). They can be percentages, frequencies, test scores, or something else.

My understanding of quantitative research is very basic, but if you want to involve quantitative data in your teacher action research study, your best bet would be to use a model: a high quality empirical study (or several) investigating the same thing. To find your model(s), you may need to consult with academic advisors knowledgeable in the area.

Validity and generalizability

Burns concludes Chapter 4, on data analysis, with the question: “How can I make sure that what I am finding results in reasonable judgments and conclusions?” (p. 130). There are some ways to help ensure this, such as 1) not just relying on one type of data, 2) collaborating with co-researchers, or consulting with participants or mentors who are not as emotionally involved in the research as you are (“Is this right? Have I explained this accurately?” p. 132), 3) giving enough time to examine the data, and 4) actively looking for things that don’t fit the general trend (to keep one’s mind open). You can also ask yourself questions to check your biases, such as “Am I being objective? Am I seeing things in the data as they really are, rather than how I want to see them?” and “Am I claiming too much about the results of the changes I made? Am I suggesting that my conclusions apply beyond my classroom?” (pp. 130-131). In addition, Burns suggests that we choose AR topics that benefit students first and foremost: “For example, experimenting all term with content or tasks that do not cover the topics students need for a competitive exam would not benefit them (or you for that matter!). In this situation you would be doing AR for your own interests and not for your students'” (p. 131).

Regarding knowledge that is generalizable enough to be useful to others, Burns gives the example of Bill (Derham, 2001), who taught a class of low intermediate English learners in Australia (p. 134). Bill discovered that 50 words per week was manageable, and that they should form a narrow focus (enough in one domain that students would have a critical mass of vocabulary to do things in that domain, for example, make a type of phone call or use an ATM). Don’t worry too much about how to make your AR findings “generalizable”—other practitioners will recognize what it is that can be generalized. For example, another teacher might experiment with Bill’s methods to find out what their class’ numerical threshold for vocabulary is—given their learners’ proficiency, the number of lessons and hours a week, etc.—and do a needs analysis to know the domains relevant for their learners.

Burns ends Chapter 4 by reminding us that data analysis in AR doesn’t need to wait until the end, and in fact shouldn’t. Rather, insights and findings should shape the direction of the research: “Like the AR cycle itself, data analysis is dynamic, cyclical and recursive” (p. 135).

Chapter 5: Reflect – reflecting and planning for further action

As stated in the beginning of the book, AR is about cycles of action. It involves at least two cycles of research because the findings from the first cycle inform pedagogical action, and then you collect more data on how the action went. In this chapter, Burns discusses the limits of AR (when to end) and how to present the results to others.

First, she goes into what kind of knowledge is generated by AR: personal practical knowledge. To illustrate, she quotes a philosopher named Karl Polanyi:

I have shown that into every act of knowing there enters a passionate contribution of the person knowing what is being known, and that this… is no mere imperfection but a vital component of his knowledge (as cited in Burns, 2010, p. 142).

While AR often involves checking your biases, at the end of the day, Polanyi’s quote argues that subjectivity is not imperfection but “a vital component of knowledge.” It’s the kind of subjectivity that leads to insight as you throw yourself into the pursuit of truth—leading to an insight only a human mind can have and propose to others. Although teachers ordinarily reflect on what they did and how it could have been done better, then act accordingly, because AR was done so intensively and systematically it is “likely to have been a much more intense experience than regular day-to-day teaching” (p. 143).

Thus, Burns asks us to examine several topics at the close of our AR: our actions, beliefs and feelings. I list some of the questions she asks about each of these things below, italicizing the questions I feel are particularly important as they lead to irreversible professional development.

The “A” refers to the actions that came about as a result of the research (p. 143).

• Why did I select these particular actions?
• How did the actions I selected work to improve the situation? For me as the teacher? For my students? For other people involved?
• Were the actions effective? Did I need to change them?

Then there are beliefs (p. 144):

• What are two of my strongest personal beliefs about teaching? What are two of my strongest personal beliefs about learning?
• How did these beliefs affect the decisions I made as I did my research?
• In what ways have my practical theories about teaching developed?

For feelings, Burns notes that we do not have to fixate on our mistakes and weaknesses. Rather, AR is about “opening ourselves up honestly, and sometimes courageously, to the problems, dilemmas, possibilities and opportunities in our teaching contexts. It means trying to find the things that make a difference, however small they seem to be” (p. 145). In other words, she asks us to consider negative things in light of possibility rather than criticism (p. 145):

• How did you deal with negative reactions? What impact did the positive reactions have?
• Did the negative reactions trigger ideas that you had not thought about before? If so, how, and what did you do?
• How did your personal feelings contribute to the way you did your AR?
• What personal feelings and experiences arose from finishing your AR?

One of the interesting—and most beautiful, in my view—outcomes of AR (or any qualitative research) is when

you have gradually realized that the issues or questions you started off with were not actually the main ones you needed to address. You may still have more questions than answers… So, depending on where this first cycle has taken you, and the time and interest you have available, you may decide [how long] to keep going. (p. 146)

If you can keep going, you might continue the AR with a different class, or change the topic of the AR to what really seems to be the issue. Here, Burns gives the example of Pam (pp. 146-147), a teacher of a culturally diverse group of immigrant adults. Pam was trying to do activities to celebrate diversity, but these were the exact things that her students did not want to discuss, and identities they did not want to call attention to, due to historical strife where they had come from: “On a class excursion, I learned that the students were aware of deep ethnic, religious and political differences because of their experiences of the part of the world they had just left [former Yugoslavia]… I suddenly realized how difficult it had been for them to maintain the veneer of courtesy and civility when I was introducing activities which demanded that they expose and discuss the differences they were attempting to ignore!” (p. 147).

Sharing the findings of AR

In addition to the meanings AR has for teacher and students, there are the meanings it has for other consumers of the research. Burns next discusses sharing, presenting, and publishing AR. She writes:

Don’t be put off… by the ideas that the ‘local’ research you do in your school, country or region will be of no interest elsewhere. Language teachers the world over face similar challenges, as I hope the examples in this book show. … Teachers usually enjoy learning about how other teachers have solved problems they may be experiencing too. … And knowing more about how teachers actually operate in classrooms may be of value to second language acquisition research in general (see Zephir, 2000). (p.149)

And of course, it’s important to present work in different formats, as the research could interest “teaching colleagues from within the school or region, school administrators or bureaucrats. Parents, students and members of the local community could also be involved” (p. 150). And there are the usual suspects, such as consumers of research at conferences or those who read research articles and their summaries.

One surprising thing that Burns mentions is that ordinary conversation (p. 151) is research dissemination. This reminds me of an article I and a classmate presented in grad school, called “Tiny Talks” (Zoshak, 2016), which you can find here. Casual conversations in hallways and lunchrooms can indeed be impactful on your and other people’s practice. We all make little changes in our teaching after even the briefest of conversations with colleagues.

Quoting many real teachers as always, Burns shares some of their thoughts, including this one I particularly like:

Writing about the steps of the research has been for me another moment of reflection, as it has made more solid the basis on which to build a new research cycle. At the same time it has made clear that in fact the object of change was not only the single aspect of my teaching practice I investigated, but the whole idea of teaching and learning… (p. 155)

What this teacher seems to be saying is that yes, you have to operationalize specific pedagogical actions and focus on analyzing the effect of those so the study isn’t all over the place, but a really good AR study can change what you believe about teaching or learning more broadly after you have analyzed and reflected on the data in preparation for sharing it with others.

When disseminating research, Burns asks us to consider who the audience is, what they already know, and what kinds of research we like reading ourselves—from the content to the ways of presentation, considering things that turn us on or off research (p. 156). It is my belief that teachers may not read research as much as researchers, but when teachers do read research, they certainly read it earnestly. There are teacher-oriented journals/blogs that are well read, compared to long journal articles everyone skims and nobody reads closely. Why do I think this? Because general platforms like Facebook, Youtube, and Reddit (those groups/channels that pertain to language teaching) are places where people have lively discussions with lots of participants. Consider sharing an infographic or a link to a view-only Google document about your research on one of these informal platforms.

When it comes to writing a fuller, more formal report of your research (for a blog post or article in a teacher journal), Burns has a helpful checklist of what to include (pp. 161-162). This does not mean that you have to present your research as a list of answers to the following questions—your written piece can be a narrative, for example—but what Burns means is that the following information should be somewhere in the piece:

1. Your name, where, and what you teach (background, experience, school, position).
2. Information about the school and your class (location, setting, student population, class(es)).
3. Your issue, problem, or questions (concerns, expectations, dilemmas, challenges and how they evolved, if applicable).
4. Reasons for the issue, problem or questions and their importance to you as a teacher (rationale, proposed theories/beliefs about teaching).
5. Literature relevant to your research issues, if applicable (how it informed/motivated your research).
6. Teaching plans and actions for changing the situation (beginning of action research cycle, focus, expectations, reasons for selecting).
7. Tools/techniques used to collect data (why you chose them and how you used them to answer your questions).
8. First observations and reflections (happenings, successes/failures, insights from actions, examples from data).
9. Next directions, if applicable (further cycles, plans, actions, data collection, more samples from data).
10. Data presentation, organisation and analysis, as applicable to cycles (quotes, excerpts from transcripts, categories, samples, tables, graphs, figures).
11. Struggles, difficulties, dilemmas, questions coming from the research cycles.
12. Insights, findings, solutions, inspirations coming from the research cycles.
13. Reflections, interpretations on the findings.
14. Reflections, feelings, interpretations on the process.
15. Personal knowledge, learning, and realisations developed from the action research experience.
16. Future directions (new questions, further steps, changes to practice, recommendations).
17. References, pictures, appendix.

In other words, there are key pieces of information that people need to understand our AR, but these can be delivered in a variety of ways. Burns says:

In my opinion, there are no limits to creative ways of presenting that might be appropriate for the different audiences you want to reach. I encourage you to be as imaginative as possible in the way you publicise your research. (p. 164)

Conclusion

Burns concludes the book with quotes from teachers around the world on what they (unexpectedly) got out of AR. I share some of my favorites here (pp. 165-166):

• “No-one else could have given me this information. It was about me, my students, and our work together.” (Jami, Mount Prospect School, Basking Ridge, New Jersey, USA)
• “Without the discipline of the research I would have missed many of these insights.” (Heather, Auckland University of Technology, New Zealand)
• “After having been engaged in AR practice for some time, I realised that my discomfort with issues I didn’t know how to deal with in my classroom could actually be explored, understood, and maybe solved.” (Heliana, Universidade de Minas Gerais, Belo Horizonte, Brazil)
• “Being involved in action research works inside you like a vaccine. Once you have been vaccinated, you are for life. Once you’ve been through an action research cycle your look on pupils and teaching changes in a way that is irreversible.” (Mariacarla, Italy)
• “During my first few years of teaching, I was painfully aware that I didn’t have all the answers—and that was a tremendous source of anxiety. Was I really helping my students, or was I doing more harm than good? … Action research has helped me transform my questions from a source of anxiety into a source of curiosity.” (Sharon, EARTH University, Guácimo de Limón, Costa Rica)

Burns asserts that

More and more, the voices of language teachers from around the world who have done AR are contributing to classroom practice, curriculum development, and knowledge about effective ways to teach English in different contexts. … This movement helps to broaden the knowledge base of language teaching (see Johnson, 2009) and contributes to our understanding of important issues in second language acquisition research. As I argued at the beginning of the book, AR is also a ‘democratic’ or ‘participatory’ movement for our field. It enables language teachers, who have usually been either the ‘subjects’ of research or excluded from research altogether, to enter an AR ‘community of practice’ and gain a stronger professional presence. Consequently, the gap between the work of educational theorists and the work of teachers becomes reduced. Rather than recommendations for practice being made at a distance from the classroom, teachers can become an integral part of testing out how feasible these recommendations really are in different contexts. (p. 166)

Why she wrote this book is why I have summarized it on the blog. Here are her parting words (p. 167):

I wrote this book because of the many positive encounters with teachers all over the world who have told me they are interested in AR. Many of these teachers had never done research but were keen, committed, and enthusiastic professionals eager to make their classrooms the best places they could for their students. They were excited about the idea of AR and were looking for user-friendly guidelines to help them make a start.

As I leave you, my reader, who has accompanied me through these chapters, my sincere wish is that this book will inspire you to get started in AR. Of course, it is impossible for me to put myself in your exact teaching situation or to answer your questions directly. But I hope the classroom voices I’ve brought into this book—many of them known to me personally—will guide and encourage you as you try out the ideas yourself or with your colleagues. I’ll leave you with ten things I’ve learned over the years about how to build success into the AR journey—and I wish you good luck!

1. Find colleagues interested in AR to work with.
2. Maintain a reflective and enquiring attitude to your teaching.
3. See classroom challenges and ‘problems’ as positive opportunities for change for you and your students.
4. Be confident in your ability to find good outcomes for classroom challenges.
5. Look for opportunities and take risks to change your teaching strategies.
6. Follow your interests and passions about teaching.
7. Keep reviewing and refining your personal practical knowledge.
8. Be a strong voice for teacher inquiry and professional development in your organization.
9. Share your research with others.
10. Don’t be discouraged—keep going!

Postscripts

There are two Postscripts showcasing 2 AR studies (pp. 169-187), which I won’t summarize here, but one was about 4th grade students’ self-assessment, comparing it side by side with teacher assessment. The other investigated how adult students learned new vocabulary from watching TV. It is not difficult to see how these studies benefit learners, as the students are asked not just to “do things” to improve their target language proficiency, but reflect on what they do and how they do it. There are examples of research instruments, memos, and charts/diagrams in these case studies, and in the Appendix of Chapter 4 on analyzing data (pp. 138-139). These examples show how to design richly triangulated AR in which the data collected and the instruments used to collect/analyze it align with the research questions. There is also a 5-page bibliography of further reading on each of the stages of AR (pp. 188-192).

References

Derham, B. (2001). Vocabulary and a CSWE II class. In A. Burns & H. de Silva Joyce (Eds.), Teachers’ voices 7: Teaching vocabulary (pp. 25-32). Sydney: National Centre for English Language Teaching and Research.

Gao, A. X. (2006). Understanding Chinese students’ teacher dependence. In T. S. C. Farrell (Ed.), Language teacher research in Asia (pp. 61-74). Alexandra, VA: TESOL.

Haneda, M. (2009). Learning about the past and preparing for the future: A longitudinal investigation of a grade 7 ‘sheltered’ social studies class. Language and Education, 23(4), 335-352. https://doi.org/10.1080/09500780902954265

Johnson, K. E. (2009). Trends in second language teacher education. In A. Burns & J. C. Richards (Eds.), The Cambridge guide to second language teacher education (pp. 20-29). Cambridge University Press.

Richards, K. (2003). Qualitative inquiry in TESOL. Springer.

Richards, K. (2006). ‘Being the teacher’: Identity and classroom conversation. Applied Linguistics, 27(1), 51-77. https://doi.org/10.1093/applin/ami041

Taylor, L. (2006). Aspect of teacher-generated language in the language classroom. In S. Borg (Ed.), Language teacher research in Europe (pp. 125-138). Alexandra, VA: TESOL.

Zephir, F. (2000). Focus on form and meaning: Perspectives of developing teachers and action-based research. Foreign Language Annals, 33(1), 19-30. https://doi.org/10.1111/j.1944-9720.2000.tb00886.x

Zimmerman, D. H. (2008). Identity, context and interaction. In C. Antaki & S. Widdicome (Eds.), Identities in talk (pp. 88-106). Sage.

Zoshak, R. (2016). ‘Tiny talks’ between colleagues: Brief narratives as mediation in teacher development. Language Teaching Research, 20(2), 209-222. https://doi.org/10.1177/1362168815627659