Starting Investigations with “I”:
Discussing Researcher Bias with Elementary-Age Students
by Jeff Chandler
May 31, 2021
This article was originally published in The Outdoor Teacher.
As a graduate student, discussions of researcher bias often accompany critical readings of study and theory, but I realized that this level of analysis rarely showed up in my lessons with elementary-age students through the Afternoon Ecologists program at IslandWood. And why not? Having taken the time to think about Next Generation Science Standards (NGSS) connections to each lesson, I know that considerations of scale and proportion are cross-cutting concepts that can be applied to a wide range of scientific investigations and phenomena, so maybe investigating researcher bias with elementary students was just another issue of scale.
Background
I believe researcher reflexivity is an important aspect of any study or investigation. Understanding how I, as the person conducting the investigation and making decisions at every turn, am thereby subtly or in more obvious ways impacting the resulting data that lead to my conclusions. Reflexivity requires constantly assessing these decisions and the way my personal perspective, experiences, and assumptions show up in the research or lesson in order challenge that bias where available, or to hold awareness of the impacts at the very least. In Researchers, Reflexivity, and Good Data: Writing to Unlearn (2000), Audrey Kleinsasser talks about the importance of researchers developing a practice of reflexivity through writing, which “makes researcher thinking about personal and theoretical commitments visible and... open to critical examination of the research process” (p. 155). Kleinsasser goes on to make the comparison of “‘researcher as instrument,’ to show that a researcher collects data that pass through the researcher's theoretical, practical, experienced, and inexperienced lenses” (p. 160). Whether we consider this instrument to be one of music or scientific observation, the result of each is shaped and produced by the function specific to that instrument. Light or air passing through a different instrument might create a different result, and such is the same with data that passes through the figured “lenses” of a researcher.
Now that we’ve thought about why researcher reflexivity is important at an academic level, I want to share a way I have introduced and explored the topic through the framing of an outdoor education lesson with elementary students. Following the lesson outline, I’ll also share some reflections and example responses I’ve heard from students. Because I first tested this activity out in the fall, we used fallen leaves as the focus of study, but this model could be adapted for use with a wide range of other subjects.
Activity Outline
Materials: sticks and leaves
Learning Objectives: Participants will:
Analysis
Through the debrief questions after each round of graphing, students are able to reflect on concrete examples of how each different form of researcher bias entered into their graphing experience. This activity provides a tangible, low-stakes, iterative way of exploring and discussing these concepts within the context of scientific investigations. By prompting their thinking about how they arrived at their conclusions each time and the decisions they made along the way, you offer opportunities to practice researcher reflexivity, to question their own process, and to identify points of revision and improvement in the design.
I have led this activity several times now with different students each time and I use each debrief as a formative assessment to gauge how they are processing and analyzing the concepts and where more conversation or reflection might be helpful. During one session, after asking the question, “What data are we missing?” a student thought for a moment, looked up with excitement, then looked up again and pointed, exclaiming, ”What about all of the leaves that are still on the trees!” These moments of realization are what I hope students gain from this activity--when they discover that the answers they thought they’d arrived at are actually just the beginning of more questions and when they begin to see themselves as part of the science rather than separate or removed.
May 31, 2021
This article was originally published in The Outdoor Teacher.
As a graduate student, discussions of researcher bias often accompany critical readings of study and theory, but I realized that this level of analysis rarely showed up in my lessons with elementary-age students through the Afternoon Ecologists program at IslandWood. And why not? Having taken the time to think about Next Generation Science Standards (NGSS) connections to each lesson, I know that considerations of scale and proportion are cross-cutting concepts that can be applied to a wide range of scientific investigations and phenomena, so maybe investigating researcher bias with elementary students was just another issue of scale.
Background
I believe researcher reflexivity is an important aspect of any study or investigation. Understanding how I, as the person conducting the investigation and making decisions at every turn, am thereby subtly or in more obvious ways impacting the resulting data that lead to my conclusions. Reflexivity requires constantly assessing these decisions and the way my personal perspective, experiences, and assumptions show up in the research or lesson in order challenge that bias where available, or to hold awareness of the impacts at the very least. In Researchers, Reflexivity, and Good Data: Writing to Unlearn (2000), Audrey Kleinsasser talks about the importance of researchers developing a practice of reflexivity through writing, which “makes researcher thinking about personal and theoretical commitments visible and... open to critical examination of the research process” (p. 155). Kleinsasser goes on to make the comparison of “‘researcher as instrument,’ to show that a researcher collects data that pass through the researcher's theoretical, practical, experienced, and inexperienced lenses” (p. 160). Whether we consider this instrument to be one of music or scientific observation, the result of each is shaped and produced by the function specific to that instrument. Light or air passing through a different instrument might create a different result, and such is the same with data that passes through the figured “lenses” of a researcher.
Now that we’ve thought about why researcher reflexivity is important at an academic level, I want to share a way I have introduced and explored the topic through the framing of an outdoor education lesson with elementary students. Following the lesson outline, I’ll also share some reflections and example responses I’ve heard from students. Because I first tested this activity out in the fall, we used fallen leaves as the focus of study, but this model could be adapted for use with a wide range of other subjects.
Activity Outline
Materials: sticks and leaves
Learning Objectives: Participants will:
- Practice collecting data in multiple ways from the same sources.
- Analyze ways in which their decision-making and interpretations impacted resulting data and conclusions.
- Consider implications and connections to other decision-making they experience in their lives.
- Start by asking all students to gather 5 interesting leaves.
- When they return to the group, have each student share about the leaves they collected.
- Explain: “Now that you’ve each collected leaves from the forest, we are going to begin to organize these leaves into a bar graph as a way to begin making connections, comparisons, and get a sense of what we collected.” Arrange two long sticks in an x/y axis on the ground and clear the space inside the grid of any leaves and other debris. (If bar graphs are not something you have talked about previously with this group,, it would be a good idea to review their general structure and mechanics.)
- Ask the group: “How could we organize the leaves we collected into a bar graph?” Some ways that might come up include: by color, by size, or by tree species (these will be three useful categories for the purpose of leading into conversations about researcher bias, so see if you can tease out at least these, along with any others).
- Demonstrate how to form bars in the bar graph by placing leaves end-to-end in a line to form a single bar.
- Start with sorting by color. Allow students to each place their leaves onto the bar graph where they think each goes. This might prompt some discussion or debate about what categories the group is using or about which category a leaf fits into, for instance about whether a leaf is more green or more yellow, or where to put leaves that are half and half or speckled. Let the students work through these questions and come to a decision.
- After they are done graphing, ask students:
- “What conclusions can we draw from our data?”
- “What were some challenges you ran into?” This should hopefully bring to mind any discussion about what to do with leaves that did not fit neatly into a category like “green” or “yellow”.
- “How did you make a decision in those moments?” Listen for where student (researcher) decision-making affected the conclusions they drew from their data earlier.
- “How did those decisions you made impact our results and the conclusions we drew from our data?”
- After they are done graphing, ask students:
- Reset: now graph by leaf size.
- Since there has already been discussion about how to form categories, students might be quick to recognize that they need a system of measurement for determining how they are organizing by size. Again, allow them to determine this and carry out the graphing process.
- If they followed the same protocols for creating bars of the bar graph as modeled in the demonstration and used in the first round, they should still be arranging leaves end-to-end to form each bar. If they decide to use a different approach, that’s ok too, but use that as an opportunity to discuss why they made the decision they did.
- After they are done graphing, begin with the same question:
- “What conclusions can we draw from our data?”
- Note here that because we are graphing by leaf size, the height of a bar in the bar graph is not necessarily an accurate representation of comparative value. For instance, in a forest with Big Leaf Maple and Japanese Maple trees, two Big Leaf Maple leaves might easily dwarf 5 Japanese Maple specimens.
- This provides a good opportunity to discuss the ways that our interpretation of data, as researchers, includes our perceptions of the data.
- Reset: for a final round, graph by species of tree (this can either be accompanied by field guides for students to look up and check their identification, or can be based on however students would like to identify and organize leaves by type of plant they came from).
- After they are done graphing, begin with the same question:
- “What conclusions can we draw from our data?”
- At this point, students might begin to draw correlations between the leaves collected and the biodiversity of the forest. E.g. “There are a lot of Big Leaf Maples in this forest.” Conclusions like this help bring out a third point of discussion around researcher bias, which connects back to the very beginning of this whole activity.
- Ask students, “How did we collect our data?” They will likely remember that they each contributed 5 leaves and they might even remember that you asked them to gather 5 “interesting” leaves.
- Follow up to ask them: “How do you think gathering ‘interesting’ leaves affects the data we have available?”
- Continued follow-up:
- “What data are we missing?”
- “How could we include that data in our study?”
- “How can we remove our interests, assumptions, and influence from the ways we collect and interpret data or the way we carry out a research study like this?”
- After they are done graphing, begin with the same question:
- Introduce the concept of researcher bias to provide a term for all of these factors and mention that this is something everyone can look for and point out as they carry out future investigations during the time together.
Analysis
Through the debrief questions after each round of graphing, students are able to reflect on concrete examples of how each different form of researcher bias entered into their graphing experience. This activity provides a tangible, low-stakes, iterative way of exploring and discussing these concepts within the context of scientific investigations. By prompting their thinking about how they arrived at their conclusions each time and the decisions they made along the way, you offer opportunities to practice researcher reflexivity, to question their own process, and to identify points of revision and improvement in the design.
I have led this activity several times now with different students each time and I use each debrief as a formative assessment to gauge how they are processing and analyzing the concepts and where more conversation or reflection might be helpful. During one session, after asking the question, “What data are we missing?” a student thought for a moment, looked up with excitement, then looked up again and pointed, exclaiming, ”What about all of the leaves that are still on the trees!” These moments of realization are what I hope students gain from this activity--when they discover that the answers they thought they’d arrived at are actually just the beginning of more questions and when they begin to see themselves as part of the science rather than separate or removed.
This activity was adapted from a leaf graphing investigation I first learned through Seattle Audubon.
The sequencing and discussion of researcher bias is new.
Reference
Kleinsasser, A. M. (2000). Researchers, Reflexivity, and Good Data: Writing to Unlearn. Theory Into Practice, 39(3), 155-162. https://doi.org/10.1207/s15430421tip3903_6
The sequencing and discussion of researcher bias is new.
Reference
Kleinsasser, A. M. (2000). Researchers, Reflexivity, and Good Data: Writing to Unlearn. Theory Into Practice, 39(3), 155-162. https://doi.org/10.1207/s15430421tip3903_6
