Ontological Objectivity, Research Methodology, and the Utility of Bias

I just finished having a conversation with B., a science teacher at the summer program where I am currently working. B. is actually an anomaly among scientists; he’s a firm believer in complex systems and taking a more subjective, imaginative view of scientific phenomena (e.g. studying protein folding patterns in an effort to design buildings that increase E.R. rapidity). Our discussion basically centered on whether or not ontological objectivity is possible, and if so, of how much use is it?

Let me begin by stating that I am in no way an absolute relativist (talk about an oxymoron). I do believe that absolute truths exist. To paraphrase Terry Eagleton, either there is a tiger in the bathroom or there is not; the tiger cannot be both there and not there simultaneously (sorry, Schrödinger). However, the number of these absolute truths is relatively small, and they are not extremely helpful in solving problems. When we begin to think about how to apply such knowledge, subjectivity inevitably begins to creep into our calculations, and with subjectivity comes bias.

An example: I am a researcher in a university lab. I discover that compound A affects the shape of the protein coat on virus B, which is directly responsible for Disease X. Okay, fine. Now what? Well, I probably want to design experiments to see if manipulating the protein coat of virus B with compound A makes the virus unable to survive in the human body, thus curing or preventing disease X. How do I go about investigating it? Here is where the biases and subjectivities come into play. Some subjective issues that directly impact my ability to solve this problem include:

1. if any funding body thinks Disease X is important enough to try to cure
2. if enough people contract Disease X to make a drug to treat X profitable
3. if Disease X presents differently in people of different genders, races, or geographic origins
4. whether or not I think it is appropriate to engage in clinical trials on animals (if I don’t, I’m not going to be curing Disease X, because the FDA won’t approve my clinical trial design)
5. if I can recruit a large enough patient pool to run Stage III clinical trials
6. if I believe the scientific method is the best way to solve this problem (if not, my drug is never going to make it onto shelves)

I could go on, and there are certainly counter-arguments that can be presented for each one of these issues, but you (hopefully) get my point.

I must note that I am not in any way saying that bias is necessarily a negative thing. Our biases may prompt us to look for new and creative ways to solve problems, to transcend disciplines to find solutions, et cetera. However, as human beings, ontological objectivity is almost impossible to attain. We can study things in a more objective or less objective fashion, but I just don’t think that humans can ever be entirely objective—we’re too enmeshed in context (personal, disciplinary, academic, religious, et cetera) to not have some sort of bias.

One bias that particularly galls me, however, is the bias towards the scientific method in non-scientific disciplines—particularly my own. Human beings are complex creatures and cannot be studied the way genomes can be sequenced. I think one of the most pernicious aspects of No Child Left Behind/ESEA is the limits it places on research. ESEA legislation states that to receive federal funding, schools must implement programs that have been proven to be effective. Sounds reasonable, right? Well, it depends on how you define the term “proven.” ESEA tends toward a restrictive definition of “proven” that includes only double-blind trials that have a significant sample size and use quantitative or mixed methods.

There are incredible problems with this definition

1. The issue of variation and generalizability. Human beings are not like drugs or chemical compounds—they are going to respond in wildly different ways to interventions.
2. One must take into account classroom context when trying to implement any sort of intervention. What may work well in one class may be a disaster in another—just ask any teacher who’s tried to teach the same lesson plan to two different classes in one day. Whether my students speak English as a first language, whether they are academically gifted, whether they have the requisite prior knowledge for a lesson, what their learning styles are—all these variables will impact the success rate of the intervention, and no clinical trial or quantitative study can possibly account for so much variation.
3. What may work in one community or school is only successful in that educational context. It is difficult if not impossible to extrapolate the intervention’s results to other schools or communities. Generalizability is just not something one can expect from most educational research. To pretend otherwise is specious at best and foolish at worst.
4. Restricting the definition of acceptable research also eliminates the use of any research that has developed from qualitative methods. This idea is doubly foolish if one considers that qualitative methods are particularly well suited to investigating complex systems that aren’t generalizable (like a classroom).
5. Education is not necessarily a scientific field of inquiry. For too long, schools of education have been attempting to gain status by pretending to be scientific. Whether or not scientific methods are the best ways of investigating the problems at hand has rarely been addressed.

If anyone is actually interested in this conundrum, and wants to read a much better analysis of the problem, I highly recommend the following article:

Eisner, E. (1992). Objectivity in Educational Research. Curriculum Inquiry, 22(1), 9-15.

Eisner explains pithily and with great elegance how the infatuation with objectivity is itself a bias and impossible in the world of Ed. Research.

Okay, off to complete lesson plans for tomorrow’s class (funnily enough, on ontological objectivity!)