Measurement and Interpretation: Let Us Speak No More of Quantitative and Qualitative Research

I have long thought that the division between quantitative and qualitative research was a false divide. There is no pure quantitative nor pure qualitative research. There is always a qualitative element in quantitative work – you might be counting things, but the choice of what’s relevant to count is an inescapable and qualitative decision. Likewise, there’s always a quantitative element in any qualitative research, even if only in the rudimentary sense that it makes a difference whether there’s a lot of something or only a little, whether something is always occurring, occurs every day, once a year, or is a unique occurrence.

More and more, though, I begin to think that the use of the terms distracts from rather than facilitates scholarly communication and would be better replaced by an emphasis on measurement and interpretation. Many social scientists, when asked, pay lip service to the notion I outlined above that there is no pure quantitative or qualitative research, but then go on acting as if there were. This, I think, is done largely uncritically and at least partly (if not largely) out of mutual contempt for number-fetishizing quant types and muddle headed, fuzzy thinking qual types. If we chucked the qual and quant labels, perhaps we could better focus on things that all decent research has in common (whether everyone knows it or not): measurement and interpretation.

Measurement: There is no immeasurable

A lot of “qualitative” social scientists, including most cultural anthropologists (including myself much of the time), tend to be wary of “quantitative” research because they perceive it as ignoring things that are not easily counted and uncritically or simplistically counting things that seem easy to count. Frankly, a lot of “quantitative” work does do these things, though there’s also a lot that doesn’t. What could be better recognized by some quant types is the interpretive nature of choosing what to count, but what qual types could recognize is that we’re all engaged in measurement. There are phenomena that are not easy to count, but there are no observable phenomena that are not measurable.

There are different sorts of measurement. Some things can only be measured in fairly basic and imprecise terms – the binary measurement of the simple presence or absence of a phenomenon or trait, or rough measurement of quantity, e.g. something is absent, present in small quantity or frequency, or present in high quantity or frequency. Other things can be very precisely measured. So, highly “qualitative” ethnography involves measurement just as much as the most “quantitative” of quantitative sociological research. Once we recognize that we’re all involved in measuring, we all ought to measure things as precisely as possible – sometimes that might involve quantification and in other cases might involve simply notation of the presence or absence of something. There’s no reason to be wary of measurement, but good reason to be wary of measurement that is less precise than it reasonably could be or purports to be more precise than it can be or is.

Interpretation: What’s the Significance of Statistical Significance?

As with measurement, all research involves interpretation whether we realize it (or like it) or not. I alluded above to the interpretive quality of measurement – knowing what it makes sense to measure is an interpretive maneuver. Further, it is always necessary to interpret the results of measurement. (Just as quant researchers are often more aware than qual of the need to measure, qual researchers are often more aware of this fact than quant.) Measurements, and even basic analyses, alone never mean anything, e.g. an analysis indicating statistical significance of a set of data doesn’t indicate at all what the meaning of the data is, but simply that there's a low probability that the data are as they are by random chance. There’s no reason to be wary of interpretation, but good reason to be wary of uncritical interpretation not based on sound logical argumentation and good measurement or to be wary of interpretation by those not aware they are engaging in interpretation.