How to evaluate qualitative research

People on my qualitative methods training courses often ask me how they can demonstrate the value of qualitative research when it appears to be so subjective or anecdotal, small scale or not generalisable. Well I have many and varied answers to this, but in this blog post I will focus on just a few. I have a feeling I will return to this at a later date. 

I want to start by saying it makes much more sense to evaluate qualitative research based on what it set out to achieve rather than criteria imported from another approach. So, if you are pondering this yourself too, then I would go back to the beginning and ask yourself ‘why did I choose qualitative methods? What was I hoping to achieve?’ Or ‘why did my client ask us to do qualitative research? What did they hope for as an outcome?’ When working with external clients it is always a good idea to start by clarifying their goals, so that if they ask if your results are statistically representative or objective you can remind them of your goals at the outset, and the true value of qualitative research. 

I also want to emphasis, right here, right now, that qualitative research is useful, practical, and able to inform actions.But how did I get to that point? 

In qualitative research surprise and relevance go hand in hand

Qualitative research has the potential to reveal deep insights that come to researchers as a bit of a surprise, or a bit left field. Often, these insights show that the research designers were asking the wrong questions in the first place, or had based their initial questions on some misunderstandings. If we really listen to people, and hear them, allowing our questions and preconceptions to be challenged, we not only understand them better but work together towards better, more relevant, solutions.

Here is one example. Anand Menon and Matthew Bevington in collaboration with the Joseph Rowntree Foundation conducted a few workshops (we might call them focus groups) with low income voters in order to tap into what Brexit means to people three years after the referendum. They unsurprisingly found a frustration with politics and politicians, feelings of being let down, of anger and betrayal. But as they continued to talk to people, they found that what unites them, what they really want to talk about passionately, is their local areas, local needs, and even some fairly straight-forward local solutions. Menon and Bevington argue “So while Westminster contorts over extra runways at Heathrow and high-speed train lines to London, it risks missing the bigger picture: many, perhaps most, poorer areas lack even the simple provisions that would allow them to succeed….Brexit has confirmed low-income voters’ worst fears about politicians. They see a litany of issues that need addressing in their local areas, and yet they sense there are no resources or political will to do anything about them.” 

These are findings that are exciting, interesting, meaningful, rich, and contextual, and what is more, they are findings that, if there is a will, we could actually do something with! This is because the researchers listened, heard, and therefore achieved relevant understandings of complex situations in people’s daily lives. 

Qualitative research is rich and contextual,enabling understandings to be acted on in a way that would be useful. The point of qualitative research is not merely to identify relationships between actions, behaviours or perceptions, as in a survey, but to look more deeply into how things relate in what ways over time and under what conditions. this leads to the use of words such as nuance, richness, depth, detail.

Qualitative research is sensitive

It is often argued that qualitative research is useful for sensitive topics,or issues that are difficult to talk about. Why is this of value? 

Let’s look at another example. Kate Reed, Elspeth Whitby and Julie Ellis used ethnographic methods to explore practices around infant post-mortems. They followed hospital staff as they went about their work, and interviewed staff and parents, and uncovered all sorts of caring acts such as bathing, dressing and talking to babies that meant the whole process could be far more meaningful and gentle than might be imagined. Their research led to changes in professional training and information sheets for parents that led quickly to increased take-up of minimally-invasive post-mortems in Sheffield, where the research took place. 

The research is meaningful, insightful, and revealing of the things people sometimes do without even realising they do it. It touches on issues of love and care, and is sensitive to emotions. But these alone are not of value. The added value comes in at the point of practical and useful applications of these insights; applications (or implementations) that have a chance of working because, first, the qualitative researchers took their time to listen, to hear, and to observe

Qualitative research is responsive

To take the ideas above just a little further, qualitative researchers learn as we go and we develop our ideas as people talk to us, as we observe practices, and as come to understand. This means we are better equipped to question our own biases. Indeed, I would argue that qualitative research can be less subjective than research that determines the definition of the problem, the questions to be asked and the order in which to ask them before starting to even engage with participants. Arguably qualitative research is objective in its desire to understand the human world in all its diversity and without prejudice. This does involve working with participants to achieve a shared understanding of a situation. But is trying to remain detached, refusing to engage or respond, trying to extract difficult and sensitive responses without any reciprocation actually any more objective? Where does that leave our definition of ‘objectivity’? I think I am going to have to come back to this one….

To conclude for now: we often talk about qualitative research as meaningful, focused on understandings, interested in emotions and experiences. We call it exploratory, giving insights into lived experience, or examining the standpoint of the research participant. But I would argue these amount to its values. 

Qualitative research is OF VALUE because, when it does all these things well, the insights produced (or the results, findings, or conclusions) are far more likely to lead to actions, interventions, recommendations, that are likely to work in practice. And this is the case, even where the only intervention we can come up with is to understand people better rather than condemn, criticise or wonder at them. 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s