Last week, at the end of a long day training in the art of qualitative interviewing, one of the course participants asked me: ‘but how can we show it is not just all anecdotal?’. I felt frustrated, to say the least, but I must have failed somewhere in my explanations of the value of a qualitative interview.
There is such a lot of angst about how to assess the value of qualitative research that still, it seems, we haven’t fully addressed this. In my experience, the problem often seems to come from the fact that people doing qualitative research feel they need to explain why they don’t do certain things – like testing for reliability, representativeness, and validity.
How do we know quantitative work is good quality? How many people have actually asked to see all the questionnaires, and checked them all? How often is the basic operationalising of a concept or the choice of potential responses queried? How do we know the design of questions is not anecdotal?
It seems to me that qualitative researchers need to 1) stop being defensive and 2) recognise what they aim to achieve, before 3) thinking about how to show they’ve achieved it.
Let’s start with the notion of an anecdote.
It is anecdotal that every week my neighbour puts her bin out near my car and it is not always easily visible. So, every week, I check before getting in the car to ensure I don’t drive over it (again). My experience has affected my behaviour, and it works in practice!
This is the same logic we use for qualitative research. We are trying to understand how people’s experiences and understandings affect (or might affect) their behaviour. I can’t design a survey to test for this because I won’t know what questions to ask, or responses to suggest. It is irrelevant whether the experience or understanding is an anecdote or not. It is how the person’s life is shaped by experience and how they communicate that, that matters.
Furthermore, if by anecdote we mean something said quickly and flippantly without depth, then one thing we should be ensuring as qualitative researchers is that we explore in depth what people tell us, by listening and hearing, and working to achieve complexity, nuance, and richness (and perhaps doubt and ambivalence).
A plea to include the people, stories, and lives in qualitative research writing, and to avoid Disembodied Quotes
When I am delivering Research Methods Training, I often ask participants to please try to avoid using disembodied quotes when writing up, or presenting. I thought it was time I clarified what I mean by this, as it is not written anywhere (to my knowledge). By disembodied quotes I mean this sort of thing:
One of the themes that emerged in the study was that research participants felt the GPs were using terminology they couldn’t understand, and they often felt both embarrassed to ask for clarification, and pressed for time.
“I always feel the doctor is in a rush, so when she told me I’ve got xxx, I didn’t want to ask what she meant. I know the answer would take a long time and I still not might understand” (woman, 40s).
This is OK. It is clear what is going on here, but it lacks richness, context, and personality. We learn very little about this woman who is speaking, and she doesn’t even have a name. Compare it with this.
Research participants were reluctant to ask questions when they felt their lack of understanding was vast, or when they believed the GP was under a lot of pressure with regards to time. Sophie, a mother of two in her 40s, had visited her GP for her own health (as opposed to taking her children) for the first time in ten years . She was very anxious about her symptoms, and had tried to look them up on the internet but had felt even more confused. Her appointment was the last one in the morning, the surgery was empty apart from herself and one other person, and the GP, Sophie told me, looked frazzled and tired. Under such conditions, she didn’t feel she could ask the questions she needed to. Sophie told me,
“I always feel the doctor is in a rush, so when she told me I’ve got xxx, I didn’t want to ask what she meant. I know the answer would take a long time and I still not might understand” (Sophie, 40s, mother of two children).
This uses many more words but if the point is to help GPs understand what meanings and understandings patients bring to the interaction, then it has done a much better job than the shorter introduction to the quote, above.
Also, when people use the disembodied quotes approach they usually follow an explanation with a series of quotes, as if they are trying to show how important this insight is quantitatively. So, the word count may well end up the same, but without the richness and context.
This desire to demonstrate the quantity of a response illustrates a basic misunderstanding of the value of qualitative research, which is to reveal meaningful insights that can be acted on to improve society.
Note: the take-home message is at the end!!!
“This is a practice of doing research, informed by a sophisticated inductivism, in which data collection, analysis, and writing are not discrete phases but inextricably linked”. (O’Reilly 2012, p.5).
As we know
“A deductive approach to research is one where a hypothesis is derived from existing theory and the empirical world is explored, and data collected, in order to test the truth or falsity of the hypothesis. A simplistically inductive approach to research is one where the researcher begins with as open a mind and as few preconceptions as possible, allowing theory to emerge from the data. A more sophisticated inductivism views theory as precursor, medium and outcome of ethnographic study and writing”. (ibid. p.29)
And we know that ethnographic research tends to be more inductive than deductive, to work hard to make ideas earn their way, to want to hear and sense the field, to learn from research participants, and to achieve a way of understanding that is meaningful.
Further, the notion of abduction (which may be a better way to understand iterative-inductive) recognises an on-going interaction between the researcher, the research field, and externally sourced and internally emergent (or created) ideas over a period of time and as part of the practice of research.
However, none of the approaches above adequately deals with the role of the literature review or the role of a theoretical framework in ethnographic research that wishes to be iterative inductive. As social science develops, transfers concepts to new settings, builds on prior understandings in given fields, it becomes more and more appropriate to consider extant (or external?) or pre-existing theories and concepts. Think of concepts such as institutional racism, stigma, theories of learning, cognitive dissonance theory, etc. Some of these have become almost common-sense terms in everyday parlance. No researcher can ignore what they already know. Furthermore PhD supervisors and colleagues often ask an ethnographer: ‘what is your theoretical framework?’ implying that they will have a toolkit of theories and concepts to pull out of a bag and use in the field.
As time has gone by, my own research has taken shape, developed, and built on previous ethnographic understandings to become something of a body of research on British abroad. I am increasingly using a practice theory (or structuration) theoretical framework, which shapes what I do and how I make sense of the worlds I inhabit. This theoretical framework is somewhat independent of the research, the field, and our iterative-inductive understandings. It is, for me, a starting point, but strangely it has come to me over time as an end-point, a reflection and reconsideration of earlier work. I use it to construct Practice Stories
“Practice stories pay attention to people’s feelings and emotions, their experiences and their free choices, but also to the wider constraints and opportunities within which they act. More than that, practice stories take account of how these different features of social life interact, and thereby how structures (like social classes, for example) get produced or reproduced. Theoretically, practice stories can draw from a wide range of social theory that comes under the description of structuration theory or practice theory”.
However, a practice theory framework does not exclude the use, application, or amendment of more substantive series that are created as part of the research or that are imported from previous studies. As I have argued in International Migration and Social Theory.
So, perhaps a theoretical framework is actually more of a broad-brush all-encompassing approach to understanding wider aspects of human behaviour, while it is the more substantive theory and concepts that we wish to hold lightly (to quote one of the recent summer school participants, a design ethnographer), or use as sensitising concepts (to quote Herbert Blumer) in iterative-inductive ethnographic research.
I’ve been teaching Ethnography at the Essex Summer School the past two weeks. It has been amazing. The participants have worked incredibly hard and have produced some wonderful research projects. I’m trying to think what it is about the summer school that is so warm and creative and makes the participants so enthusiastic. I think it might partly be that we are all there together for an intensive period of time. Also, everyone has chosen to come and already has a passion for the subject they have chosen. All I can say is that it is the best possible teaching experience. Imagine students that do all the reading that you ask them to do and more besides, students who work well beyond the classroom hours and have to be reminded to take a break, students who have a lot to contribute, who think, challenge, contradict, and bring their own experiences and insights so that as a teacher you are not only teaching but also learning. I have come home with a list of references to follow up, things I am excited to read, lots of ideas buzzing around my head, and an enthusiasm for my own work. Thank you to the especially enthusiastic and inspiring participants this year. You know who you are. And thank you to Mel and Jo and Dan, who support us all so well.
Participants eagerly take part in an activity
I spent a really lovely few days in Edinburgh recently providing training for The SRA. The courses were Designing Qualitative Research; Qualitative Interviewing; Conducting Focus Groups. All three courses were full up, with waiting lists, and the participants were fantastic – so much expertise and enthusiasm in one room! Some participants attended two or three days so we got to know each other well, and people made new friends. I tried using ‘play dough’ for the first time in the Focus Groups course. I think it went well – we had fun anyway. Participants came up with some interesting interpretations of un/healthy! We also learned that it might be better to get participants to shape their dough in groups rather than on their own, to avoid the pressure to perform, and the silence. It went very quiet while they sculpted seriously.
It is always lovely to have a few days in Edinburgh and meet up with old friends as well.