What follows is prepared text from a meeting I recently addressed. It is a rough outline of the differences between quantitative and qualitative research methods. It is influenced principally by David Silverman’s work on qualitative data analysis and is an attempt to address both methods relationally.
“I have previously recalled how reviewed research on minimising selection bias demonstrates that it is not in the choosing of the groups of people we engage with that matters so much as the assessment of the impact of the selection bias. For the present study where [project description] and using a qualitative method, it is more important to be aware of the impact of the biases. This is a post-data gathering stage process. In seeking to plan a project like this, using conversation and discourse analysis, representativeness of response is not important. In quantitative research, respondents are chosen as statistical representatives of their peers, e.g. mothers of school-going children aged 25-44 asked questions give responses which are to represent those of all mothers in the same demographic.
In qualitative research, interpretive understandings of specific contexts are sought. In this field of research, the accounts that people give of their lives and experiences are not simply representations of the world; they are part of the world they describe (Hammersley and Atkinson, 1983). In this sense, descriptions of their experiences are accretions of everyday life and are not ‘problems’ to be solved in analysis. Variation between what people say about a specific topic are not divergences from a fixed point. Rather, they are common sense devices (Cicourel) for making sense of our own environments.
The statements made by people in the course of research of this manner represent two things. Firstly, a narrative that is shaped by and shapes the world in which she lives, for example when a parent says that she prefers to enrol their child in the nearest school because that is what other parents do, she is framing her own experiences within the behaviour of others. Secondly, this is a factual statement that is uttered by those we initially trusted to become respondents / informants. Therefore, in our example, we might say that we now have evidence that parents tend to choose the nearest school for their child. This in effect is how the data that is gathered both “reproduces and rearticulates cultural particulars grounded in patterns of social organisation” (Silverman, 1993). It is what both the interviewer and the interviewee (as part of a context-based group for our own purposes) are doing through talk that matters for a research project of this nature.
In the 1970s and 1980s, considerable concern was expressed by positivists and interactionists that a bias existed in qualitative research methods. Informants and respondents were said to be distorting social reality or concealing what the interviewer most wants to know. Instead of seeing respondents to qualitative research as somehow not fully moral beings whose answers are not ‘up to scratch’, we might treat such responses as “displays of perspectives and moral forms” (Silverman). In the 1990s, those practising radical ethno-methods created a deeper understanding of ways of meaning: the epistemic interview. The respondent is not merely someone who fills up the data receptacle which the research figuratively holds. Instead, she is a participant in a two way discussion about the nature of truth itself, being challenged during the interview to question her own presumptions and prejudices.
While the present project may not require the employment of this epistemic model, there is a case to be made for challenging the basis upon which the participants in the research understand what [project subject] may mean to them. Through such a probing relationship with the group in a specific context, an understanding of the parents’ position can be reached which does not rely on them being representative of all parents. In this regard, choosing a qualitative method over others structures the data from the outset as non-representative because that leads to a different understanding of a process of data gathering. Qualitative research methods like discourse analysis are collaborative and can lead to new learning for both the researcher and the participant. Some writers claim that the boundaries between the two are often made more porous by certain forms of group-based work.
So what does the reliability of data mean in this context? The identification of categories used to analyse text (in this project) will be assisted by a computer program called STAS. This program helps to code the text gathered during the data collection stage of the project. Of course, STAS is only as reliable as the categories that the researcher wishes to use. Some of the categories fall out of the initial analysis, others require a deeper reading of context. This can be facilitated by inter-raters, or other people who analyse the data from scratch and can identify themes from a reading. Transcripts can also be analysed in a group setting without seeking to become trapped in finding the ‘perfect’ conversation through transcript reading. In terms of validity, we might contrast again the quantitative with the qualitative. Results from a large, numerically-based, survey can be refuted and of course there are statistical tests that can be applied to test the rigour of certain assumptions. But these processes can both conceal and reveal social processes in their own context. This is characterised by asking the simple question most often applied to surveys: how do attitudes relate to what we actually do in practice? (Sudnow) Some areas of our social reality cannot be measured by statistics. As a practising researcher I am frequently confronted by this question, most particularly when the results defy our common understandings.
Conclusions are not explanations and this is more valid for the results of qualitative analysis that for that of numeric analysis. However, what is often recorded as merely anecdotal also has its own tyrannies. Believing that anecdotal evidence provided by the analysis of talk can bring policy arising from such a project into disrepute. Glaser and Strauss’s ‘grounded theory’ made some attempts to overcome these compromises. We have already seen in the 2008 work carried out by this researcher that:
- Parents are able to get their first choice of school but plan well in advance of enrolment, which high proportions find easy to complete.
- High percentages of parents wish to see a continuing role for the Churches in the provision of primary schooling and for most parents, faith in education retains its importance.
- Teachers have a role to play in the strengthening of the faith of their child and that the school, even with increasing classroom religious and ethnic diversity, is still the best place for sacramental preparation.
- Parents are sufficiently involved in the preparation for First Holy Communion of their own child but would like to see more involvement from parents generally.
- There is a correlation of reasonable strength between the ‘religious’ factors in the survey, i.e. respondents who believe that the school and teachers have a role to play in faith formation are likely to choose schools because of the worship provided and the religious education provided in this particular school.
- Parents choose a school because of the chances it gives to allow their child to progress through the education system and prioritise the school’s ability to meet their child’s needs and abilities.
- Religious factors are important for parents in choosing a school for their child but not as important as more formal pedagogical factors.
Triangulating these facts arising from this project with the outcomes of considered discussion with parents will give a deeper understanding of how to frame policy. It is not the intention that a ‘true’ reason for choosing one school over another exists and a researcher merely has to reveal people’s ‘real reasons’. Rather it is that the “situated work” (Denzin) of the analysis stage is seen to be distinguishing the ‘why’ from the ‘how’, and prioritising the former when it comes to triangulating one project with another.”