Within and Across Case Comparisons with ATLAS.ti
Written by: Dr. Susanne Friese
Further Analysis after Coding – Within and Across Case Comparisons with ATLAS.ti
This article begins with a short methodological introduction to within and across-case analysis and then shows how such an analysis can be implemented in ATLAS.ti. This chapter shows how you can use the Code-Document Table, the code co-occurrence analysis, global filters and also networks to facilitate such an analysis. The examples shown are based on a small sample project that is used for illustration only. The screenshots were created in a beta version of ATLAS.ti 9 so that you can look forward to what’s coming soon. Most of the procedures, however, are also available in version 8.
In qualitative research, analysis is built from stories told by research participants, observations, reports and the like and based on the identification of key aspects in the phenomenon under investigation (Ayres et al., 2003). These key aspects are referred to as categories or themes. They can be identified a priori or developed during the process of analysis (Coffey & Atkinson, 1996; Patton, 2002; Sandelowski, 1993). Categories or themes may be manifested across individuals, single documents, or might apply to all participants or data in the study.
At times, qualitative researchers only produce a list of main themes (or categories and their sub codes) and do not take the analysis further. Richards (1998) referred to this type of analysis as ‘garden path analysis’ (p. 324). This means that the list of themes found in the data and their variations are only described, but they are not related to each other. A list of themes by itself, however, has little explanatory power. They contribute little to idiographic generalizations that are typical for qualitative research. They are only its ingredient. In order for an analysis to be complete, the themes (or categories) need to be reintegrated so that it can be seen how they work together in an actual (or constructed) case (Ayres et al, 2003).
In statistical research, the analysis is based on variables and the use of measurements. Just as the list of variables and their distribution only constitute the raw material for a statistical analysis, so too does the list of themes in qualitative research. If you want to take the analysis a step further, the identified key elements need to be related to each other. Within-case and across-case analyses are a means to do that. Through recontextualizing the data, a within-case analysis helps you see how key aspects vary in individual cases; an across-case analysis allows you to develop a synthesis that captures the essence or variation of experience across individuals to see patterns and commonalities. Thus, both approaches are necessary to capture the true nature of an experience, both through its parts and as a whole. Only then can individual experience be recognized in a generalized way (Ayres et al., 2003).
How to conduct Within Case and Across Case Comparisons in ATLAS.ti
Depending on how a case is defined in your data set, you have three possibilities to compare data within and across cases. If your case is on the document or document group level, you can use the Code-Document Table. If you analyse focus group data and define each respondent as a case, you can use the special focus group tool to auto code all of the respondents and their characteristics. With such data, the cases are embedded within each document and you need to use the Code Co-occurrence Table for a within or across case comparison. The Code Co-occurrence tool is explained in more detail in the following article: Co-occurrence Analysis with ATLAS.ti. Another option is to use the Query Tool in combination with the scope option, which allows you to focus your analysis on a particular case, or to compare data across cases.
In this article, you will learn how to conduct a within-case and across-case analysis if your cases are based on the document or document group level. As version 9 of ATLAS.ti will be released soon, the analysis shown here has been conducted in the new version. Apart from the Sankey diagrams, you can do all of what is shown in this article also in version 8 of ATLAS.ti.
The third option, using the Query Tool for such a comparison (and other queries), will be the topic of the next article in this series.
Using the Code-Document Table and Co-occurrence Analysis for Within and Across Case Comparisons
The Code-Document Table allows you to compare the distribution of code frequencies between documents or groups of documents. You can see how often a code has been applied in a document or document group, plus you can access the data behind each of the numbers in the table to recontextualize them. Being able to breakdown the codings of a code by individual cases or groups is often an eye opener. When you are coding the data, you work in a linear fashion through each document attaching all codes that apply. Using the table allows you to inspect the data for selected codes or code groups only; you can zoom in on an individual case level or expand the analysis to an aggregated level represented by document groups.
To illustrate the various analysis steps, a sample project is used whose data base consists of evaluations of the computer game Minecraft found online. Extracted are evaluations by parents who do not play the game, parents who have some experience playing Minecraft themselves, and others who play the game but are not parents.
Figure 1 shows the list of categories and sub codes that were identified. You can see the age ranges for which the game is recommended, the perceived benefits and downsides of the game, whether it is recommended or not and tips for parents; the number or text segments that capture a ‘positive’, a ‘positive but’, or a ‘negative’ evaluation; you find information about how the game is described, the features of the game, the various modes, and the online community. I could continue to describe the various categories in more detail, but this would mean I don’t take the analysis beyond the ‘garden path’.
To see how the various categories are connected, compare the various groups of respondents, here: parents who play, parents who do not play, and players (other), is a useful next step. To create a Code-Document Table:
- In ATLAS.ti Win, select the Analyze ribbon and from there Code-Document Table.
- In ATLAS.ti Mac, select the Analysis menu and from there Code-Document Table.
On the left-hand side, you see four selection lists to create the table. You can select items from the list of codes and code groups for the rows, and items from the list of documents or document groups for the columns of the table. Once you have created the table, you can also switch rows and columns.
Figure 2 shows a comparison for those three groups for the categories BENEFIT and DOWNSIDE. The table cells show the absolute frequencies of how often a code occurs within a document group. The table colouring gives you immediate feedback on which aspects are mentioned most or least frequently by each group (new in version 9). If you click on a cell, you can read the data that is behind the number, and you can also see whether other codes have been applied (new in version 9). With a double-click on a quotation, you can inspect the data within the larger context of the document.
The table also gives you some information about how often a code occurs in the entire data set, the number of documents in a selected group, and the number of quotations in each group.
Given the above example, we see that there are 18 respondents in the ‘parents who do not play’ group, 7 respondents in the group ‘parents who play’, and 15 respondents in the group ‘players (other)’. The total number of quotations in the first group is 108, as compared to 63 and 74 in the two other groups. Thus, comparing absolute frequencies does not give you an adequate picture. Therefore, you can normalize the data. The reference point for normalization is the document with the highest number of quotations for codes in the table. In this example, it is the first group. This means that the number of quotations per code is multiplied by the ratio of the sum of all quotations of the reference group (here: 46) and the sum of all quotations of the respective other groups. In the above example, the ratios are 46/22 and 46/13.
- In the Windows version you find the normalization option in the ribbon, in the Mac version under options.
- As normalization mostly gives you numbers with decimal places, you may want to display relative frequencies in addition to absolute frequencies. In ATLAS.ti Windows you can activate relative frequencies in the ribbon; in ATLAS.ti Mac click on the option icon.
- If you display document or document groups as columns, activate row relative frequencies for an across-case comparison.
Normalization highlights that people who play themselves report more benefits and fewer downsides. The only downside that players from a non-parental perspective report is that the game might become boring over time. This can also be illustrated by the Sankey visualization of the Code-Document Table (new in version 9). The benefits are shown in green, the downsides in red.
Overall, parents who do not play report more benefits than downsides. They mainly fear the social and emotional dangers for their children when playing online with strangers. Interestingly, parents who play themselves see less educational benefits in the game than the respondents from the two other groups. For them, the creative element of the game appears to be more important. We actually are now cycling from an across-case comparison to a within-case comparison. Both the table colouring and the Sankey diagram also give an indication of the within case distribution of the selected codes.
- If you want a numerical breakdown of the within-case distribution, activate column relative frequencies. Optionally, you can deactivate absolute frequencies, as shown in Figure 6.
To look further into the similarities and differences, the necessary next step is to read the data that is behind each of the table cells. To do so, click on a cell and read through the quotations in the Quotation Reader that is displayed next to the table (version 9 / In version 8, the quotations are displayed below the table). In the article Co-occurrence Analysis with ATLAS.ti I already pointed out the importance of writing during the process of analysing qualitative data. This can be done in ATLAS.ti memos.
- When creating a table and looking at code distributions within and across cases and at the quotations that belong to each cell, open up a memo alongside and write down your observations. If you find interesting quotations, you can immediately link them to the memo or insert them via copy and paste.
When you begin writing, relations in your data will emerge – I would say – almost by themselves. Of course, it takes some analysis experience to be able to see what enfolds in front of your eyes. It is, however, simply a matter of doing it. I would like to re-iterate a quote I already referred to in my last article: You need to do analysis in order to understand analysis (Freeman, 2017).
In the process of writing, I realized that there might be some interrelations between the codes of the category BENEFIT. See the note in the memo above in Figure 7. This can be examined by running a co-occurrence analysis. The results are shown below:
Within-Case and Across-Case Analysis by Means of the Global Filter Setting
As can be seen in Figure 8, ‘buildings things’ was frequently mentioned together with ‘creative’ and ‘educational’. Whether this holds true when you compare the data across cases can be further examined by setting global filters for each group.
- To set a global filter, open the document group section in the Project Explorer. Right-click on a group and select the option Set Global Filter.
What we can see here is that the relationship between ‘building things – creative – educational’ only applies to the first group. The educational aspect does not figure in for parents who play themselves; they associate ‘buildings things’ more with the collaborative aspect of the game, skill development and creativity. For other players, the relationship is there but literally very thin.
Global filters can also be applied to the tables. Figure 10 shows an example. We have seen that the game is mostly perceived positively, but there are two respondents in the sample that are very negative. As Ayres et al. (2003) emphasized, it is important to look at individual cases in order to not minimize or even lose their voice and the significance of the single perspective through the generalization of the overall pattern.
Figure 10 shows the results of reducing the data to two individual cases captured by the document group ‘Evaluation: negative’. These two respondents did not mention any benefits, only the downsides of the game, describing it as a dangerous place, not suitable for children.
While coding the data, I already noticed that the perspective that Minecraft is a dangerous place was not supported by other respondents, and I connected their statements about the dangerous side of the game via hyperlinks.
Hyperlinks can also be visualized via a network (Figure 12). In the network, we see responses from case 36, case 27, case 41 and case 23. Thus, through networks we can also examine across-case comparisons, as shown here on the individual document level.
Step-by-step, through cycling back and forth between individual cases, within-case and across-case comparisons, you can advance your analysis. This process was described by Tesch (1990) as a hermeneutic spiral. The generation of meaning and, subsequently, the idiographic generalization is achieved through an iterative process of comparisons at all levels, and in all accounts.
Qualitative researchers typically collect data from multiple participants and contexts. When analysing the data, the researcher must develop an interpretation that, on the one hand, reflects and is true to the experience of the individual or a case, and on the other hand, applies also across all the various accounts that make up the data set. In this article, I have shown how to leave the ‘garden path,’ and how you can begin to see connections in your data, be it within or across groups, or within or across categories and themes. With the tools ATLAS.ti provides, you can easily iterate between information that is relevant to all respondents, to only a subset in your data, or exclusively to only one or a few informants.
I would like to point out that the examples shown in this article are based on a small sample project with a limited number of respondents. The absolute frequencies are small, and relative frequencies therefore exaggerate the significance of a code. The same applies to normalizing the data. You always need to evaluate within the context of your own data whether these options are meaningful or not.
Ayres, L., Kavanaugh, K. and Knafl, K.A. (2003). Within-Case and Across-Case Approaches to Qualitative Data Analysis. Qualitative Health Research, Vol. 13 No. 6, July 2003, 871-883.
Coffey, A., & Atkinson, P. (1996). Making sense of qualitative data. Thousand Oaks, CA: Sage.
Freeman, Melissa (2017). Modes of Thinking for Qualitative Data Analysis. NY: Routledge.
Friese, S. (2019). Qualitative Data Analysis with ATLAS.ti. London: Sage.
Patton, M. Q. (2002). Qualitative evaluation and research methods (3rd ed.). Thousand Oaks, CA: Sage.
Richards, L. (1998). Closeness to data: The changing goals of qualitative data handling. Qualitative Health Research, 8, 319-328.
Sandelowski, M. (1993). Theory unmasked: The uses and guises of theory in qualitative research. Research in Nursing & Health, 16, 213-218.
Tesch, R. (1990). Qualitative research. New York: Falmer.
Friese. S (2020). Further Analysis after Coding: Within and Across-Case Analysis with ATLAS.ti. Retrieved from https://atlasti.com/2020/27/07/within-and-across-case-comparisons-with-atlasti/
About the author
Dr. Susanne Friese
Dr. Susanne Friese started working with computer software for qualitative data analysis in 1992. Her initial contact with CAQDAS tools was from 1992 to 1994, as she was employed at Qualis Research in the USA. In following years, she worked with the CAQDAS Project in England (1994 – 1996), where she taught classes on The Ethnograph and Nud*ist (today NVivo). Two additional software programs, MAXQDA and ATLAS.ti, followed shortly. Susanne has accompanied numerous projects around the world in a consulting capacity, authored didactic materials and is the author to the ATLAS.ti User’s Manual, sample projects and other documentations. The third edition of her book “Qualitative Data Analysis with ATLAS.ti” was published in early 2019 with SAGE publications. Susanne’s academic home is the Max Planck Institute for the Study of Religious and Ethnic Diversity in Göttingen (Germany), where she pursues her methodological interest, especially regarding qualitative methods and computer-assisted qualitative data analysis.