Using ATLAS.ti for a Discourse Analysis Study of Blogging in an Educational Context

July 20, 2014

Authors: Trena Paulus* and Jessica Lester

Abstract

While some scholars have claimed that data analysis software tools are not useful for discourse analysis work (Macmillan, 2005), in this presentation we report on our use of ATLAS.ti for conducting a discourse analysis of blog posts and comments made in the context of an undergraduate nutrition course (Lester & Paulus, 2011; Paulus& Lester, 2013). We describe several aspects of our analysis, including: managing the data through document families and quotations, collaborating as a team through the merging and memos, engaging in “unmotivated looking” through memos, maintaining a focus on discursive actions through hyperlinking and network views, and narrowing the analytic focus through codes and queries. While we found ATLAS.ti to be extremely useful for documenting our analytic decisions  in a transparent and systematic way, we also call for features that would allow the analysis of online interactional data in a more seamless way.

Introduction

While a growing number of qualitative researchers are using data analysis software, few describe in much detailhowthey use these tools, such as which features of the software were used (Paulus, Woods, Atkins & Macklin, 2013). Even more troubling is that there is a persistent misconception that packages such asATLAS.ti were designed for only one type of analysis, usually grounded theory (Davidson & di Gregorio, 2011.) Published work that has sought to describe the use of tools for other methodologies, such as ATLAS.ti, often take a “media comparison” approach (Clark, 1994; Kozma, 1994)with sometimes contradictory results, such as Schoenfelder (2011) who compared the use of two prominent software packages (NVivo 8 and MAXQDA 10) for discourse analysis and Macmillan (2005) who compared NVivo, MAXQDA and QUALRUS (versions not specified) for discourse analysis work. Schoenfelder (2011) concluded that both had useful features, but that MAXQDA was more flexible and thus more useful for his purposes. In contrast, Macmillan (2005) concluded that none of them were suitable for discourse analysis work because of their inherent reliance on coding.

We tend to agree with Schoenfelder’s (2011) observation that perceived limitations of CAQDAS packages are often “basedon the individual analytic approach rather than on principal methodological incompatibilities” (section 1).Fielding (2008) pointed out that most users under utilize the features of data analysis software, and so it may be that individual analysts are not fully aware of what the software is capable of and are thus disappointed with their experience. This is consistent with Fielding’s (2008) response to technological determinists who fear that tools inevitably drive methods. He argued that the software tools were not designed to support only one type of qualitative analysis, but can be appropriated by users in a variety of ways.

To address these fears and lack of awareness, in this paper we describe several aspects of our use of ATLAS.ti for a discourse analysis study of undergraduate blog data, including: managing the data through document families and quotations, collaborating as a team through the merging and memos, engaging in “unmotivated looking” through memos, maintaining a focus on discursive action through hyperlinking and network views, and narrowing the analytic focus through codes and queries.

Context of the Study

The analytic process we describe here is from a study of an instructional task for undergraduate students enrolled in a large nutrition lecture course (Paulus & Spence, 2010). Our findings are described in detail in Lester and Paulus (2011) and Paulus and Lester (2013). In short, students were assigned to small groups in which they shared a blog post on their experiences with or beliefs about dietary supplements and then commented on the posts of other students, all using the blogging tool within the university’s course management system. The focus of our analysis was on how students wrote about their experiences with dietary supplements prior to attending a lecture on the topic (pre-lecture posts) and how they wrote about what they learned after attending the lecture (post-lecture posts.)

Data Management Through Document Families and Quotations

Konopasek (2008) has described ATLAS.ti as a kind of “textual laboratory” within which every aspect of a research project can be connected, made visible on the screen,and instantly accessible. Being able to read, annotate, code, visualize, and interpret in one space, within reach of not only our data but also our theoretical literature, research proposals, and ethics guides, improved our ability to systematically document all decisions that we made throughout the research process (diGregorio & Davidson, 2008).Our use of ATLAS.ti allowed us to “make visible” our analysis process in ways that it would not have been possible otherwise. We started our study documentation in ATLAS.ti by creating a free-standing memo in which we explained the purpose of the study, the research questions, the date we created the project in ATLAS.ti, and the purpose of our first layer of analysis. Because this was a memo about the project in general we did not attach it to any data or documents. Writing memos ensured a transparent and reflexive audit trail of all of our decisions.

ATLAS.ti was particularly useful for this project because of the large amount of data: over 300 blog posts and over 1600 comments made by 168 students in 14 discussion groups. Two features were particularly useful in managing our dataset. First, we were able to track the known characteristics of our data (Friese, 2012; Lewins & Silver, 2007) using the primary document families. Families allowed us to group our data by whether it was a blog post or a blog comment, which discussion group it came from, and whether it was a pre-lecture or a post-lecture post or comment (see Figure 1).

Figure 1

Figure 1. Primary document families

The second feature that was useful was the ability to create units of analysis using the quotations feature. ATLAS.ti’s ability to create quotations independently of codes or memos provides greater flexibility for the micro-analysis of data than the other CAQDAS packages (Paulus, Lester, Dempster, forthcoming.) Each blog post and each comment were designated as a quotation (see Figure 2), and memos and eventually codes were attached to these quotations individually and then merged.

Quotations as units of analysis

Figure 2. Quotations as units of analysis

One constraint with ATLAS.ti is that there is no way to easily import online interactional data such as blog posts and comments without either copying and pasting the data into word processing documents and/or transforming the web pages into PDFs before uploading them as primary documents. We hope that ATLAS.ti can soon make it possible to import this type of data in a less labor-intensive way.

Collaborating Through Memos and Merging

The use of ATLAS.ti was particularly important in that we were working as a team to collaborate on the analysis. Our process was as follows: Trena created the initial HU by assigning all primary documents and creating quotations to serve as units of analysis. She bundled and then sent the HU to Jessica after which we each worked with our own copy of the HU, adding memos to particular quotations that stood out to us, writing free-standing memos that captured questions and insights about the project as a whole, and creating hyperlinks to illustrate relationships between portions of the text. Jessica then sent her project file (HU) back to Trena, who merged them and sent a copy of the merged file to Jessica. As she did not modify the documents, Jessica did not need to send a copy bundle file, as at this juncture the primary documents were available to both Trena and Jessica on their personal computers. During meetings we began to create some initial coding categories that would help us narrow our focus and refine our analysis as described above. We found the memo feature particularly useful to support our collaborative work, with each memo date stamped to track our process.

“Unmotivated Looking” and Initial Analysis Through Memos

Richards and Richards (1998) reminded us that “before computers, many researchers did not code segments of text”, but rather “felt through, explored, read and re-read…compared and systematically built upon data records, keeping growing memo records about the accruing evidence and their exploration of its narrative…” (p. 214). This was a particular useful tool for us, particularly as we engaged in a form of discourse analysis that does not typically engage in “coding” data similarly to other forms of qualitative research. More specifically, traditions of discourse analysis the draw upon conversation analysis and discursive psychology often do not “code” data, as the concept is commonly understood. Rather, reading through the data repeatedly, noting its interesting aspects,and engaging in “unmotivated looking”(Sacks,1984) are common initial steps. As such, in our study, we engaged in an “unmotivated looking”, using the initial memos to chronicle our early impressions of the data.

Three broad questions guided our analysis: (1) What is the discourse doing?; (2) How is the discourse constructed to do this?; and (3) What resources are present and being used to perform this activity?(Potter, 2004; Wood & Kroger, 2000). We used the memo feature to engage in analyzing the data in relation to these questions, linking memos to the quotations as we interpreted the data (see Figure 3.)

Figure 3. Memo feature for initial analysis

Figure 3. Memo feature for initial analysis

Maintaining a Focus on Discursive Action Through Hyperlinking and Network Views

Next, we used the hyperlinking tool to link together blog comments and blog posts to reflect the use of certain phrases that seemed to be sequential, such as when students began their blog posts by saying “I don’t know much about dietary supplements” but then went on to share details of what they did actually know (Lester & Paulus, 2011). Hyperlinking of our initial interpretive memos also helped us explore how the students often used personal experience to justify supplement use and, in contrast, they used their beliefs to justify their skepticism around supplement use. We later imported these hyperlinks into a network view where we could visually explore the data further as shown in Figure 4.

Fig4

The hyperlinking tool can be especially useful for conversation analysts in order to identify and link adjacency pairs, as well as establish “next turn proof” (Schegloff, 2007) as evidence for choosing particular labels for conversational moves. Another option for engaging in this level of analysis is to use the precede and follows operator, as discourse analysts often work to make sense of how participants orient to conversational turns.

Narrowing the Analytic Focus Through Codes and Queries

After initially exploring our data through memos, hyperlinks and network views, we were able to create some broad categories by using the coding feature to help us narrow our data set for further exploration. For example, after coding for “personal experience” as a justification for student beliefs around supplements, we retrieved all 77 quotations for closer analysis as shown in Figure 5.

Figure 5. Use of the code and retrieve function.

Figure 5. Use of the code and retrieve function.

Coded datasets can be systematically searched through use of the query tool, which Lewins and Silver (2007) referred to as “interrogating the dataset.” In our study, we used the co- occurrence query to test our hunch about the connection between the use of personal experience to support pro-dietary supplement use views and the use of abstract beliefs to support anti-dietary supplement use. We ran a co-occurrence query for the codespersonal experience, pro- supplements, anti-supplements and beliefs, to see whether the codes for personal experience did indeed co-occur with codes for pro-supplement views more often than anti-supplement views as shown in Figure 6.

Co-occourrence query

Co-occourrence query

Another tool that is particularly useful for discourse analysts is the ability to perform simple text searches, as shown in Figure 7.

Figure 7. Text Search

Figure 7. Text Search

Conclusion

We found ATLAS.ti to be extremely useful for documenting our analytic decisions in a transparent and systematic way and encourage discourse analysis researchers to consider its use. We hope that other researchers who use CAQDAS tools begin to document their use in more systematic and transparent ways. Housley and Smith (2011), for example, described in detail their use of ATLAS.ti to engage in team coding, analyzing “research meeting talk” in an effort to open up “the black box of collaborative team-based coding practices within social scientific research” (p. 431). King (2010) noted the reluctance of conversation analysts to use data analysis software packages and in response described their use of NVivo to engage in membership category analysis, describing in detail the features they used and concluding that the ability to search and retrieve data, to store it in a single place and hyperlink it to text and audio files and to constantly update or return to data and audit its analysis (all of which NVivo, as well as ATLAS.ti and mother CAQDAS packages are capable of), means that they should be considered (p. 13).

We find, however, that CAQDAS tools remain limited in their ability to easily support the analysis of online interactional data. While NVivo’s NCapture tool allows easy capture of certain social media data like Facebook, Twitter,Youtube and LinkedIn, it does not support, nor does any tool, the easy capture of first generation interactional data such as our blog data. We call for features that would allow the analysis of online interactional data in a more seamless way.

References

Clark, R.E. (1994). ‘Media willnever influence learning.’ Educational Technology Research & Development, 42(2), 21–9.

Davidson, J. & Di Gregorio, S. (2011). Qualitative research and technology: In the midst of a revolution. In N.K. Denzin & Y.S. Lincoln (eds)The SAGE Handbook of Qualitative Research, 4th edition (pp. 627-643). Thousand Oaks, CA: SAGE Publications.

Di Gregorio, S. & Davidson, J. (2008).Qualitative research design for software users. Berkshire, England: Open University Press/McGraw Hill.

Fielding, N. (2008). The role of computer-assisted qualitative data analysis: Impact on emergent methods in qualitative research. In S. Hesse-Biber & P. Leavy (Edss) Handbook of Emergent Methods (pp. 655-673). New York, NY: The Guilford Press.

Friese, S. (2012). Qualitative data analysiswithATLAS.ti. London: Sage Publications.

Housley, W.& Smith, R.J. (2011). Telling the CAQDAS code: Membership categorization and the accomplishment of ‘coding rules’ in research team talk. Discourse Studies 13(4),417-434.

King, A. (2010). ‘Membership matters’: applying Membership Categorization Analysis (MCA) to qualitative data using Computer-Assisted Qualitative Data Analysis (CAQDAS) software. International Journal of Social Research Methodology 13(1), 1-16.

Konopasek, Z. (2008). Making thinking visible withATLAS.ti: Computer-assisted qualitative analysis as textual practices.FORUM: Qualitative Social Research 9(2), Art. 12

Kozma, R.B. (1994). The influence of media on learning: The debate continues. School Library Media Research, 22(4), 233–9.

Lester, J. & Paulus, T. (2011). Accountability and public displays of knowing in an undergraduate computer-mediated communication context.Discourse Studies 13(6), 671-686.

Lewins, A. & Silver, C. (2007). Using software in qualitative research: a step by step guide. London: Sage Publications.

MacMillan, K. (2005). More than justcoding?EvaluatingCAQDAS in a discourse analysis of new texts.FORUM: Qualitative Social Research 6(3), Art. 25

Paulus, T., Wood, M., Atkins, D. & Macklin, R. (September, 2013). Current reporting practices of ATLAS.ti users in published research studies. First Annual ATLAS.ti User Conference. Berlin, Germany.

Paulus, T., Lester, J. & Dempster, P. (Forthcoming). Digital tools for qualitative research. London: Sage Publications.

Paulus, T.M. & Spence, M. (2010). Using blogs to identify misconceptions in a large undergraduate lecture course.Tech Trends 54(5),62-68.

Paulus, T. & Lester, J. (2013).Making learning ordinary: Ways undergraduates display learning in a CMC task. Text & Talk 33(1), 53-70.

Potter, J. (2004). Discourse analysis. In Alan Bryman & Melissa A. Hardy (eds.), Handbook of data analysis (pp. 607-624). London: Sage.

Richards, T.J. & Richards, L. (1998). Using computers in qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.),Collecting and interpreting qualitative materials(pp. 211-244). Thousand Oaks, CA: Sage.

Sacks, H. (1984). Notes on methodology. In J. M. Atkinson & J. Heritage (Eds.), Structures of social action: Studies in Conversation Analysis(pp. 21–27). Cambridge: Cambridge University Press.

Schegloff, E. A. (2007). Sequence organization in interaction: A primer in conversation analysis. Cambridge, UK: Cambridge University Press, 2007.

Schoenfelder, W. (2011). CAQDAS and qualitative syllogism logic:NVivo8 and MAXQDA10 compared. FORUM: Qualitative Social Research 12(1), Art. 21.

Wood, L.A. & Kroger, R.O. (2000).Doing discourse analysis: Methods for studying action in talk and text. Thousand Oaks, CA: Sage Publications.

About the Authors

TrenaTrena Paulus, PhD. Associate Professor Department of Educational Psychology & Counseling, University of Tennessee.

Corresponding author, 515 BEC, Department of Educational Psychology & Counseling, 1122 Volunteer Boulevard, University of Tennessee, Knoxville, TN 37996

Jesica Lester

Jessica Lester, PhD. Assistant Professor Department of Counseling and Educational Psychology, Indiana University.

http://indiana.academia.edu/JessicaNinaLester