Navigating the Intersection of Qualitative Analysis and Technology: Strategies Driving Tactics in the Age of AI

Introduction

The incorporation of generative-ai capabilities into ATLAS.ti provides many new opportunities for qualitative researchers, and I'm excited to see how these tools will impact the field. But its not only the tools and how they're being used that interests me.

As a teacher of qualitative methods I've also been thinking about how the availability of these tools affect the way we go about incorporating them into methods teaching and learning. And that's brought me to reflect on my own pedagogic practice once again.

A bit of background

I've been interested in qualitative software like ATLAS.ti since I did my Masters at the University of Surrey in 1997, and a year later began teaching these tools when I joined the CAQDAS Networking Project (CNP), which I now direct.

The CNP was founded in 1994 to raise awareness about, and build capacity in the use of software for qualitative analysis, and as one of the pioneer programs, I've been using and experimenting with ATLAS.ti for more than 25 years now.

During that time one thing I've been most interested in, is how students and new users engage with tools when they first learn about them, and adopt them into their analytic practice.

The Five-Level QDA method

Many years of observing learners, and several research projects into CAQDAS adoption led me to work with Nick Woolf on developing our pedagogy, the Five-Level QDA method, which resulted in our ATLAS.ti textbook.

The development of the Five-Level QDA method, and description of its principles can be found in our Sage Foundations entry as well as the textbook, and its central message is that analytic strategies - what we plan to do - drive software tactics - how we do the analysis.

As the years have gone by, the number and diversity of tools provided within ATLAS.ti have expanded and become increasingly powerful. That's great for methodological advancement and creativity in analytic practice. But just because something is possible, doesn't mean it's the most appropriate thing to do. Let me give you an analogy.

An analogy to illustrate the importance of strategies driving tactics

Imagine I needed to get from London to Paris, and I needed to get there as quickly as possible (in other words, my strategy - my plan - was to get from London to Paris as quickly as possible). In order to fulfil my plan, I would consider the available tactics - in this situation, the different modes of transport available to me. I might choose to get on an aeroplane as the quickest way to get from London to Paris. But what if my strategy - my plan - wasn't to get there as quickly as possible, but with the minimum carbon footprint possible. An aeroplane would clearly not be the most appropriate tactic for this strategy. Instead I may get on a train, or even cycle to the coast, swim across the channel and then walk to Paris.

The decisions we make about how to harness the powerful tools ATLAS.ti provides us, are similar.

There are usually several ways we could accomplish any given analytic task within the software, and so we have to make choices. Like our choice of transport, our choice of software tools has implications, which is why it's so important to ensure the needs of our analysis drive our use of tools, on in the language of Five-Level QDA, that our strategies drive our tactics.

Doesn't it work both ways?

When Nick and I first presented our pedagogy publicly, at the ICQI in 2016, a few people challenged the direction of the relationship between strategies and tactics we were emphasizing; that strategies drive tactics.

They suggested that the availability of tools affects what's possible analytically, and thus can shape research design. For example, the ability to effortlessly, speedily and reliably count the occurrence of keywords across vast quantities of textual materials opens up content analysis approaches in ways not practically possible before those tools were available.

The emphasis of the directionality

It's true that with the development of technology come new possibilities; that's not something we reject. Indeed, it's characteristic of technological progress, and one of the things that keeps me interested in the field of computer-assisted qualitative analysis.

Our point is that whilst tactics can inform strategies, strategies should drive tactics.

In other words, researchers absolutely should consider new tools, reflect and experiment with them to determine their potential, to see if they can usefully be incorporated into their practice.

It's this open-mindedness coupled with practical and methodological reflection that we mean when we say tactics (tools) can inform strategies (methods).

But if that's not done with careful consideration, if new tools are used because they appear to be a 'silver bullet', a magic weapon to solve the challenges of doing qualitative analysis, you can end up in a big mess very quickly, because you're letting tactics (tools) drive strategies (methods).

I've seen the results of this in my workshops on many occasions, particularly when what appear to be 'cool' CAQDAS features are misunderstood and used inappropriately. An extreme example I've written about previously came from a student who expected they didn't need to read any articles to do a literature review using qualitative software. Another example related to current technological developments would be if audio/video recordings of interviews, focus-group discussions or other encounters were transcribed using automated ai-driven tools (which have significantly increased in accuracy in recent years and months) without checking the resulting transcript for errors.

New tools, new possibilities, new ethics

Observing the ways generative-ai may change how we can do qualitative analysis is truly fascinating. It will be a while before things settle down and we get to see which of the new tools the qualitative community of practice will adopt into their practice.

At the moment there is much experimentation, and debate. Like when any new technology emerges, there are the early adopters and advocates, and the sceptics.

There are rightly many ethical concerns about the use of these tools. And there will be many continued discussions about them, and guidelines developed around these technologies; not just in terms of how we use them, but also how we report on their use, how we inform research participants about them, how ethics boards and publishers react to research that uses them.

Reaching a balance

But just like it would be inappropriate to use ai transcription tools to automate what is often seen as a tedious, time-consuming process, without checking the resulting transcript for errors, it would be a mistake to have generative-ai tools do their thing with qualitative materials (coding, summarising and so on) and accept the results without checking them or considering whether and how they actually add to our interpretations.

That's why the difference between informing and driving is so important.

I've many times heard researchers say "I've done my coding, what next" - this sounds alarm bells for me in terms of research design and analytic planning. It will be all the more alarming if researchers use ai coding tools - like ATLAS.ti's AI coding or Intentional AI Coding - to quickly "get coding done" and blindly accept it as accurate and meaningful without reviewing, refining and augmenting in relation to the analysis objectives.

Awareness is key

Although the ethical and methodological concerns that are being raised are valid, there are also misunderstandings about how the technology actually works. This is where the teachers of qualitative methods play a fundamental role.

Students will inevitably experiment and utilize these tools, and they will improve over time as the technology continues to develop.

Just like I would want and expect my children to learn about how to use social media platforms appropriately, I would want and expect students of qualitative methods to learn about analysis tools so they can use them appropriately.

This has always been one of my key messages - long before the advent of generative-ai. But it is even more important now. And that can only happen if researchers are aware of the options and the possibilities they afford.

So, experiment with ATLAS.ti's AI coding, Intentional AI coding, Code-suggestions, Conversational AI, and AI Summaries. Attend webinars and training events that discuss the use of these and related tools in qualitative projects. Discuss the tools with your colleagues, students and advisors.

The strategies/tactics distinction has never been more important

If you're a teacher of qualitative methods, don't stick your head in the sand about these developments and thereby neglect to equip your students with the appropriate mindsets about these tools and skills to use them appropriately. They will likely use them whether you teach them about how to do so or not.

We have a responsibility to the current and future generations of qualitative researchers, and being explicit about the difference between analytic strategies (what we plan to do) and software tactics (how we do it), and the relationship between the two, has never been more important.

Christina Silver PhD is Associate Professor (Teaching) in the Department of Sociology at the University of Surrey, where she is Director of the CAQDAS Networking Project. She is also co-founder and director of Qualitative Data Analysis Services, Senior Fellow of the Higher Education Academy (SFHEA) and Fellow of the Academy of Social Sciences (FCaSS). She is author of many publications about the use of digital tools for qualitative analysis, including Using Software in Qualitative Research: A Step by Step Guide (with Ann Lewins) and Qualitative Analysis with ATLAS.ti: The Five-Level QDA Method (with Nick Woolf).