“Why Reliable Data Matter to Qualitative Scholars” by Professor Klaus Krippendorff, keynote speech at the 3rd ATLAS.ti User Conference
ATLAS.ti 8 now offers an integrated tool to calculate the agreement among multiple coders directly within the software. Intercoder agreement measures are important for discerning the reliability of a qualitative data analysis, and Dr. Klaus Krippendorff has been seminal in helping the ATLAS.ti developers design and implement the intercoder agreement tool. ATLAS.ti 8 users can now calculate their percentage agreement, Holsti index, and Krippendorff’s alpha directly within ATLAS.ti. All the mathematics from Dr. Krippendorff’s work for calculating agreement have been integrated into the tool by ATLAS.ti’s developers, so you can now quickly and easily see how reliably the different coders of your project have analyzed the data. Teamwork in ATLAS.ti has never been easier: distribute the project among your team members, carry out your analyses, merge your projects all together, and analyze how each person has coded the body of data with the intercoder agreement tool.
Dr. Krippendorff shared his insights about reliability in qualitative data in the keynote speech of the 3rd ATLAS.ti User Conference. Dr. Krippendorff (PhD in Communication, University of Illinois, Urbana, 1967) is a Professor of Communication and Gregory Bateson Term Professor of Cybernetics, Language, and Culture at the University of Pennsylvania´s Annenberg School for Communication. Besides numerous publications in journals of communication, sociological methodology, cybernetics, and system theory, he authored Information Theory, Structural Models for Qualitative Data, a Dictionary of Cybernetics, edited Communication and Control in Society, and coedited The Analysis of Communication Content and Developments and Scientific Theories and Computer Techniques.
In his keynote speech, Dr. Krippendorff talked about different types of reliability, the conditions for generating reliable data, the pre-requisites for coding, and how reliability can be inferred from measurement. He has worked extensively with questions of reliability in qualitative research, and today qualitative researchers all around the world calculate Krippendorff’s alpha in their own research to discern the reliability of their coding.
The criteria for good qualitative scholarship has been debated for years, but the concepts of reliability and validity have been often rejected as “positivist” criteria. Dr. Krippendorff effectively pointed out that reliability is simply the ability to rely on the use of something given, which is something that all researchers are interested in – to demonstrate that their study is reliably presenting the story of the analyzed data. In other words, reliability provides assurances that the given data can be trusted in subsequent analyses of the phenomena they claim to represent. When it comes to reliably analyzing qualitative data, Dr. Krippendorff shared some advice:
- The coders of the data are instrumental in demonstrating reliability, but they need to be interchangeable.
- The coding instructions that account for the data need to be reproducible elsewhere.
- When the phenomena in question are unstructured, symbolically complex, or transient in nature (speech, social happenings, etc.), the coding instructions are the only means to reconstruct what gave rise to the data and what they mean.
We thank you, Dr. Krippendorff, for your enlightening keynote speech and for your great contribution to the new intercoder agreement tool of ATLAS.ti 8!
If you want to learn more about Dr. Krippendorff’s work and reliability in qualitative research, you can read about his latest publications here: https://www.asc.upenn.edu/people/faculty/klaus-krippendorff-phd