What is mixed methods research?

When starting the research process, researchers sometimes think they have to decide whether qualitative research or quantitative research is more appropriate for their research design. However, the more important question is whether the methods they employ in data collection and analysis sufficiently capture the phenomenon they want to study. In some cases, answering this question requires using multiple methods of research.

Mixed methods research is a research paradigm that involves collecting qualitative data and quantitative data on the same object of inquiry. Researchers who employ mixed methods research synthesize qualitative findings with quantitative findings to achieve a better understanding.

Let's look at the established research paradigms, then mixed methods research, why it's useful, and which research methods complement each other. Then we'll examine how ATLAS.ti can help you execute a mixed methods design.

What is a mixed methods design?

Mixed methods research is followed out of the need to understand concepts or phenomena at a deep level. A standalone quantitative study or qualitative study can provide great insight. Still, one method alone may not be able to capture all knowledge necessary to fully understand a topic or issue.

Those who conduct mixed methods research acknowledge the importance of pursuing both qualitative and quantitative research to achieve more complete results. However, this is not simply an issue of collecting more data just for its own sake. Mixed methods design is purposeful in carefully crafting research questions and employing appropriate research methods to essentially fill in the gaps of knowledge surrounding a particular research inquiry.

To determine which methods and data can address particular research needs, let's look at the capabilities of and differences between qualitative and quantitative data collection.

Qualitative and quantitative data

Researchers are often quick to make conclusions about whether qualitative research is better than quantitative research or vice versa. The reality is that quantitative and qualitative data can both look at the world in different ways that are useful at various points of a research inquiry. Qualitative and quantitative research are established research paradigms precisely because they provide relevant insights with the appropriate research design, data collection, and analysis.

One of the main goals of qualitative research is to generate a description of a social phenomenon. When something is difficult to quantify, it needs to be broken down into more constituent elements that are, by themselves, easier to perceive. In educational evaluation, for example, it is difficult to evaluate good academic writing with just a single score alone. Writing teachers employ a rubric to measure writing by a number of aspects which may include argumentation, organization, and cohesion.

Qualitative methods of research tend to collect data for an analysis that is capable of generating frameworks of constituent elements. Such a framework can then be used in subsequent research, evaluation, or decision-making processes. Researchers can collect qualitative data from observations, interviews, or records searches. Qualitative data analysis then aims to identify patterns and themes frequently appearing in the collected data.

The efficacy of experimental drugs in clinical trials, for example, is seldom easy to measure through quantitative methods alone. Qualitative research methods are often employed to determine a research participant's well-being, emotional state of mind, and other factors to help researchers decide the overall success of their clinical trials.

Quantitative research

If qualitative methods describe a concept or phenomenon, quantitative methods employ the resulting framework to measure that concept or phenomenon. Quantitative research methodology takes the theories generated from qualitative findings to collect quantitative data that can be used to measure a concept or phenomenon at scale.

Ultimately, numbers and values inform decision-making processes in many contexts. Quantitative results are useful in research areas where precision is valued or required. Still, they are also used in social and behavioral research to numerically describe phenomena that may not appear to be naturally quantifiable.

Mixing methods

Quantitative and qualitative strands of research are often pitted against each other for various reasons. Researchers might shun qualitative data collection as it is often time-consuming. In contrast, quantitative data collection is often critiqued for its reductive power (i.e., reducing ambiguous concepts into simplistic numerical values). Many scholarly disciplines, as a result, tend to prefer one research paradigm over the other (e.g., chemistry tends toward quantitative data collection, while anthropology tends toward qualitative data collection).

In the long run of any sufficiently complex research inquiry, however, it is seldom necessary to remain confined to one research approach. The main objective of scientific research is to organize knowledge through theories about the world around us. As a result, researchers employ mixed methods to combine theory generation in qualitative research with confirmatory testing in quantitative research to ultimately produce a robust theory and new knowledge.

However, research studies that combine qualitative and quantitative methods for the sake of having multiple methods of data collection and analysis are not as persuasive or impactful as true mixed methods studies where research methods are purposefully chosen to achieve a better understanding.

An example of mixed methods research

The objective of mixed methods research designs is to employ different inquiry components under one larger study. However, it might be easier to think of mixed methods research designs as having at least one qualitative study and one quantitative study, each with related but ultimately separate research questions. Examining a mixed methods research design in this way might make it easier to understand the need for pursuing multiple methods in certain cases.

• Consider the following example:

Remote work performance and job satisfaction

- RQ1: How have work outputs at XYZ Company changed since the shift to fully remote work?

- RQ2: What perceptions do remote workers at XYZ Company have about the shift to fully remote work?

In general terms, the goal of the study is to examine the efficacy of remote work in comparison to traditional, in-office work at one company. Actually determining this efficacy requires looking at the phenomenon of remote work through different methods.

As a result, one possible mixed methods study might look at the performance metrics of the company. Research question 1 (RQ1) is posed to conduct a quantitative research study that collects data on possibly quantifiable concepts related to work (e.g., amount of sales generated, number of new clients acquired). In this case, the researchers collect quantitative data to compare post-remote work performance to pre-remote work performance and determine if productivity has changed over time.

While this is a useful angle to examine remote work, it does not tell the whole story. After all, if people at Company XYZ are more or less productive than before, what are the reasons that explain this change? To address research question 2 (RQ2), researchers collect qualitative data on the level of satisfaction employees have with their jobs. Qualitative data from interviews with employees can be used to determine which aspects of their job they find satisfying or not.

With all the data collected, mixed methods researchers can combine the initial quantitative results and the initial qualitative results to form a deeper understanding of their topic of inquiry. In this case, if the quantitative data shows that worker productivity has suffered since the switch to remote work, the qualitative data might illuminate the aspects of remote work that employees don't like.

Other mixed methods research examples

While there are many different forms of mixed-methods research, the research approach is generally the same across mixed-methods research designs. A mixed methods research design is likely to require researchers to collect quantitative and qualitative data relevant to an overarching topic that necessitates examination from different methods. A couple of examples are:

Literacy development among children

RQ1: What is the rate of literacy development among children at ABC School based on scores from a standardized reading test?

RQ2: What are the instructional practices common in classrooms with high-performing students on standardized reading tests?

Market research for a new computer model

RQ1: How much time does it take to complete a series of tasks on an experimental computer model compared to a comparable computer model?

RQ2: What factors do potential customers take into consideration when buying a new computer?

Notice that qualitative and quantitative data pursue related but ultimately different aspects of the phenomena under study. As a result, the discrete inquiries in a mixed methods study will most likely employ different methods to collect data.

Triangulation in mixed methods research

Researchers do not employ mixed methods research just for the sake of having different methods in one research inquiry. The objective behind mixing methods is to generate new knowledge and strengthen understanding of that knowledge by examining it from different angles. This is a concept in research called triangulation, which refers to affirming a given location based on measures taken from different points. The equivalent notion in research is that viewing the same object of inquiry from multiple angles will provide a more reliable understanding of that object.

To further understand the utility of a mixed methods approach, imagine you and your friends are looking at a merry-go-round. You can only see one part of it at any one time, while other parts are obscured from your view. On the other hand, if your friends are positioned to see the merry-go-round from different angles, your combined observations can capture a more complete picture of the object you are studying.

Mixed methods research relies on multiple research methods, data sets, or theoretical approaches to assemble a more comprehensive picture of a concept or phenomenon. Especially in qualitative research or social science research, any set of findings can be considered more credible if they are supported with evidentiary data that comes from different perspectives.

Method triangulation

Method triangulation involves combining qualitative and quantitative methods together to study different but related aspects. In this respect, quantitative and qualitative research study the same phenomenon to lend support to each method's findings. Note that the goal of triangulated mixed methods research is not to simply use multiple methods to arrive at the same answer but to generate a better understanding of a phenomenon that one method alone cannot sufficiently capture.

In this case, method triangulation is a useful concept for a mixed methods researcher because it requires them to acknowledge the strengths and weaknesses of each particular research method. At scale, quantitative methods cannot capture concepts that are unquantifiable (e.g., beauty, convenience). In contrast, qualitative methods often do not conduct data collection at scales necessary to make generalizations about phenomena. Integrating quantitative and qualitative research components under the same mixed methods design ensures a comprehensive examination of a phenomenon that one method alone cannot accomplish.

Ethnography provides ample opportunities to pursue method triangulation. Data collection in ethnographic research often involves collecting qualitative data through observations and interviews. In contrast, data analysis can assess quantitative data by identifying patterns in behavior and perspectives and determining their frequencies.

Another example is a mixed methods study that examines patient outcomes at a hospital. Initial qualitative results might come from field notes from observations of doctors and nurses and interview data with patients. The quantitative findings might come from conducting a statistical analysis of the money and resources used for each patient observed or interviewed to determine whether the expenditure is commensurate with the patient outcomes achieved.

A standalone quantitative study might look only at the financial aspects of health care, while a qualitative study might do better at examining the social and emotional aspects. Conducting both of these studies in tandem can help researchers determine actionable insights for streamlining health care services while maintaining satisfactory standards of care.

Data triangulation

Mixed methods research usually depends on method triangulation, but it's important to identify other forms of triangulation that can strengthen the findings in any research. A study that relies on data triangulation looks at different sets of data. For example, an educational researcher might examine student outcomes at different schools or at the same school but at different times. Data triangulation is useful in affirming that the findings in one context are applicable across other contexts.

Theory triangulation

Another kind of triangulation less commonly associated with mixed methods research deals with analyzing data using different theories. A sequential research design, for example, may use the initial quantitative results from a survey study to generate a conceptual framework for the analysis of a subsequent qualitative study. At the same time, existing theories may also be employed in that analysis to compare and contrasts the kinds of insights and outcomes that each may produce.

Theory generation in mixed methods research

Many forms of research seek to generate or develop a theoretical framework to understand the object of inquiry. There are two common forms of theory generation, and both can manifest in the research questions that are posed in any study.

Research questions can either be exploratory, which try to define or gain a greater understanding of a phenomenon, or confirmatory, which try to test a theory or hypothesis regarding that phenomenon. With some exceptions, exploratory research questions call for collecting qualitative data, while confirmatory research questions require quantitative data.

In that respect, common mixed methods designs combine qualitative and quantitative components to generate a theory and either strengthen or challenge that theory, respectively. To understand what that theory generation looks like when employing mixed methods, we need to examine some of the different kinds of mixed methods research designs.

Types of mixed methods research designs

Data collection and analysis in mixed methods research depends on the research design you adopt. Ultimately, it might be easy to think about the different research designs in terms of the timing of the discrete inquiries within a mixed methods inquiry.

Concurrent triangulation design

A study that collects quantitative and qualitative data simultaneously is a common form of mixed methods design to achieve triangulation. The goal of a concurrent triangulation design is to observe the object of inquiry from multiple methods.

For example, imagine an educational researcher who wants to examine the efficacy of an after-school reading program. The researcher can then pursue two concurrent studies, one that qualitatively observes the reading program in action between educators and students and another that quantitatively tests students' reading comprehension. Over time, the researcher can draw correlations between improvements in test scores and any observations of the students in the program.

Exploratory sequential design

Another way to look at mixed methods research is with the idea that data collection and analysis are cyclical and evolve as new knowledge is generated. Researchers might undertake an exploratory sequential design if they don't yet know the aspects of a concept or phenomenon they want to test. In short, they need to conduct a qualitative study first in order to generate a conceptual framework to apply in a subsequent quantitative study.

Exploratory sequential design is useful in market research, for example, to identify the potential needs and preferences of prospective customers. Focus group research with a group of target customers can inquire about what they are looking for when choosing from a line of products. The researcher can take the initial qualitative findings to inform the design of a subsequent survey study that can confirm the extent to which the preferences of the focus group are reflected in the larger market.

Researchers can also conduct a quantitative study to preface observations in a qualitative study. Imagine that an educational researcher is adopting mixed methods approaches when examining learning outcomes among schools within a given geographical area. They might start by examining test scores published by these schools, using the initial quantitative results to determine where students are struggling and might need intervention. The resulting qualitative study might conduct observations in struggling schools to determine potential shortcomings in teaching and learning.

Concurrent nested design

This research design involves conducting multiple inquiries at the same time for the purpose of using one inquiry to strengthen the other. In a mixed methods approach, concurrent nested design places one research paradigm within another (e.g., a quantitative study within a qualitative study).

Sequential transformative design

This is a mixed methods research design with a critical or social justice orientation, meaning that the research is ultimately conducted to challenge the understanding of existing theory or produce meaningful social change, respectively. In either case, a sequential mixed methods research design can have a transformative effect by employing one study to create the rationale for a second critical or social justice research inquiry.

Using ATLAS.ti for mixed methods research

As you employ multiple research methods for a single mixed methods research design, you might find that your data collection will involve large sets of data, presenting a challenge in managing all that information in an orderly manner. Whether you are conducting research through qualitative data collection, quantitative data collection, or both, ATLAS.ti can help you organize and analyze your data. A robust mixed methods approach requires systematic organization of your data collection to ensure efficient and insightful analysis.

Document groups

Data in ATLAS.ti is stored in documents, which can be classified by the data type they contain. ATLAS.ti allows you to analyze text, images, video, audio, and more, and each document's data type is marked in the Document Manager for easy organization.

However, you may also need to divide your documents by type of study or method employed. In that case, you can use Document Groups in ATLAS.ti to label your documents so your project has categories for quantitative and qualitative data, interviews and focus groups, observations and test scores. Documents can belong to multiple document groups, allowing for easy organization of documents into multiple categories.

Query Tool

Once you have fully coded your data, it might be a challenge to narrow down your analysis to the relevant data you're looking for. If you have to sift through large numbers of documents, the Query Tool can help you look for the most relevant quotations based on the codes you have applied to your data.

Global filters

Studies that employ mixed methods research can accumulate such vast amounts of qualitative and quantitative data that it might become cumbersome for the human eye to keep track of it all manually. Even the most organized project in ATLAS.ti can have thousands of documents or hundreds of codes, making it a challenge to find the right data.

In ATLAS.ti, you can set a global filter using any of the elements of your project. For example, if you have a document group labeled "interviews," you can set a global filter for that document group, which will lead ATLAS.ti to only show the documents in that group.

Working with both qualitative and quantitative software

ATLAS.ti has a number of tools that provide visualizations to help illustrate quantitative findings. However, you may find that other software, such as Microsoft Excel or SPSS, can help you further analyze and visualize the quantitative research components in your study. As a result, ATLAS.ti allows you to export your analysis into a Microsoft Excel spreadsheet. The Code Co-Occurrence Analysis and Code-Document Analysis tools, for example, can export their resulting tables into Microsoft Excel, which includes tools for deeper statistical analysis or for creating other kinds of data visualizations.

ATLAS.ti projects can also be exported as syntax files that can be imported into other statistical analysis software such as SPSS and R. These files convert qualitative data into quantitative data for further statistical analyses, regressions, and quantitative visualizations. Researchers can fully realize the convergence between qualitative and quantitative research when using multiple software platforms to conduct their analysis.