Data Collection in Mixed Methods Research

Combining qualitative and quantitative approaches, mixed methods research relies on flexible data collection strategies to explore complex research questions. Whether data is gathered simultaneously through concurrent designs or in stages with a sequential approach, the goal is to create a comprehensive understanding of the topic. Each method presents unique challenges, requiring careful alignment with the overall research design.

Introduction

In mixed methods research, data collection is designed to address specific research questions or hypotheses, aligning closely with the chosen mixed methods design. Researchers can collect data concurrently—gathering quantitative and qualitative data at the same time—or sequentially, where one data type is collected and analyzed before the other. Both approaches have distinct challenges that require careful planning.

Data collection in mixed methods research typically unfolds in several phases: sampling, obtaining permissions, determining data types, choosing recording methods, and administering data collection procedures. While these phases apply to both qualitative and quantitative research, they differ slightly in execution due to the unique characteristics of each approach (Creswell & Plano Clark, 2017).

Sampling

Qualitative sampling

In qualitative data collection, researchers deliberately select participants and locations that provide rich and relevant data. This approach, known as purposeful sampling, ensures that participants have direct experience with the primary phenomenon or concept being studied. A variety of purposeful sampling strategies exist, each serving unique research purposes. For example, maximal variation sampling involves selecting participants with diverse perspectives on the topic, with variation often based on characteristics like race, gender, or education level. The aim is to reflect a wide range of viewpoints, enriching the study's insights. Alternatively, extreme case sampling focuses on unusual or exceptional cases, while homogeneous sampling targets participants within a subgroup that shares specific characteristics.

In terms of sample size, qualitative research prioritizes depth over breadth. Instead of seeking large samples, researchers focus on a smaller group to allow for detailed exploration of each participant or site. Larger samples often reduce the depth of insights obtained from individuals. Sample sizes vary depending on the research design, with narrative studies typically involving one or two participants and grounded theory projects including up to 50 or 60. Qualitative studies can also be based on samples of 10 to 20 participants, chosen to align with the research questions and methodology, such as phenomenology ethnography or case studies.

Quantitative sampling

In quantitative data collection, sampling aims to select participants that accurately represent the broader population to generalize findings. Initially, researchers define the population of interest and carefully choose a sample from it. Random sampling is often employed to ensure each individual in the population has an equal probability of being selected, reducing selection bias. This method uses systematic procedures, such as random number tables.

To include specific characteristics in the sample, researchers may use stratified sampling. For example, if females outnumber males in the population, stratified sampling ensures proportional representation by separating the population into subgroups (strata) and randomly sampling from each. This process ensures that the sample reflects the population's diversity on key characteristics.

The sample size in quantitative research must be adequate for statistical analysis that makes inferences about the population. Researchers use sample size formulas or power analysis to calculate the necessary size for their study's design. In experimental designs, power analysis is critical, while sampling error formulas guide survey research. These approaches minimize sampling error—the difference between the sample estimate and the population parameter—ensuring findings are robust and generalizable.

Permissions

To conduct research, researchers must obtain permission to collect data from individuals and sites. This process typically involves multiple levels of approval. Firstly, permission must be secured from individuals managing or overseeing the research sites, such as administrators or directors. Secondly, consent is required from participants who will provide data, and in some cases, their legal representatives, such as parents of minors. For academic research, campus-based institutional review boards (IRBs) play a crucial role by approving the study to ensure that the rights and safety of participants are protected. Researchers have to adhere to the conditions set by IRBs, including obtaining signed consent forms from participants before data collection begins.

In qualitative research, detailed documentation of procedures is critical, as it often involves personal interactions in sensitive environments, such as participants' homes or workplaces. This level of access necessitates collaboration with gatekeepers—individuals within organizations who can facilitate entry and support the research process. While this is particularly emphasized in qualitative studies, quantitative research also requires permissions, especially in hard-to-access settings, like government agencies.

Failure to secure proper permissions, especially from IRBs, can lead to significant consequences, such as disqualification of findings from publication and risks to the institution’s funding. Additionally, the type of data collection—qualitative or quantitative—affects the risk to participants. Qualitative research, which involves deeper engagement, might expose participants to privacy risks, such as disclosure of sensitive behaviors. Quantitative research, on the other hand, generally involves less intrusive methods, such as surveys or observational checklists, which tend to carry lower risks of harm.

Information to be collected

Researchers must evaluate the types of data available and weigh their options to determine which sources are most suitable for addressing their research questions or hypotheses. In qualitative research, data types include open-ended interviews, observations, documents (private and public), and audiovisual materials such as videos, photographs, and sounds. New data forms, like text messaging, emails, social media posts, continue to expand these possibilities.

In contrast, researchers can collect quantitative data using instruments like aptitude tests to measure performance or attitude scales to assess opinions. They also use structured interviews and observations, where response categories are predefined. Quantitative data is often numerical, drawn from sources such as census records, attendance logs, or progress reports. Similar to qualitative research, researchers must carefully consider which options will best answer their specific research questions or hypotheses.

Data recording

The process of data collection involves systematically gathering and recording information in a way that allows for preservation and analysis by an individual researcher or a research team. For qualitative data, recording tools need to be carefully prepared. For example, when collecting interview data, an interview protocol should be developed that includes a list of questions and space for recording key details such as the time, date, and location of the interview. Researchers often audio record interviews and later transcribe them. An interview protocol helps maintain organization, and using more than one audio recording device ensures having a backup copy in case of any technical difficulties. Similarly, an observational protocol allows researchers to document observed events and processes, while also making reflective notes on emerging themes or concerns. Additional forms can be created for reviewing documents and recording visual data, such as photographs.

In quantitative research, researchers either select or create instruments, such as attitudinal scales with established reliability and validity. Alternatively, they may use validated checklists based on prior use. When working with numeric data from documents, researchers often develop forms to summarize the information. Selecting instruments or checklists involves ensuring they are reliable, valid, and aligned with the research questions and hypotheses, as well as using appropriate scales to report the collected data.

Administering the data collection

Administering data collection involves executing the processes necessary for gathering information. In qualitative research, significant focus is placed on anticipating and addressing potential challenges that could compromise data quality. Examples include the time required to recruit participants, the researcher’s observational role, ensuring the functionality of recording devices, locating relevant documents, and appropriately positioning videotaping equipment. Additionally, researchers must enter research sites respectfully and avoid disrupting ongoing activities. Ethical considerations, such as offering reciprocity to participants for their contributions, safeguarding sensitive data, and clearly explaining the research purpose are critical.

In quantitative research, these ethical principles remain essential, but consistency in data collection processes is also vital to avoid introducing bias. Standardized procedures should be established for using instruments, checklists, and public records to maintain uniformity. When multiple investigators are involved, thorough training is necessary to ensure the data collection process is carried out consistently and accurately each time.

Data collection in different designs

It is important to understand the general processes for collecting data in qualitative and quantitative research since mixed methods research integrates both types. Before addressing specific designs, consider these general guidelines for mixed methods data collection:

• The procedures for collecting data vary depending on the mixed methods design. Data collection can occur concurrently or sequentially. Concurrent designs collect qualitative and quantitative data simultaneously, and the degree of independence between the two forms can vary (e.g., triangulation or embedded designs). Sequential designs collect one type of data first, then use the findings to inform the subsequent phase, such as in explanatory or exploratory designs.
• In concurrent data collection, qualitative and quantitative data tend to be independent. Conversely, sequential designs emphasize a relationship between the two forms, connecting the results of one phase to the next​.
• Regardless of the design, both data collection approaches should employ rigorous qualitative and quantitative procedures. For example, qualitative methods might involve in-depth interviews with detailed coding frameworks, while quantitative methods could rely on validated instruments for statistical analysis.
• All research involving human subjects requires Institutional Review Board (IRB) approval. For concurrent designs, both data collection methods should be outlined at the proposal stage. For sequential designs, it is usually necessary to describe the initial phase and indicate that follow-up procedures might evolve based on initial findings. When follow-up procedures are established, an IRB addendum may be necessary.

Concurrent data collection guidelines

Concurrent forms of data collection are used in convergent parallel designs and its variants, as well as embedded designs using a concurrent data collection phase.

• Data is usually collected during the same timeframe, and it can be collected separately or in the same instrument, such as a survey with a mix of open-ended and closed-ended questions. They may be collected from the same level in an organization or at different levels. In a multilevel variant of the design, quantitative and qualitative data collection occur at different levels (or with different units of analysis).
• Quantitative and qualitative data collection in a study can be assigned equal or unequal importance. For instance, an embedded design incorporating qualitative data into an intervention trial may give less priority for qualitative data collection.
• In concurrent data collection, the analysis may serve various purposes: to align and integrate findings (as seen in the convergent parallel design); to use one form of data (e.g., quantitative) to validate the other (e.g., qualitative); to transform one type of data for comparison with the other (as a form of triangulation); or to generate complementary data to answer distinct research questions (as in the embedded experimental or correlational design).

Concurrent data collection challenges

With these general characteristics in mind, it's important to be mindful of specific issues that can arise in using a concurrent form of data collection.

Participant selection: A significant consideration is whether to select the same or different individuals for the quantitative and qualitative samples. Although no definitive consensus exists, many mixed methods researchers prefer selecting the same individuals for both data types, as this facilitates easier comparison and integration. Selecting different participants can introduce variability in personal characteristics, potentially confounding the comparison. For example, in triangulation and embedded designs, the same individuals or sites are often used. One study could collect quantitative data (e.g., using structured observation checklists) and qualitative data (e.g., key informant interviews) from the same cases to streamline the data integration process (Creswell, 2003).

Sample size: Another issue is whether to use the same number of participants for the quantitative and qualitative arms of the study. Typically, quantitative samples (preferably randomly selected) are larger than qualitative samples (usually purposefully selected), which creates challenges for comparing or integrating the two datasets. In embedded designs, one type of data is generally supplemental to the other type of data, and each often answers different questions (e.g., examining patient experiences during treatment versus assessing the treatment's impact on outcomes). Researchers can also consider addressing discrepant sample size issues by increasing the number of qualitative participants while sacrificing some detail, weighting qualitative cases to align them with quantitative data, or acknowledging the limitations of comparing datasets with unequal sample sizes (Creswell, 2003; Padgett, 2004).

Handling contradictory results: When mixing data types, contradictions may arise between quantitative and qualitative findings. According to Creswell (2003), one approach to resolving such contradictions is to collect additional data, though this may not always be feasible or timely. Another option is to present the findings in parallel, acknowledging the need for further research (Padgett, 2004). Alternatively, such contradictions can serve as a starting point for new lines of inquiry.

Sequential data collection guidelines

Sequential data collection follows a different process than concurrent data collection. The following guidelines outline the sequential approach:

• Sequential data collection occurs in stages, where one data collection phase is completed before another begins. This approach is used in explanatory and exploratory designs, as well as in the sequential variations of the embedded design.
• In sequential designs, the quantitative and qualitative data are connected and interdependent. One phase informs and builds on the other.
• The emphasis on either the first or second phase of data collection can vary. The primary focus may be either qualitative or quantitative, depending on the research problem and the investigator's goals.
• Sequential data collection typically follows three stages. In the first stage, data collection and analysis focus on either qualitative methods (in exploratory designs) or quantitative methods (in explanatory designs). In embedded designs, either approach may be used in the first stage. During stage 2, researchers decide how the results of stage 1 will guide the next phase of data collection and analysis. The purpose of stage 2 is to ensure that stage 3, which involves the second round of data collection and analysis, builds directly on the results of stage 1. This three-stage process integrates the findings across phases, using stage 2 to link the initial results to the final data collection.

Sequential data collection design challenges

Sequential data collection also bears unique challenges that researchers can consider to bolster the quality of their data collection:

Sampling the same or different individuals: The decision to sample the same individuals in stage 1 and stage 3 depends on the research design. In explanatory designs or embedded designs with a follow-up phase, the same participants are typically included in both stages. This approach allows researchers to use qualitative data to provide detailed insights into the quantitative findings by selecting individuals who can offer the most relevant information.

In contrast, exploratory designs often involve different participants in stage 1 and stage 3. The goal of stage 3 in these designs is to generalize the findings to a larger population, which requires sampling a new and larger group of participants.

Using the same or different sample sizes: The sample sizes for stage 1 and stage 3 do not need to match. Similar to concurrent designs, qualitative data collection typically involves a much smaller sample than quantitative data collection. Since the aim of sequential designs is not to merge or directly compare the datasets, having unequal sample sizes is less problematic.

Selecting results for follow-up: In explanatory designs, researchers must determine which results from stage 1’s quantitative analysis warrant further exploration in stage 3’s qualitative phase. This involves identifying key findings that need deeper explanation, such as statistically significant or nonsignificant results, significant predictors, group differences, outlier cases, or unique demographic characteristics. Researchers may also choose participants who volunteer for qualitative follow-up. This decision-making process is also important in sequential designs that begin with qualitative data and then collect quantitative data to verify specific aspects of the qualitative findings. Researchers thus need to ensure that stage 3 focuses on the most meaningful aspects of the initial findings, providing a more comprehensive understanding of the research question.

Conclusion

Mixed methods data collection provides a comprehensive approach to exploring complex research questions by combining the strengths of qualitative and quantitative data. This approach allows for a deeper understanding of the context and the measurable patterns within a study. Whether using concurrent or sequential designs, the focus remains on selecting appropriate sampling strategies, employing effective data collection methods, and ensuring systematic recording of information.

Challenges such as balancing sample sizes, managing data from diverse sources, and maintaining consistency in data quality are inherent in mixed methods research. Addressing these challenges requires careful planning and attention to methodological rigor throughout the data collection process.

After data collection, the next steps involve data analysis, interpretation of results, and reporting findings. These stages will be discussed in the following articles, focusing on how to process, analyze, and present mixed methods data effectively.

References

1. Creswell, J. W., & Plano Clark, V. L. (2017). Designing and conducting mixed methods research (3rd ed.). SAGE.
2. Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed methods approaches (2nd ed.). Sage.
3. Padgett, D. K. (2004). The qualitative research experience. Wadsworth/Thomson Learning.