Survey research: Definition and methods

A survey asks multiple questions about respondents' perspectives or experiences regarding a particular topic. Conducting survey research can help market researchers and scholars understand perspectives among a target population and generate meaningful insights.

Let's examine how survey research can help you identify trends and gain a deeper understanding of the perspectives of your respondents.

Why are surveys useful in research?

When researchers collect survey data, they often do so to analyze respondents' perspectives and experiences to draw conclusions about the opinions or dispositions about this group of people. Research surveys can be used to study just about any topic, such as:

• Customer surveys to gain customer insights for market research
• Census surveys to collect demographic data
• Needs analysis surveys for institutions to plan future strategies
• NPS surveys (net promoter score) to assess customer experience and loyalty
• Open-ended surveys to collect data for qualitative research

What is survey data analysis?

The research questions and objectives for the study will inform the survey design and survey report. In turn, a good survey design allows researchers to identify actionable insights when they analyze survey data. Survey analysis methods aim to take the data collected from surveys and generate valuable insights about a particular issue or group of people.

Understanding and evaluating survey research

The types of questions asked will inform the survey data analysis methods and the conclusions that can be drawn from survey respondents. Multiple-choice questions, for example, are fast and easy to analyze, while questions with open-ended responses can collect much richer data.

There are other concerns affecting survey design and, thus, the methods for analyzing survey data:

• Reaching a sufficient number of people and motivating them to answer the survey
• Ensuring that survey questions are understood correctly to collect valuable data from respondents
• Crafting a survey that respondents will complete without providing incomplete responses
• Deciding between in-person surveys or online surveys when collecting data

Let's look at the different types of survey questions to understand what questions are best suited for different objectives that a researcher or data scientist may have.

Types of questions

In surveys, you usually ask a set of questions to a target group (e.g., a company's customers, a specific demographic group, etc.). Surveys ask the same questions to different people to provide researchers with comparative data between respondents and aggregate data representing a group’s perspectives.

Close-ended questions

A common question type in survey data research is the close-ended question. These questions can have yes/no options, true/false options, or an extended list of multiple-choice options that respondents can choose from.

Advantages of close-ended questions

Close-ended questions are the easiest to answer and analyze because of their simplicity. Researchers can turn responses into quantitative data with numerical values for statistical analysis when there are only a fixed number of answers.

Cross-tabulation analysis can also identify patterns. For example, when analyzing data from a survey about lifestyle choices, researchers may be interested in the percentage of people who say they sleep more than six hours a night and exercise twice a week.

To allow for cross-tabulation, a survey needs close-ended questions about respondents' sleep and exercise habits with a list of choices for each (e.g., "less than six hours of sleep," "more than six hours of sleep", etc.). Survey data analysis can check for the number of respondents who answered with a specific combination of choices (e.g., "more than six hours of sleep" and "exercise twice a week").

The results of close-ended questions can also contribute to the formation of questions for other research methods, such as interviews and focus groups, when following up with survey respondents.

Challenges for close-ended questions

Close-ended questions restrict the array of possible answers. If a survey has to ask what kinds of exercise a respondent does, for example, it is likely impossible to list all of them at once. Thus, close-ended questions might leave out other potential answers.

Likert scale questions

A special kind of close-ended question asks respondents to select from a range of numbers. Some examples of Likert scale questions include:

• Rate your level of satisfaction with this product
• Rate the likelihood that you will use this service again
• Rate your level of familiarity with this topic

Survey items like these almost always come with a numerical scale (e.g., one to five). Taking the last survey item as an example, a one on a five-point scale may represent little to no familiarity, while a five may represent total familiarity.

Advantages of Likert scale questions

It's easy to perform a survey results analysis on Likert scale questions because the answers exist in an ordered range on an interval scale, unlike other close-ended questions where some items may not be related (e.g., badminton and soccer in a list of favorite sports). As a result, researchers may employ Likert scale questions when they want to perform quantitative analysis of an ordered phenomenon (e.g., level of satisfaction, willingness to do something, etc.).

Challenges for Likert scale questions

The numbers in a Likert scale usually represent different values and are open to interpretation. For example, if someone selects three on a five-point scale to indicate their level of satisfaction with a product, do they mean that they are undecided about their satisfaction or feel balanced between the product's good and bad points?

Without further inquiry or more probing survey questions, knowing more about their answer may be impossible. Researchers may benefit from combining survey results with other forms of data, such as interviews and observations. In addition, open-ended questions may be included in the survey to allow respondents space to provide details on their answers.

Open-ended questions

People are very different and have various attitudes and insights. Researchers may thus want to consider asking questions in an open-ended manner to effectively analyze an issue or phenomenon.

Advantages of open-ended questions

Respondents can write free text in an open-ended question rather than choose from a list of answers. This is helpful when the list of possible answers to a question (e.g., "Who's your favorite professional athlete?") may be too long or it is difficult to determine a meaningful list of answers beforehand.

Another benefit is that respondents can provide detailed answers. For example, a survey on customer experience may ask respondents how they have experienced a product or service. In such a case, a respondent might need at least a paragraph to provide a sufficient answer, and these questions can generate rich qualitative data. Moreover, open-ended questions give participants the space to explain their thought processes or mention other aspects that the researcher or previous studies may not have taken into consideration.

Sampling strategies in survey research

Simply determining how many respondents are necessary for your survey is insufficient for rigorous research. Understanding and applying the correct sampling strategy in survey research is crucial for achieving representative and helpful results. It ensures that the respondents included in your survey are a good fit for your research objectives and can provide valuable insights about the research issue or group you are studying.

Probability sampling

In probability sampling, every individual in your target population has an equal chance of being selected to participate in the survey. This type of sampling allows you to make generalizations about the broader population from your survey data. There are several subtypes of probability sampling, such as simple random sampling, systematic sampling, stratified sampling, and cluster sampling.

However, probability sampling can be time-consuming and costly, as it may require comprehensive lists of potential respondents and often involves a larger sample size to ensure representation. Moreover, achieving statistical significance and generalizability are typically goals of quantitative research methods, which can make this particularly important to consider if pursuing mixed methods studies for instance, but less relevant for studies that primarily follow a qualitative approach.

Non-probability sampling

Non-probability sampling, in contrast, does not involve random selection. In this method, some individuals have a greater chance of being selected than others, or the selection is done intentionally by the researcher. Examples of non-probability sampling include convenience sampling, theoretical sampling, judgmental or purposive sampling, quota sampling, and snowball sampling.

While this method is typically easier and less costly than probability sampling, it comes with the downside that it may not provide a representative sample of the population, making it harder to generalize the findings. This can be a considerable downside for quantitative methods or mixed methods studies. However, in qualitative research, purposive and theoretical sampling are often preferred because they help researchers ensure that they are collecting detailed data from participants who can shed light on the research question or help flesh out the researcher’s conceptual framework.

How to decide which sampling strategy to use

The choice between probability and non-probability sampling strategies depends on various factors. These factors include your research objectives, available resources (time, money, and personnel), the nature of the population, and the degree of precision required. You should consider these factors carefully when deciding on which methodology you will follow in your research. Remember that the key aim is to choose a method that will best help you answer your research question while being mindful of your resources and constraints.

Designing effective surveys

Creating effective survey questions is an art that combines clarity, relevance, and precision. Good questions will capture accurate data that truly reflects the respondents' perspectives and experiences. Conversely, poorly designed questions can confound data collection and interfere with the researcher's ability to draw meaningful conclusions from their survey.

Writing clear and concise questions

Clear and concise questions are critical in survey design. Respondents should be able to easily understand the question without having to make guesses about what you're asking. Avoid using complex words, industry jargon, or long sentences. If the question is too long, the respondent may lose interest or might not fully understand what's being asked. Be specific and direct in your questions to ensure you collect meaningful data.

Avoiding bias and leading questions

One of the main pitfalls in survey question design is bias. Bias in questions can lead respondents towards a particular answer, skewing your results. For instance, a question that says, "Don't you think that our product is the best on the market?" is leading because it suggests that our product is superior. Instead, you could ask, "How would you rate our product compared to others available on the market?" This phrasing is neutral and allows the respondent to provide their genuine opinion.

Using neutral language

Neutral language is essential to encourage sincere responses. It's crucial that the wording of your questions does not influence the respondent's answer. Use terms that are non-controversial and don't carry positive or negative connotations. For instance, instead of asking, "How would you rate this excellent medical treatment?" you could ask, "How would you rate this medical treatment?" This allows respondents to form and express their own opinion.

Structuring your survey

The structure of your survey can significantly influence the quality of responses you receive. A well-structured survey flows logically, respects respondents' time, and captures the information you need for your research.

Ultimately, a survey that avoids survey fatigue through a logical structure and content that is easy to understand shows respect for the respondent's time and thought process. It can also greatly improve the quality of your data by reducing the risk of dropouts or rushed answers. Always be mindful of the respondent's experience when designing your survey structure.

Logical sequence of questions

A logical flow of questions aids the respondent's understanding and enhances the coherence of the response. Start with general questions and gradually move towards more specific ones. You might also consider starting with less sensitive questions to build rapport and trust, then progress to more sensitive or personal topics. If you're asking several questions on the same topic, group them together to make the survey easier to follow.

The role of demographic questions

Demographic questions about age, gender, location, income, education, and similar topics can provide important context for your analysis. However, be mindful of where you place these questions within your survey. If placed at the beginning, they might deter some respondents from completing the survey due to privacy concerns. Therefore, it's often recommended to place demographic questions at the end unless they are crucial for qualifying respondents.

Balancing the number of questions

The number of questions in your survey can impact the response rate and the quality of responses. A long survey can deter potential respondents or cause them to rush through answers.

On the other hand, a very short survey might not provide the depth of information you need. Striking a balance is key: ensure you're asking enough questions to capture the necessary data, but not so many that it overwhelms the respondent.

What are the different types of survey research?

The way you deliver your survey can impact who responds and how they engage with your questions. Different delivery methods come with their own sets of advantages and disadvantages.

When selecting a survey delivery method, consider your target population, available resources, and the nature of the questions you're asking. The right method for you will balance these factors to achieve the best possible response rate and data quality.

Online surveys

Online surveys have become extremely popular due to their convenience and cost-effectiveness. They are quick to distribute and collect, can easily reach large or far-away geographical areas, and can automate data collection. Furthermore, an online survey allows respondents to answer at their own pace.

However, an online survey can also have limitations. They may exclude those without internet access or who are not tech-savvy, creating a digital divide. Additionally, online surveys may have lower response rates because they can be easily ignored or overlooked.

Telephone surveys

Telephone surveys allow for more personal interaction, which can result in higher response rates. They also allow the researcher to clarify any questions on the spot if respondents find them confusing.

On the downside, telephone surveys can be time-consuming and more expensive than online surveys. They are also more likely to suffer from response bias, as respondents may feel pressured to give socially acceptable answers.

In-person surveys

In-person surveys can lead to higher-quality data because they allow for a more comprehensive response from your target audience. They also allow for recording non-verbal cues, which can provide additional insights. Furthermore, complicated or sensitive questions can be better handled in person.

However, in-person surveys can be the most costly and time-consuming to administer. They also limit the geographical reach of your survey and can introduce interviewer bias, where the presence of the interviewer influences the respondent's answers.

Organizing survey responses for data analysis

Once you've collected your survey responses, the next crucial step is organizing the data to facilitate analysis. Spreadsheets and Comma Separated Value (CSV) files are two common formats for organizing survey data. Both formats are also commonly used in qualitative data analysis software to analyze survey results.

Remember that consistency is critical when organizing your data, regardless of your chosen format. Ensure the same structure is used for all responses, and always double-check for errors or inconsistencies before beginning your analysis. An organized and consistent dataset will make the analysis stage of your research significantly smoother.

Using spreadsheets for survey data

Spreadsheets, like those created in Excel or Google Sheets, are popular for organizing survey data. In a spreadsheet, each row typically represents a separate respondent, and each column represents a question from the survey.

The benefits of using spreadsheets include the ability to easily sort and filter data, the option to use formulas for quick calculations, and the visual simplicity of the grid format. Spreadsheets also offer compatibility with many qualitative and quantitative data analysis software packages.

However, spreadsheets can become unwieldy if dealing with a very large volume of data or complex data types. They may also lack the functionality of more specialized data management tools, particularly for advanced statistical analysis.

Using CSV files for survey data

Comma Separated Value (CSV) files are a type of plain text file where each line represents a row of data, and commas separate values (or fields). They are a universal format and are widely accepted across different software applications, including those for qualitative data analysis.

One major advantage of CSV files is their simplicity and compatibility. They can be opened and edited in a variety of programs, including plain text editors. This makes them a flexible choice for data organization.

However, like spreadsheets, CSV files can be hard to navigate when dealing with large or complex datasets. They also lack some of the more advanced features of spreadsheets, like the ability to use formulas or create charts and graphs directly within the file.