Skip Nav

Annex 1: Strengths and weaknesses of quantitative evaluation designs

Resource Links:

❶Data from quantitative research—such as market size, demographics, and user preferences—provides important information for business decisions. While at NASA, Bryan worked on a variety of research studies, encompassing communication and human factors and interacting with hundreds of participants.

Lack of resources for data collection

How to Cite
Language & Lit
You are here

Because the data is in a numeric form, we can apply statistical tests in making statements about the data. These include descriptive statistics like the mean, median, and standard deviation, but can also include inferential statistics like t-tests, ANOVAs, or multiple regression correlations MRC. Statistical analysis lets us derive important facts from research data, including preference trends, differences between groups, and demographics.

Multivariate statistics like the MRC or stepwise correlation regression break the data down even further and determine what factors—such as variances in preferences—we can attribute to differences between specific groups such as age groups. Quantitative studies often employ automated means of collecting data such as surveys, but we can also use other static methods—for example, examining preferences through two-alternative, forced-choice studies or examining error rates and time on task using competitive benchmarks.

Some respondents may feel that President Obama is too liberal, while others may feel that he is too conservative in his actions, but without the necessary data, there is no way to tell. In a product-development environment, this data deficiency can lead to critical errors in the design of a product.

Additionally, only someone with a firm grasp of how they should use and interpret quantitative statistics should conduct such a study.

For most tests, there is an overreliance on the p-value and sample size. The p-value is a statistic that indicates the likelihood that research findings were the result of chance.

If a p-value is less than. If your study is underpowered because of its having two small a sample size, you may fail to achieve statistical significance—even if the finding is accurate. The reality is not too far off. However, it is possible to increase sample sizes to a point where statistical significance is barely meaningful. In such a situation, it is important to look at the effect size — a statistic that tells you how strongly your variables effect the variance.

Basically, statistical significance tells you whether your findings are real, while effect size tells you how much they matter. Typically, if you are able to achieve statistical significance with a smaller sample size, the effect size is fairly substantial. It is important to take both statistical significance and effect size into account when interpreting your data.

Data from qualitative studies describes the qualities or characteristics of something. You cannot easily reduce these descriptions to numbers—as you can the findings from quantitative research; though you can achieve this through an encoding process. Qualitative research studies can provide you with details about human behavior, emotion, and personality characteristics that quantitative studies cannot match.

While quantitative research requires the standardization of data collection to allow statistical comparison, qualitative research requires flexibility, allowing you to respond to user data as it emerges during a session. Thus, qualitative research usually takes the form of either some form of naturalistic observation such as ethnography or structured interviews.

In this case, a researcher must observe and document behaviors, opinions, patterns, needs, pain points, and other types of information without yet fully understanding what data will be meaningful.

Following data collection, rather than performing a statistical analysis, researchers look for trends in the data. When it comes to identifying trends, researchers look for statements that are identical across different research participants. The rule of thumb is that hearing a statement from just one participant is an anecdote; from two, a coincidence; and hearing it from three makes it a trend. The trends that you identify can then guide product development, business decisions, and marketing strategies.

Because you cannot subject these trends to statistical analysis, you cannot validate trends by calculating a p-value or an effect size—as you could validate quantitative data—so you must employ them with care. Plus, you should continually verify such data through an ongoing qualitative research program. With enough time and budget, you can engage in an activity called behavioral coding , which involves assigning numeric identifiers to qualitative behavior, thus transforming them into quantitative data that you can then subject to statistical analysis.

In addition to the analyses we described earlier, behavioral coding lets you perform a variety of additional analyses such as lag sequential analysis , a statistical test that identifies sequences of behavior—for example, those for Web site navigation or task workflows.?

However, applying behavioral coding to your observations is extremely time consuming and expensive. Plus, typically, only very highly trained researchers are qualified to encode behavior.

Thus, this approach tends to be cost prohibitive. Additionally, because it is not possible to automate qualitative-data collection as effectively as you can automate quantitative-data collection, it is usually extremely time consuming and expensive to gather large amounts of data, as would be typical for quantitative research studies. As a result, qualitative research tends to have less statistical power than quantitative research when it comes to discovering and verifying trends.

While quantitative and qualitative research approaches each have their strengths and weaknesses, they can be extremely effective in combination with one another. You can use qualitative research to identify the factors that affect the areas under investigation, then use that information to devise quantitative research that assesses how these factors would affect user preferences.

To continue our earlier example regarding display preferences: An example of a qualitative trend might be that younger users prefer autostereoscopic displays only on mobile devices, while older users prefer traditional displays on all devices.

You may have discovered this by asking an open-ended, qualitative question along these lines: In a subsequent quantitative study, you could address these factors through a series of questions such as: An automated system assigns a numeric value to whatever option a participant chooses, allowing a researcher to quickly gather and analyze large amounts of data.

When setting out to perform user research—whether performing the research yourself or assigning it to an employee or a consultant—it is important to understand the different applications of these two approaches to research. This understanding can help you to choose the appropriate research approach yourself, understand why a researcher has chosen a particular approach, or communicate with researchers or stakeholders about a research approach and your overarching research strategy.

In what other ways do you use and combine qualitative and quantitative research? The quantitative approach is so vital, even in our daily lives, because in most, if not all things we do in life, we measure to see how much there is of something. Quantitative method is part of our daily life, even from birth, data are constantly being collected, assessed, and re-assessed as we grow. I also support the quantitative data because it is much used and almost whatever we do involves it.

Both quantitative and qualitative research are important on their own. It depends on the situation where a researcher conducts a particular research, or he can go for the mixed method, too. For now, I am in need of sampling and non-sampling errors. Please help me understand its applications and the ways that can be checked?

Types of sampling and all related information on this chapter. Quantitative data provides the facts, but facts about people are just another construct of our society.

Business understands that neither method should be relied upon exclusively, which is why they use both. Anyone who thinks this is a competition between the two methods to somehow win out needs to read the article again. I also think that the quantitative approach is more important than the qualitative approach because we use it more and more in our life time.

I would suggest using both quantitative and qualitative. Data collection is based on participants' meanings rather than a more objective collection of statistics. Qualitative research often involves cross-case comparisons. Qualitative research tends to cause a researcher to become immersed in the research topic. For example, a researcher using qualitative research may conduct in-depth interviews, interact with participants and rely on her own observations.

A researcher using quantitative research methods remains separated from the subject matter. The researcher remains objective when conducting research. Instead of conducting in-depth interviews, a researcher may use analysis and questionnaires to test a hypothesis. An advantage of using quantitative research is that the researcher remains more objective while proving or disproving a hypothesis. Quantitative and qualitative research both encompass planning before conducting or analyzing research.

Quantitative research, however, involves more planning, which becomes a disadvantage. For instance all aspects of a research study must be carefully designed before collecting any data. A researcher needs a concrete hypothesis and needs to know the type of research involvedsuch as questionnaires and test questions. With qualitative research, the design typically emerges as the research study develops.

Quantitative research depends on data and involves testing a hypothesis, but it can miss contextual details. For example, a researcher doesn't provide a detailed description when using quantitative research. Instead the researcher depends on numbers and statistics to prove a hypothesis.

Qualitative Studies

Main Topics

Privacy Policy

Limitations and weakness of quantitative research methods By Priya Chetty on September 7, According to Saunders et al. (), research methodology serves as the backbone of a research study.

Privacy FAQs

Strengths. Weaknesses. Study findings can be generalized to the population about which information is required. Samples of individuals, communities, or organizations can be selected to ensure that the results will be representative of the population studied.

About Our Ads

quantitative research methodologies under a same research. Research Question Comparatively evaluate the strengths and weaknesses of quantitative and qualitative research methodologies. II. Literature Reviews The Strengths of Quantitative Research Methodology The quantitative as survey approach has two significant advantages. Strengths and limitations Quantitative method Quantitive data are pieces of information that can be counted and which are usually gathered by surveys from large numbers of respondents randomly selected for inclusion.

Cookie Info

Using Quantitative and Qualitative Research Together While quantitative and qualitative research approaches each have their strengths and weaknesses, they can be extremely effective in combination with one another. strengths and limitations of qualitative and quantitative research methods European Journal of Education Studies - Volume 3 │ Issue 9 │ large groups of people, which are generally better representativeness.