The length and complexity of describing research designs in your paper can vary considerably, but any well-developed design will achieve the following : Identify the research problem clearly and justify its selection, particularly in relation to any valid alternative designs that could have been used, Review and synthesize previously published literature associated with the research problem, Clearly and explicitly specify hypotheses [i.
Action Research Design Definition and Purpose The essentials of action research design follow a characteristic cycle whereby initially an exploratory stance is adopted, where an understanding of a problem is developed and plans are made for some form of interventionary strategy. This is a collaborative and adaptive research design that lends itself to use in work or community situations. Design focuses on pragmatic and solution-driven research outcomes rather than testing theories.
When practitioners use action research, it has the potential to increase the amount they learn consciously from their experience; the action research cycle can be regarded as a learning cycle. Action research studies often have direct and obvious relevance to improving practice and advocating for change.
There are no hidden controls or preemption of direction by the researcher. It is harder to do than conducting conventional research because the researcher takes on responsibilities of advocating for change as well as for researching the topic. Action research is much harder to write up because it is less likely that you can use a standard format to report your findings effectively [i. Personal over-involvement of the researcher may bias research results. The cyclic nature of action research to achieve its twin outcomes of action [e. Advocating for change usually requires buy-in from study participants.
Case Study Design Definition and Purpose A case study is an in-depth study of a particular research problem rather than a sweeping statistical survey or comprehesive comparative inquiry. Approach excels at bringing us to an understanding of a complex issue through detailed contextual analysis of a limited number of events or conditions and their relationships.
A researcher using a case study design can apply a variety of methodologies and rely on a variety of sources to investigate a research problem. Design can extend experience or add strength to what is already known through previous research. Social scientists, in particular, make wide use of this research design to examine contemporary real-life situations and provide the basis for the application of concepts and theories and the extension of methodologies.
The design can provide detailed descriptions of specific and rare cases. A single or small number of cases offers little basis for establishing reliability or to generalize the findings to a wider population of people, places, or things. Intense exposure to the study of a case may bias a researcher's interpretation of the findings. Design does not facilitate assessment of cause and effect relationships. Vital information may be missing, making the case hard to interpret.
The case may not be representative or typical of the larger problem being investigated. If the criteria for selecting a case is because it represents a very unusual or unique phenomenon or problem for study, then your intepretation of the findings can only apply to that particular case. Conditions necessary for determining causality: Empirical association -- a valid conclusion is based on finding an association between the independent variable and the dependent variable.
Appropriate time order -- to conclude that causation was involved, one must see that cases were exposed to variation in the independent variable before variation in the dependent variable. Nonspuriousness -- a relationship between two variables that is not due to variation in a third variable. Causality research designs assist researchers in understanding why the world works the way it does through the process of proving a causal link between variables and by the process of eliminating other possibilities. Replication is possible.
There is greater confidence the study has internal validity due to the systematic subject selection and equity of groups being compared. Not all relationships are casual! The possibility always exists that, by sheer coincidence, two unrelated events appear to be related [e. Conclusions about causal relationships are difficult to determine due to a variety of extraneous and confounding variables that exist in a social environment.
This means causality can only be inferred, never proven. If two variables are correlated, the cause must come before the effect. Cohort Design Definition and Purpose Often used in the medical sciences, but also found in the applied social sciences, a cohort study generally refers to a study conducted over a period of time involving members of a population which the subject or representative member comes from, and who are united by some commonality or similarity.
Date of entry and exit from the study is individually defined, therefore, the size of the study population is not constant. In open cohort studies, researchers can only calculate rate based data, such as, incidence rates and variants thereof. Closed Cohort Studies [static populations, such as patients entered into a clinical trial] involve participants who enter into the study at one defining point in time and where it is presumed that no new participants can enter the cohort.
Given this, the number of study participants remains constant or can only decrease. The use of cohorts is often mandatory because a randomized control study may be unethical. For example, you cannot deliberately expose people to asbestos, you can only study its effects on those who have already been exposed. Research that measures risk factors often relies upon cohort designs. Cohort analysis is highly flexible and can provide insight into effects over time and related to a variety of different types of changes [e. Either original data or secondary data can be used in this design.
In cases where a comparative analysis of two cohorts is made [e. These factors are known as confounding variables. Cohort studies can end up taking a long time to complete if the researcher must wait for the conditions of interest to develop within the group. This also increases the chance that key variables change during the course of the study, potentially impacting the validity of the findings.
Due to the lack of randominization in the cohort design, its external validity is lower than that of study designs where the researcher randomly assigns participants. Cross-Sectional Design Definition and Purpose Cross-sectional research designs have three distinctive features: no time dimension; a reliance on existing differences rather than change following intervention; and, groups are selected based on existing differences rather than random allocation.
Cross-sectional studies provide a clear 'snapshot' of the outcome and the characteristics associated with it, at a specific point in time. Unlike an experimental design, where there is an active intervention by the researcher to produce and measure change or to create differences, cross-sectional designs focus on studying and drawing inferences from existing differences between people, subjects, or phenomena. Entails collecting data at and concerning one point in time. While longitudinal studies involve taking multiple measures over an extended period of time, cross-sectional research is focused on finding relationships between variables at one moment in time.
Groups identified for study are purposely selected based upon existing differences in the sample rather than seeking random sampling. Cross-section studies are capable of using data from a large number of subjects and, unlike observational studies, is not geographically bound. Can estimate prevalence of an outcome of interest because the sample is usually taken from the whole population. Because cross-sectional designs generally use survey techniques to gather data, they are relatively inexpensive and take up little time to conduct.
Finding people, subjects, or phenomena to study that are very similar except in one specific variable can be difficult. Results are static and time bound and, therefore, give no indication of a sequence of events or reveal historical or temporal contexts. Studies cannot be utilized to establish cause and effect relationships. This design only provides a snapshot of analysis so there is always the possibility that a study could have differing results if another time-frame had been chosen. There is no follow up to the findings.
Descriptive Design Definition and Purpose Descriptive research designs help provide answers to the questions of who, what, when, where, and how associated with a particular research problem; a descriptive study cannot conclusively ascertain answers to why. The subject is being observed in a completely natural and unchanged natural environment. True experiments, whilst giving analyzable data, often adversely influence the normal behavior of the subject [a. Descriptive research is often used as a pre-cursor to more quantitative research designs with the general overview giving some valuable pointers as to what variables are worth testing quantitatively.
If the limitations are understood, they can be a useful tool in developing a more focused study. Descriptive studies can yield rich data that lead to important recommendations in practice. Appoach collects a large amount of data for detailed analysis. The results from a descriptive research cannot be used to discover a definitive answer or to disprove a hypothesis. Because descriptive designs often utilize observational methods [as opposed to quantitative methods], the results cannot be replicated.
The descriptive function of research is heavily dependent on instrumentation for measurement and observation. Experimental Design Definition and Purpose A blueprint of the procedure that enables the researcher to maintain control over all factors that may affect the result of an experiment. Experimental research allows the researcher to control the situation.
Experimental research designs support the ability to limit alternative explanations and to infer direct causal relationships in the study.
Quantitative Applications in the Social Sciences (QASS) | SAGE Publications Inc
Approach provides the highest level of evidence for single studies. The design is artificial, and results may not generalize well to the real world. The artificial settings of experiments may alter the behaviors or responses of participants. Experimental designs can be costly if special equipment or facilities are needed. Some research problems cannot be studied using an experiment because of ethical or technical reasons.
Difficult to apply ethnographic and other qualitative methods to experimentally designed studies. Exploratory Design Definition and Purpose An exploratory design is conducted about a research problem when there are few or no earlier studies to refer to or rely upon to predict an outcome.
The goals of exploratory research are intended to produce the following possible insights: Familiarity with basic details, settings, and concerns. Well grounded picture of the situation being developed. Generation of new ideas and assumptions. Development of tentative theories or hypotheses. Determination about whether a study is feasible in the future. Issues get refined for more systematic investigation and formulation of new research questions. Direction for future research and techniques get developed. Design is a useful approach for gaining background information on a particular topic.
Exploratory research is flexible and can address research questions of all types what, why, how. Provides an opportunity to define new terms and clarify existing concepts. Exploratory research is often used to generate formal hypotheses and develop more precise research problems.
In the policy arena or applied to practice, exploratory studies help establish research priorities and where resources should be allocated. Exploratory research generally utilizes small sample sizes and, thus, findings are typically not generalizable to the population at large. The exploratory nature of the research inhibits an ability to make definitive conclusions about the findings.
They provide insight but not definitive conclusions. The research process underpinning exploratory studies is flexible but often unstructured, leading to only tentative results that have limited value to decision-makers.
Design lacks rigorous standards applied to methods of data gathering and analysis because one of the areas for exploration could be to determine what method or methodologies could best fit the research problem. Historical Design Definition and Purpose The purpose of a historical research design is to collect, verify, and synthesize evidence from the past to establish facts that defend or refute a hypothesis.
The historical research design is unobtrusive; the act of research does not affect the results of the study. The historical approach is well suited for trend analysis. Historical records can add important contextual background required to more fully understand and interpret a research problem. There is often no possibility of researcher-subject interaction that could affect the findings. Historical sources can be used over and over to study different research problems or to replicate a previous study.
The ability to fulfill the aims of your research are directly related to the amount and quality of documentation available to understand the research problem. Since historical research relies on data from the past, there is no way to manipulate it to control for contemporary contexts. Interpreting historical sources can be very time consuming. The sources of historical materials must be archived consistently to ensure access.
This may especially challenging for digital or online-only sources. Original authors bring their own perspectives and biases to the interpretation of past events and these biases are more difficult to ascertain in historical resources.
Field Experiments and Natural Experiments
Due to the lack of control over external variables, historical research is very weak with regard to the demands of internal validity. It is rare that the entirety of historical documentation needed to fully address a research problem is available for interpretation, therefore, gaps need to be acknowledged. Longitudinal Design Definition and Purpose A longitudinal study follows the same sample over time and makes repeated observations. Longitudinal data facilitate the analysis of the duration of a particular phenomenon.
Enables survey researchers to get close to the kinds of causal explanations usually attainable only with experiments. The design permits the measurement of differences or change in a variable from one period to another [i. Indeed, it is possible to construct examples in which such an experiment has very little chance of rejecting a null hypothesis of no effect. How should one proceed in this case? Although any given study may have limited diagnostic value, the accumulation of studies generates an informative posterior distribution.
Unlike the failure to publish insignificant results, the failure to embark on low-power studies does not lead to bias, but it does slow the process of scientific discovery. In the words of Campbell and Stanley , 3 , we must. We must instill in our students the expectation of tedium and disappointment and the duty of thorough persistence, by now so well achieved in the biological and physical sciences.
Beyond the substantive knowledge that they generate, field experiments have the potential to make a profound methodological contribution. Rather than evaluate statistical methods in terms of the abstract plausibility of their underlying assumptions, researchers in the wake of LaLonde have made the performance of statistical methods an empirical question, by comparing estimates based on observational data to experimental benchmarks. Experimentation has been buoyed in part by the p. Arceneaux et al.
This finding echoes results in labor economics and medicine, where observational methods have enjoyed mixed success in approximating experimental benchmarks Heckman et al. By making the performance of observational methods an empirical research question, field experimentation is changing the terms of debate in the field of political methodology. Adams, W. Effects of telephone canvassing on turnout and preferences: a field experiment.
- The Supremest Human Being Part-2 (The Supremest Human Being Series).
- The Holy Spirit: Medieval Roman Catholic and Reformation Traditions.
- Why randomize?.
- Ten Stories for Coffee Breaks.
- Quantitative Research Designs in Educational Research - Education - Oxford Bibliographies;
- Methods of Randomization in Experimental Design : Valentim R. Alferes : !
Public Opinion Quarterly , 53— Find this resource:. Angrist, J. Identification of casual effects using instrumental variables. Journal of the American Statistical Association , — Ansolabehere, S. Old voters, new voters, and the personal vote: using redistricting to measure the incumbency advantage. American Journal of Political Science , 17— Arceneaux, K. Comparing experimental and matching methods using a large-scale voter mobilization experiment. Political Analysis , 1— Benz, M. Do people behave in experiments as in the field?
Evidence from donations. Campbell, D. Experimental and Quasi-Experimental Designs for Research. Boston: Houghton-Mifflin. Cardy, E. An experimental field study of the GOTV and persuasion effects of partisan direct mail and phone calls. Chapin, F. Experimental Designs in Sociological Research. New York: Harper and Brothers. Chattopadhyay, R. Women as policy makers: evidence from a randomized policy experiment in India. Econometrica , — Chin, M. A foot in the door: an experimental study of PAC and constituency effects on access. Journal of Politics , — Concato, J. Randomized, controlled trials, observational studies, and the hierarchy of research designs.
New England Journal of Medicine , — Cover, A. Baby books and ballots: the impact of congressional mail on constituent opinion. American Political Science Review , — Doherty, D. Personal income and attitudes toward redistribution: a study of lottery winners. Political Psychology , — Dunn, G.
Estimating treatment effects from randomized clinical trials with noncompliance and loss to follow-up: the role of instrumental variables methods. Statistical Methods in Medical Research , — Eldersveld, S. Experimental propaganda techniques and voting behavior. The effects of canvassing, direct mail, and telephone contact on voter turnout: a field experiment. The illusion of learning from observational research. Shapiro, R.
Smith, and T. New York: Cambridge University Press. Does the media matter? A field experiment measuring the effect of newspapers on voting behavior and political opinions. American Economic Journal: Applied Economics. Glazerman, S. Nonexperimental versus experimental estimates of earnings impacts.
Gosnell, H. Chicago: University of Chicago Press. Guan, M. Non-coercive mobilization in state-controlled elections: an experimental study in Beijing. Comparative Political Studies , — Habyarimana, J. The Co-ethnic Advantage. Harrison, G. Field experiments. Journal of Economic Literature , — Hastings, J. Economic outcomes and the decision to vote: the effect of randomized school admissions on voter participation. Unpublished manuscript, Department of Economics, Yale University.
Heckman, J. Assessing the case for social experiments. Journal of Economic Perspectives , 9: 85— Matching as an econometric evaluation estimator. Review of Economic Studies , — Holland, P. Statistics and causal inference. Howell, W. Uses of theory in randomized field trials: lessons from school voucher research on disaggregation, missing data, and the generalization of findings. Anerican Behavioral Scientist , — Hyde, S. Foreign democracy promotion, norm development and democratization: explaining the causes and consequences of internationally monitored elections.
Imbens, G. Bayesian inference for causal effects in randomized experiments with noncompliance. Annals of Statistics , — Estimating the effect of unearned income on labor earnings, savings, and consumption: evidence from a survey of lottery winners. American Economic Review , — Johnson, J.
Political Science Research Methods , 4th edn. King, G. Designing Social Inquiry. Kling, J. Neighborhood effects on crime for female and male youth: evidence from a randomized housing voucher experiment. Quarterly Journal of Economics , 87— LaLonde, R. Evaluating the econometric evaluations of training programs with experimental data. Comparative politics and the comparative method.
Lowell, A. The physiology of politics. American Political Science Review , 4: 1— Mahoney, J. Comparative Historical Analysis in the Social Sciences. Manski, C. Nonparametric bounds on treatment effects. American Economic Review Papers and Proceedings , — Michelson, M. Getting out the Latino vote: how door-to-door canvassing influences voter turnout in rural central California. Political Behavior , — Miguel, E. Worms: identifying impacts on education and health in the presence of treatment externalities.
Economic shocks and civil conflict: an instrumental variables approach. Journal of Political Economy , — Miller, R. Stimulating voter turnout in a primary: field experiment with a precinct committeeman. International Political Science Review , 2: — Moffitt, R. The role of randomized field trials in social science research: a perspective from evaluations of reforms of social welfare programs. American Behavioral Scientist , — Newhouse, J. Free for All? Boston: Harvard University Press. Neyman, J. On the application of probability theory to agricultural experiments.
Essay on principles. Section 9. Roczniki Nauk Roiniczych , 1—51; repr. Nickerson, D. Is voting contagious? Evidence from two field experiments. American Political Science Review , 49— Olken, B. Monitoring corruption: evidence from a field experiment in Indonesia.
Study Design Terminology
Pechman, J. Washington, DC: Brookings Institution. Pettersson-Lidbom, P. Does the size of the legislature affect the size of government? Evidence from two natural experiments. Unpublished manuscript, Department of Economics, Stockholm University. Posner, D.
The political salience of cultural difference: why Chewas and Tumbukas are allies in Zambia and adversaries in Malawi. Raudenbush, S. Statistical analysis and optimal design for cluster randomized trials. Psychological Methods , 2: — Rubin, D. Bayesian inference for causal effects: the role of randomization. Annals of Statistics , 6: 34— Comment: Neyman and causal inference in experiments and observational studies.
Statistical Science , 5 4 : — Sears, D. Journal of Personality and Social Psychology , — Sherman, L. Deterrent effects of police raids on crack houses: a randomized, controlled experiment. Justice Quarterly , — Wantchekon, L. Clientelism and voting behavior: evidence from a field experiment in Benin. World Politics , — This assumption would be violated if treatment assignment were perfectly predicted by X.
Alan S. Donald P. Green Ph. The author of four books and more than one hundred essays, Green's research interests span a wide array of topics: voting behavior, partisanship, campaign finance, hate crime, and research methods. Much of his current work uses field experimentation to study the ways in which political campaigns mobilize and persuade voters, but he has also conducted experimental research on the effects of the mass media, civic education classes, and criminal sentencing.
Unlike qualitative studies, quantitative usability studies aim to result in findings that are statistically likely to generalize to the whole user population. Often, the main goal of quantitative usability studies is to compare — a site with its competitors, two different iterations of a design, or two different groups of users such as experts vs.
Like in any scientific experiment in which we want to detect causal relationships, a quantitative study involves two types of variables :. If the study produces statistically significant results, then we can say that a change in the independent variable caused a change in the dependent variable. If we wanted to measure which of the two sites, A or B, is better for the task of reserving a car, we could choose Site with two possible values or levels — A and B as the independent variable, and the time on task and the accuracy for booking a car could be the dependent variables.
The goal of the study would be to see whether the dependent variables time and accuracy change when we vary the site or they stay the same. If they stay the same, then none of the sites is better than the other. To plan our study, the next question that we need to answer is whether the study design should be between-subjects or within-subjects — that is, whether a participant in the study should be exposed to all the different conditions for the independent variable in our study within-subjects or only to one condition between-subjects.
The choice of experimental design will affect the type of statistical analysis that should be used on your data. It is possible that an experiment design is both within-subjects and between-subjects. For example, assume that, in the case of our car-rental study, we were also interested in knowing how participants younger than 30 perform compared with older participants. In this case we would have two independent variables:.
For the study, we will recruit an equal number of participants in each age group. In this case, the study is within-subjects with respect to the independent variable Site because each person sees both levels of this variable — that is, both site A and site B. However, the study is between-subjects with respect to Age : one person can only be in a single age group either under or over 30, not both.
Well, technically, you could pick a group of underyear olds and wait until they turn 30 to have them test the sites again, but this setup is highly impractical for most real-world situations. Some independent variables may impose the choice of design. Age is one of them, as seen above.
Others are Expertise if we want to compare experts and novices , User Type if we want to compare different user groups or personas — for example, business traveler vs. Outside usability, drug trials are one common case of between-subject design: participants are exposed to only one treatment: either the drug being tested or a placebo, not both. Unfortunately, there is no easy answer to this question.
As seen above, sometimes your independent variables will dictate the experimental design. But in many situations, both designs may be possible.
Related Methods of Randomization in Experimental Design (Quantitative Applications in the Social Sciences)
Copyright 2019 - All Right Reserved