experimental group is to control group what ________ is to ________.

Experimental research, often considered to be the "gold standard" in research designs, is one of the most rigorous of all research designs. In this pattern, one or more independent variables are manipulated by the researcher (as treatments), subjects are randomly assigned to different handling levels (random consignment), and the results of the treatments on outcomes (dependent variables) are observed. The unique force of experimental research is its internal validity (causality) due to its ability to link crusade and consequence through handling manipulation, while controlling for the spurious effect of extraneous variable.

Experimental research is all-time suited for explanatory enquiry (rather than for descriptive or exploratory enquiry), where the goal of the written report is to examine cause-effect relationships. It also works well for research that involves a relatively limited and well-defined gear up of contained variables that can either exist manipulated or controlled. Experimental enquiry can be conducted in laboratory or field settings. Laboratory experiments , conducted in laboratory (artificial) settings, tend to exist high in internal validity, but this comes at the cost of low external validity (generalizability), because the bogus (laboratory) setting in which the study is conducted may not reflect the real world. Field experiments , conducted in field settings such every bit in a real arrangement, and high in both internal and external validity. Just such experiments are relatively rare, because of the difficulties associated with manipulating treatments and decision-making for extraneous effects in a field setting.

Experimental research can be grouped into two wide categories: true experimental designs and quasi-experimental designs. Both designs require treatment manipulation, but while true experiments likewise require random consignment, quasi-experiments practice not. Sometimes, nosotros also refer to non-experimental research, which is non really a research design, but an spread-out term that includes all types of enquiry that do not employ treatment manipulation or random assignment, such as survey inquiry, observational research, and correlational studies.

Basic Concepts

Handling and control groups. In experimental research, some subjects are administered ane or more experimental stimulus called a treatment (the treatment grouping ) while other subjects are non given such a stimulus (the control grouping ). The treatment may be considered successful if subjects in the treatment grouping rate more than favorably on outcome variables than control group subjects. Multiple levels of experimental stimulus may be administered, in which case, there may be more ane treatment group. For example, in club to examination the effects of a new drug intended to treat a sure medical condition like dementia, if a sample of dementia patients is randomly divided into iii groups, with the first group receiving a high dosage of the drug, the second group receiving a low dosage, and the third group receives a placebo such as a saccharide pill (control grouping), then the offset 2 groups are experimental groups and the third group is a command grouping. Afterwards administering the drug for a catamenia of fourth dimension, if the condition of the experimental group subjects improved significantly more than the command group subjects, we can say that the drug is constructive. Nosotros can also compare the conditions of the loftier and low dosage experimental groups to determine if the high dose is more effective than the low dose.

Treatment manipulation. Treatments are the unique feature of experimental research that sets this design autonomously from all other research methods. Treatment manipulation helps control for the "crusade" in crusade-consequence relationships. Naturally, the validity of experimental inquiry depends on how well the treatment was manipulated. Treatment manipulation must be checked using pretests and pilot tests prior to the experimental study. Any measurements conducted earlier the treatment is administered are called pretest measures , while those conducted subsequently the treatment are posttest measures .

Random selection and assignment. Random selection is the process of randomly drawing a sample from a population or a sampling frame. This approach is typically employed in survey research, and assures that each unit in the population has a positive hazard of being selected into the sample. Random assignment is notwithstanding a process of randomly assigning subjects to experimental or control groups. This is a standard practice in true experimental research to ensure that treatment groups are similar (equivalent) to each other and to the control grouping, prior to treatment administration. Random selection is related to sampling, and is therefore, more closely related to the external validity (generalizability) of findings. Still, random consignment is related to pattern, and is therefore well-nigh related to internal validity. It is possible to take both random selection and random assignment in well-designed experimental inquiry, just quasi-experimental research involves neither random selection nor random assignment.

Threats to internal validity. Although experimental designs are considered more rigorous than other enquiry methods in terms of the internal validity of their inferences (by virtue of their ability to control causes through treatment manipulation), they are not immune to internal validity threats. Some of these threats to internal validity are described below, within the context of a written report of the impact of a special remedial math tutoring program for improving the math abilities of high school students.

• History threat is the possibility that the observed furnishings (dependent variables) are caused by extraneous or historical events rather than by the experimental treatment. For instance, students' post-remedial math score improvement may accept been caused past their preparation for a math exam at their school, rather than the remedial math program.

• Maturation threat refers to the possibility that observed effects are caused past natural maturation of subjects (e.k., a general comeback in their intellectual ability to understand complex concepts) rather than the experimental treatment.

• Testing threat is a threat in pre-mail service designs where subjects' posttest responses are conditioned by their pretest responses. For instance, if students remember their answers from the pretest evaluation, they may tend to repeat them in the posttest examination. Not conducting a pretest tin help avoid this threat.

• Instrumentation threat , which also occurs in pre-post designs, refers to the possibility that the difference between pretest and posttest scores is not due to the remedial math program, but due to changes in the administered test, such as the posttest having a college or lower caste of difficulty than the pretest.

• Mortality threat refers to the possibility that subjects may exist dropping out of the study at differential rates between the handling and control groups due to a systematic reason, such that the dropouts were mostly students who scored low on the pretest. If the low-performing students drop out, the results of the posttest will be artificially inflated by the preponderance of high-performing students.

• Regression threat , as well called a regression to the hateful, refers to the statistical trend of a group's overall performance on a measure out during a posttest to regress toward the hateful of that measure rather than in the anticipated direction. For case, if subjects scored loftier on a pretest, they will have a tendency to score lower on the posttest (closer to the mean) because their loftier scores (away from the mean) during the pretest was possibly a statistical aberration. This problem tends to be more prevalent in non-random samples and when the ii measures are imperfectly correlated.

Ii-Grouping Experimental Designs

The simplest true experimental designs are two group designs involving ane treatment group and ane control group, and are ideally suited for testing the effects of a single independent variable that can be manipulated as a handling. The two basic two-group designs are the pretest-posttest control group design and the posttest-but control group pattern, while variations may include covariance designs. These designs are often depicted using a standardized blueprint notation, where R represents random assignment of subjects to groups, X represents the handling administered to the treatment group, and O represents pretest or posttest observations of the dependent variable (with dissimilar subscripts to distinguish between pretest and posttest observations of treatment and control groups).

Pretest-posttest control group design . In this design, subjects are randomly assigned to treatment and command groups, subjected to an initial (pretest) measurement of the dependent variables of involvement, the handling group is administered a handling (representing the independent variable of interest), and the dependent variables measured again (posttest). The notation of this design is shown in Figure 10.ane.

Figure x.1. Pretest-posttest control group blueprint

The outcome Due east of the experimental treatment in the pretest posttest design is measured every bit the difference in the posttest and pretest scores between the treatment and command groups:

Due east = (O 2 – O i ) – (O four – O 3 )

Statistical analysis of this design involves a simple analysis of variance (ANOVA) betwixt the treatment and control groups. The pretest posttest design handles several threats to internal validity, such as maturation, testing, and regression, since these threats can be expected to influence both handling and control groups in a similar (random) manner. The option threat is controlled via random assignment. Still, additional threats to internal validity may be. For instance, mortality can exist a problem if in that location are differential dropout rates between the 2 groups, and the pretest measurement may bias the posttest measurement (especially if the pretest introduces unusual topics or content).

Posttest-but control group design . This design is a simpler version of the pretest-posttest design where pretest measurements are omitted. The blueprint note is shown in Figure 10.2.

Figure 10.two. Posttest simply command group design

The treatment result is measured simply equally the deviation in the posttest scores betwixt the ii groups:

E = (O 1 – O ii )

The appropriate statistical analysis of this design is as well a two- grouping analysis of variance (ANOVA). The simplicity of this design makes it more attractive than the pretest-posttest design in terms of internal validity. This pattern controls for maturation, testing, regression, pick, and pretest-posttest interaction, though the mortality threat may keep to exist.

Covariance designs . Sometimes, measures of dependent variables may be influenced by extraneous variables called covariates . Covariates are those variables that are non of central interest to an experimental written report, but should nevertheless exist controlled in an experimental blueprint in order to eliminate their potential effect on the dependent variable and therefore let for a more accurate detection of the furnishings of the independent variables of interest. The experimental designs discussed earlier did non control for such covariates. A covariance design (also chosen a concomitant variable pattern) is a special type of pretest posttest command group blueprint where the pretest mensurate is essentially a measurement of the covariates of interest rather than that of the dependent variables. The design notation is shown in Figure ten.3, where C represents the covariates:

Figure ten.iii. Covariance design

Because the pretest measure is not a measurement of the dependent variable, but rather a covariate, the handling effect is measured every bit the divergence in the posttest scores between the treatment and control groups as:

Due east = (O ane – O ii )

Due to the presence of covariates, the right statistical analysis of this design is a two-group analysis of covariance (ANCOVA). This design has all the advantages of post-test just design, just with internal validity due to the decision-making of covariates. Covariance designs tin also exist extended to pretest-posttest control group design.

Factorial Designs

Two-group designs are inadequate if your inquiry requires manipulation of 2 or more independent variables (treatments). In such cases, you would need four or higher-group designs. Such designs, quite popular in experimental research, are commonly called factorial designs. Each independent variable in this design is called a factor , and each sub-division of a factor is called a level . Factorial designs enable the researcher to examine not simply the private effect of each treatment on the dependent variables (chosen main effects), simply also their articulation outcome (called interaction furnishings).

The most basic factorial design is a 2 x 2 factorial design, which consists of 2 treatments, each with ii levels (such as loftier/low or present/absent-minded). For instance, let'southward say that y'all want to compare the learning outcomes of two dissimilar types of instructional techniques (in-class and online instruction), and you also desire to examine whether these effects vary with the time of instruction (i.v or 3 hours per week). In this example, you have ii factors: instructional type and instructional fourth dimension; each with ii levels (in-class and online for instructional type, and one.5 and 3 hours/calendar week for instructional fourth dimension), as shown in Effigy 8.i. If y'all wish to add a third level of instructional fourth dimension (say half dozen hours/week), then the 2nd gene will consist of iii levels and you will have a 2 x iii factorial design. On the other hand, if you wish to add a third factor such as grouping piece of work (present versus absent), you volition have a ii x two 10 ii factorial design. In this notation, each number represents a factor, and the value of each factor represents the number of levels in that factor.

Figure x.4. 2 x 2 factorial design

Factorial designs can also exist depicted using a design notation, such as that shown on the right panel of Figure 10.iv. R represents random consignment of subjects to handling groups, X represents the handling groups themselves (the subscripts of 10 represents the level of each cistron), and O represent observations of the dependent variable. Discover that the ii 10 2 factorial design will accept four treatment groups, corresponding to the four combinations of the two levels of each factor. Correspondingly, the two ten 3 blueprint will take half-dozen handling groups, and the 2 x 2 x ii pattern volition have eight handling groups. As a rule of thumb, each cell in a factorial design should have a minimum sample size of 20 (this judge is derived from Cohen's ability calculations based on medium issue sizes). And then a 2 x 2 x 2 factorial design requires a minimum total sample size of 160 subjects, with at least 20 subjects in each prison cell. As you tin can see, the cost of information collection tin can increase substantially with more levels or factors in your factorial pattern. Sometimes, due to resource constraints, some cells in such factorial designs may not receive any treatment at all, which are called incomplete factorial designs . Such incomplete designs injure our ability to draw inferences near the incomplete factors.

In a factorial pattern, a main effect is said to exist if the dependent variable shows a pregnant departure between multiple levels of one cistron, at all levels of other factors. No modify in the dependent variable across factor levels is the null example (baseline), from which main effects are evaluated. In the above instance, you may come across a chief effect of instructional blazon, instructional fourth dimension, or both on learning outcomes. An interaction upshot exists when the effect of differences in one factor depends upon the level of a second factor. In our example, if the effect of instructional type on learning outcomes is greater for three hours/calendar week of instructional fourth dimension than for 1.5 hours/week, then we can say that there is an interaction event between instructional type and instructional time on learning outcomes. Note that the presence of interaction effects dominate and brand main effects irrelevant, and it is not meaningful to translate main effects if interaction effects are meaning.

Hybrid Experimental Designs

Hybrid designs are those that are formed past combining features of more established designs. Three such hybrid designs are randomized bocks pattern, Solomon four-group design, and switched replications blueprint.

Randomized block blueprint. This is a variation of the posttest-simply or pretest-posttest control group pattern where the subject field population tin can be grouped into relatively homogeneous subgroups (called blocks ) inside which the experiment is replicated. For example, if you lot want to replicate the same posttest-only blueprint amongst university students and full -time working professionals (ii homogeneous blocks), subjects in both blocks are randomly separate betwixt handling group (receiving the same handling) or control group (come across Effigy 10.5). The purpose of this design is to reduce the "noise" or variance in data that may be attributable to differences between the blocks so that the bodily event of interest tin can exist detected more accurately.

Figure 10.5. Randomized blocks blueprint

Solomon four-group design . In this design, the sample is divided into two treatment groups and two control groups. One treatment group and one control group receive the pretest, and the other ii groups do not. This design represents a combination of posttest-merely and pretest-posttest command group design, and is intended to test for the potential biasing event of pretest measurement on posttest measures that tends to occur in pretest-posttest designs but non in posttest only designs. The design note is shown in Figure 10.half-dozen.

Figure ten.6. Solomon four-group blueprint

Switched replication design . This is a two-group design implemented in two phases with 3 waves of measurement. The treatment group in the first phase serves as the control group in the second phase, and the control group in the first phase becomes the treatment grouping in the second phase, as illustrated in Figure 10.seven. In other words, the original design is repeated or replicated temporally with treatment/control roles switched between the 2 groups. By the terminate of the report, all participants will have received the treatment either during the commencement or the second phase. This design is most feasible in organizational contexts where organizational programs (e.thou., employee training) are implemented in a phased manner or are repeated at regular intervals.

Figure 10.seven. Switched replication design

Quasi-Experimental Designs

Quasi-experimental designs are nigh identical to true experimental designs, merely lacking i key ingredient: random assignment. For case, one entire class department or one organization is used as the handling group, while another section of the same course or a unlike organisation in the same industry is used as the control grouping. This lack of random assignment potentially results in groups that are not-equivalent, such every bit ane grouping possessing greater mastery of a sure content than the other group, say by virtue of having a amend teacher in a previous semester, which introduces the possibility of option bias . Quasi-experimental designs are therefore inferior to true experimental designs in interval validity due to the presence of a variety of selection related threats such every bit selection-maturation threat (the treatment and control groups maturing at different rates), selection-history threat (the treatment and control groups existence differentially impact by extraneous or historical events), option-regression threat (the handling and command groups regressing toward the hateful between pretest and posttest at unlike rates), selection-instrumentation threat (the treatment and control groups responding differently to the measurement), selection-testing (the treatment and control groups responding differently to the pretest), and selection-bloodshed (the handling and control groups demonstrating differential dropout rates). Given these choice threats, it is generally preferable to avoid quasi-experimental designs to the greatest extent possible.

Many true experimental designs can be converted to quasi-experimental designs past omitting random assignment. For instance, the quasi-equivalent version of pretest-posttest control grouping design is chosen nonequivalent groups design (NEGD), equally shown in Effigy ten.eight, with random consignment R replaced by non-equivalent (non-random) assignment N . Likewise, the quasi -experimental version of switched replication design is called not-equivalent switched replication design (encounter Figure ten.9).

Figure ten.eight. NEGD design

Figure x.9. Non-equivalent switched replication design

In add-on, there are quite a few unique non -equivalent designs without corresponding true experimental pattern cousins. Some of the more useful of these designs are discussed next.

Regression-discontinuity (RD) design . This is a not-equivalent pretest-posttest pattern where subjects are assigned to treatment or control group based on a cutoff score on a preprogram measure. For case, patients who are severely ill may exist assigned to a treatment grouping to test the efficacy of a new drug or treatment protocol and those who are mildly ill are assigned to the command group. In another example, students who are lagging behind on standardized exam scores may be selected for a remedial curriculum program intended to ameliorate their functioning, while those who score loftier on such tests are non selected from the remedial programme. The design notation can exist represented equally follows, where C represents the cutoff score:

Effigy 10.10. RD pattern

Because of the use of a cutoff score, information technology is possible that the observed results may exist a function of the cutoff score rather than the treatment, which introduces a new threat to internal validity. However, using the cutoff score besides ensures that express or costly resource are distributed to people who need them the almost rather than randomly across a population, while simultaneously allowing a quasi-experimental handling. The control group scores in the RD design does non serve as a benchmark for comparing handling group scores, given the systematic non-equivalence betwixt the two groups. Rather, if there is no discontinuity betwixt pretest and posttest scores in the control group, simply such a aperture persists in the treatment group, then this aperture is viewed every bit bear witness of the treatment effect.

Proxy pretest design . This pattern, shown in Figure x.xi, looks very similar to the standard NEGD (pretest-posttest) design, with 1 critical divergence: the pretest score is collected subsequently the treatment is administered. A typical application of this blueprint is when a researcher is brought in to test the efficacy of a program (e.g., an educational plan) after the program has already started and pretest information is not available. Under such circumstances, the best option for the researcher is oftentimes to apply a different prerecorded mensurate, such as students' grade point average before the start of the programme, as a proxy for pretest data. A variation of the proxy pretest blueprint is to use subjects' posttest recollection of pretest data, which may be subject to recall bias, only nevertheless may provide a mensurate of perceived gain or change in the dependent variable.

Figure 10.xi. Proxy pretest pattern

Separate pretest-posttest samples design . This design is useful if it is not possible to collect pretest and posttest data from the aforementioned subjects for some reason. Equally shown in Figure x.12, there are iv groups in this design, but two groups come from a single non-equivalent group, while the other two groups come from a different non-equivalent group. For example, you want to test customer satisfaction with a new online service that is implemented in one city but non in some other. In this case, customers in the first metropolis serve as the treatment group and those in the second city constitute the command group. If it is non possible to obtain pretest and posttest measures from the same customers, you can measure customer satisfaction at one point in fourth dimension, implement the new service program, and measure customer satisfaction (with a different prepare of customers) later on the plan is implemented. Customer satisfaction is too measured in the control group at the same times as in the treatment group, merely without the new program implementation. The design is non particularly strong, considering you cannot examine the changes in whatsoever specific client'south satisfaction score before and after the implementation, only y'all can only examine average customer satisfaction scores. Despite the lower internal validity, this pattern may still be a useful mode of collecting quasi-experimental information when pretest and posttest data are non available from the aforementioned subjects.

Figure 10.12. Separate pretest-posttest samples design

Nonequivalent dependent variable (NEDV) pattern . This is a single-group pre-post quasi-experimental blueprint with two outcome measures, where one measure is theoretically expected to be influenced past the handling and the other measure is not. For instance, if you are designing a new calculus curriculum for loftier school students, this curriculum is likely to influence students' posttest calculus scores only not algebra scores. Even so, the posttest algebra scores may still vary due to extraneous factors such as history or maturation. Hence, the pre-postal service algebra scores tin can be used equally a control measure, while that of pre-post calculus can be treated equally the treatment mensurate. The pattern annotation, shown in Effigy 10.13, indicates the unmarried group by a unmarried Northward , followed by pretest O 1 and posttest O 2 for calculus and algebra for the same group of students. This design is weak in internal validity, just its advantage lies in non having to use a separate control group.

An interesting variation of the NEDV design is a blueprint matching NEDV design , which employs multiple outcome variables and a theory that explains how much each variable will be affected past the handling. The researcher can then examine if the theoretical prediction is matched in actual observations. This pattern-matching technique, based on the caste of correspondence betwixt theoretical and observed patterns is a powerful manner of alleviating internal validity concerns in the original NEDV design.

Figure 10.13. NEDV design

Perils of Experimental Research

Experimental research is one of the most difficult of research designs, and should not be taken lightly. This type of enquiry is oft best with a multitude of methodological problems. First, though experimental enquiry requires theories for framing hypotheses for testing, much of current experimental research is atheoretical. Without theories, the hypotheses being tested tend to be advertizement hoc, peradventure casuistic, and meaningless. 2d, many of the measurement instruments used in experimental inquiry are non tested for reliability and validity, and are incomparable across studies. Consequently, results generated using such instruments are likewise unequalled. 3rd, many experimental research employ inappropriate inquiry designs, such as irrelevant dependent variables, no interaction effects, no experimental controls, and non-equivalent stimulus across treatment groups. Findings from such studies tend to lack internal validity and are highly suspect. Fourth, the treatments (tasks) used in experimental inquiry may exist diverse, incomparable, and inconsistent across studies and sometimes inappropriate for the subject population. For instance, undergraduate student subjects are often asked to pretend that they are marketing managers and asked to perform a circuitous upkeep allocation task in which they have no experience or expertise. The use of such inappropriate tasks, introduces new threats to internal validity (i.east., subject'due south performance may be an artifact of the content or difficulty of the task setting), generates findings that are not-interpretable and meaningless, and makes integration of findings beyond studies incommunicable.

The design of proper experimental treatments is a very important chore in experimental design, considering the handling is the raison d'etre of the experimental method, and must never be rushed or neglected. To design an adequate and appropriate chore, researchers should use prevalidated tasks if available, carry treatment manipulation checks to cheque for the capability of such tasks (by debriefing subjects after performing the assigned job), conduct airplane pilot tests (repeatedly, if necessary), and if doubt, using tasks that are simpler and familiar for the respondent sample than tasks that are complex or unfamiliar.

In summary, this chapter introduced key concepts in the experimental design research method and introduced a variety of true experimental and quasi-experimental designs. Although these designs vary widely in internal validity, designs with less internal validity should not be overlooked and may sometimes exist useful nether specific circumstances and empirical contingencies.

zachariassolish.blogspot.com

Source: https://courses.lumenlearning.com/atd-herkimer-researchmethodsforsocialscience/chapter/chapter-10-experimental-research/

Share this post