{"id":157,"date":"2018-12-28T13:31:57","date_gmt":"2018-12-28T13:31:57","guid":{"rendered":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/chapter\/module-3-chapter-2\/"},"modified":"2024-08-19T12:19:11","modified_gmt":"2024-08-19T12:19:11","slug":"module-3-chapter-2","status":"publish","type":"chapter","link":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/chapter\/module-3-chapter-2\/","title":{"rendered":"Module 3 Chapter 2: Quantitative Design Strategies"},"content":{"raw":"Previously, you reviewed different approaches to intervention and evaluation research and learned about a level of evidence (evidence hierarchy) model. These approaches and frameworks relate to how studies are designed to answer different types of research questions. The design strategies differ in the degree to which they address <strong><em>internal validity<\/em><\/strong> concerns\u2014the ability to conclude that changes or differences can be attributed to an intervention rather than to other causes. Designs for quantitative intervention research are the focus of this chapter.\n\nIn this chapter, you learn about:\n<ul>\n \t<li>pre-experimental, quasi-experimental, and experimental designs in relation to strength of evidence and internal validity concerns;<\/li>\n \t<li>how quantitative and single-system study designs are diagrammed;<\/li>\n \t<li>examples where different designs were used in social work intervention or evaluation research.<\/li>\n<\/ul>\n<h2><strong>Addressing Internal Validity Concerns via Study Design Strategies<\/strong><\/h2>\nThe study designs we examine in this chapter differ in terms of their capacity to address specific types of internal validity concerns. As a reminder of what you learned in our previous course, improving internal validity is about increasing investigator confidence that the outcomes observed in an experimental study are as believed, and are due to the experimental variables being studied. In the case of intervention research, the experimental variable being studied is the intervention.\n\nThree general internal validity challenges are important to consider addressing with intervention research.\n<ul>\n \t<li>Were there actually changes that occurred with the intervention (comparing participants\u2019 status pre-intervention to their post-intervention status)?<\/li>\n \t<li>Did observed changes persist over time (comparing participants\u2019 post-intervention status to their status at some later post-intervention follow-up time)?<\/li>\n \t<li>Is the observed change most likely due to the intervention itself?<\/li>\n<\/ul>\nLet\u2019s consider how these three questions might relate to the kind of study design choices that investigators make.\n<h2><strong>Types of Intervention Research Designs<\/strong><\/h2>\nThe intervention study design options can be loosely categorized into three types: pre-experimental, quasi-experimental, and experimental. <strong><em>Pre-experimental designs<\/em><\/strong> do not include comparison (control) groups, only the one group of interest. <strong><em>Quasi-experimental designs<\/em><\/strong> do include comparison (control) groups, but individuals are not randomly assigned to these groups\u2014the groups are created naturally. <strong><em>Experimental designs<\/em><\/strong> include random assignment of study participants to the groups being compared in the study. These three types of designs differ in terms of their attempts to control as many alternative explanation factors as possible\u2014the extent to which they address internal validity concerns.\n<h3><strong>Introducing short-hand for intervention study design diagrams<\/strong><strong>. <\/strong><\/h3>\nIt can become quite cumbersome to describe all the elements of an intervention study\u2019s design in words and complete sentences; sometimes it is easier to use a diagram instead. This type of short-hand quickly communicates important information about a study design. It can become somewhat confusing, like understanding how the diagram of a football play drawn on a chalkboard translates into reality.\n\n<img class=\"aligncenter size-full wp-image-125\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2018\/12\/diagram-of-a-football-play-drawn-on-a-chalkboard.png\" alt=\"diagram of a football play drawn on a chalkboard\" width=\"281\" height=\"210\">\n\nThe first thing to know about this intervention study design short-hand is the symbols used and what each symbol means.\n<ul>\n \t<li><strong>X<\/strong> is used to designate that an intervention is being administered.<\/li>\n \t<li><strong>[X] <\/strong>designates that an \u201cintervention\u201d or event naturally occurred, rather than one imposed by the investigator (for example, a change in policy, a natural disaster, or trauma-inducing crisis event).<\/li>\n \t<li><strong>O<\/strong> is used to designate that an observation is being made (data are collected).<\/li>\n \t<li><strong>Subscript numbers<\/strong> are used to designate which intervention or observation is relevant at that point. For example, X<sub>1<\/sub> might refer to the new intervention that is being tested and X<sub>2<\/sub> might refer to the second condition where the usual intervention is delivered. And, O<sub>1<\/sub> might refer to the first observation period (maybe before the intervention), O<sub>2<\/sub> to the second observation period (maybe after the intervention), and O<sub>3<\/sub> to a third observation period (maybe after a longer-term follow-up period).<\/li>\n \t<li><strong>R <\/strong>is used to designate that individual elements were randomly assigned to the different intervention conditions. (Important reminder: This <strong><em>random assignment<\/em><\/strong> is not about <strong><em>random selection<\/em><\/strong> of a sample to represent a population; it is about using a randomization strategy for placing participants into the different groups in the experimental design).<\/li>\n<\/ul>\n<h3><strong>Costs &amp; Benefits of Various Design Strategies.\u00a0 <\/strong><\/h3>\nBefore we get into discussing the specific strategies that might be adopted for intervention research, it is important to understand that every design has its advantages and its disadvantages. There is no such thing as a single, perfect design to which all intervention research should adhere. Investigators are faced with a set of choices that must be carefully weighed. As we explore their available options, one feature that will become apparent is that some designs are more \u201ccostly\u201d to implement than others. This \u201ccost\u201d term is being used broadly here: it is not simply a matter of dollars, though that is an important, practical consideration.\n<ul>\n \t<li>First, a design may \u201ccost\u201d more in terms of more data collection points. There are significant costs associated with each time investigators have to collect data from participants\u2014time, space, effort, materials, reimbursement or incentive payments, data entry, and more. For this reason, longitudinal studies often are more costly than cross-sectional studies.<\/li>\n \t<li>Second, \u201ccosts\u201d increase with higher numbers of study participants. Greater study numbers \u201ccost\u201d dollars (e.g., advertising expenses, reimbursement or incentive payments, cost of materials used in the study, data entry), but also \u201ccost\u201d more in terms of a greater commitment in time and effort from study staff members, and in terms of greater numbers of persons being exposed to the potential risks associated with the intervention being studied.<\/li>\n \t<li>Third, some longitudinal study designs \u201ccost\u201d more in terms of the potential for higher rates of participant drop-out from the study over time. Each person who quits a study before it is completed increases the amount of wasted resources, since their data are incomplete (and possibly unusable), and that person may need to be replaced at duplicate cost.<\/li>\n<\/ul>\n<img class=\"aligncenter size-full wp-image-126\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/US-money-floating-around-in-the-air.png\" alt=\"US money floating around in the air\" width=\"325\" height=\"206\">\n<h2><strong>Ten Typical Evaluation\/Intervention Study Designs<\/strong><\/h2>\nThis section presents 10 general study designs that typically appear in intervention and evaluation research. These examples are presented in a general order of increasing ability to address internal validity concerns, but this is offset by increasing costs in resources, participant numbers, and participant burden to implement. Many variations on these general designs appear in the literature; these are 10 general strategies.\n<h3><strong>#1: Case study. <\/strong><\/h3>\nOriginal, novel, or new interventions are sometimes delivered under unusual or unique circumstances. In these instances, when very little is known about intervening under those circumstances, knowledge is extended by sharing a <strong><em>case study<\/em><\/strong>. Eventually, when several case studies can be reviewed together, a clearer picture might emerge about intervening around that condition. At that point, theories and interventions can more systematically be tested. Case studies are considered pre-experimental designs.\n\nAn example happened with an adolescent named Jeanna Giese at Milwaukee\u2019s Children\u2019s Hospital of Wisconsin. In 2004, Jeanna was bitten by a bat and three weeks later was diagnosed with full-blown rabies when it was too late to administer a vaccine. At the time, no treatments were known to be successful for rabies once it has developed; the rabies vaccine only works before the disease symptoms develop. Until this case, full-blown rabies was reported to be 100% fatal. The hospital staff implemented an innovative, theory and evidence-informed treatment plan which became known as the \u201cMilwaukee Protocol.\u201d The case study design can be diagrammed the following way, where X represents the \u201cMilwaukee Protocol\u201d intervention and O represents the observed outcomes of the treatment delivered in this case.\n\n<img class=\"aligncenter size-full wp-image-127\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/X-and-O-representing-intervention-and-outcomes.png\" alt=\"X and O representing intervention and outcomes\" width=\"189\" height=\"84\">\n\nJeanna Giese became the first person in the world known to survive full-blown rabies. The intervention team published the case study, and a handful of individuals around the world have been successfully treated with this protocol\u2014rabies continues to be a highly fatal, global concern. The Milwaukee Protocol is considered somewhat controversial, largely because so few others have survived full-blown rabies even with this intervention being administered; some authors argue that this single case was successful because of unique characteristics of the patient, not because of the intervention protocol\u2019s characteristics (Jackson, 2013).\n\nThis argument reflects a major drawback of case studies, which are pre-experimental designs: the unique sample\/small sample size means that individual characteristics or differences can have a powerful influence on the outcomes observed. Thus, the internal validity problem is that the outcomes might be explained by some factors other than the intervention being studied. In addition, with a very small sample size the study results cannot be generalized to the larger population of individuals experiencing the targeted problem\u2014this is an <strong><em>external validity<\/em><\/strong> argument. (We call this an \u201cN of 1\u201d study, where the symbol N refers to the sample size.) The important message here is that case studies are the beginning of a knowledge building trajectory, they are not the end; they inform future research and, possibly, inform practice under circumstances where uncertainty is high with very new problems or solutions. And, just in case you are curious: although some permanent neurological consequences remained, Jeanna Giese completed a college education, was married in 2014, and in 2016 became the mother of twins.\n\n<img class=\"aligncenter size-full wp-image-128\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Illustration-of-bats-flying.png\" alt=\"Illustration of bats flying\" width=\"377\" height=\"268\">\n<h4><strong>#2: Post-intervention only.<\/strong><\/h4>\nLooking a great deal like the case study design is a simple pre-experimental design where the number of individuals providing data after an intervention is greater than the single or very small number in the case study design. For example, a social work continuing education training session (the intervention) might collect data from training participants at the end of the session to see what they learned during the training event. The trainers might ask participants to rate how much they learned about each topic covered in the training (nothing, a little, some, a lot, very much) or they might present participants with a quiz to test their post-training knowledge of content taught in the session. The <strong><em>post-only design<\/em><\/strong> diagram is the same as what we saw with the single case study; the only difference is that the sample size is greater\u2014it includes everyone who completed the evaluation form at the end of the training session rather than just a single case.\n\n<img class=\"aligncenter size-full wp-image-127\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/X-and-O-representing-intervention-and-outcomes.png\" alt=\"X and O representing intervention and outcomes\" width=\"189\" height=\"84\">\n\nThe post-intervention only design is cross-sectional in nature (only one data collection point with each participant). This design strategy is extremely vulnerable to internal validity threats. The investigator does not know if the group\u2019s knowledge changed compared to before the training session: participants quizzed on knowledge may already have known the material before the training; or, a perception of how much they learned may not accurately depict how much they learned. The study design does not inform the investigators if the participants\u2019 learning persisted over time after completing the training. The investigators also do not have a high level of confidence that the training session was the most likely cause of any changes observed\u2014they cannot rule out other possible explanations.\n\nIn response to the internal validity threat concerning ability to detect change with an intervention, an investigator might ask study participants to compare themselves before and after the intervention took place. That would still be a simple post- only design because there is only one time point for data collection: post-intervention. This kind of retrospective approach is vulnerable to bias because it relies on an individual\u2019s ability to accurately recall the past and make a valid comparison to the present, a comparison that hopefully is not influenced by their present state-of-mind. It helps to remember what you learned from SWK 3401 about the unreliability of the information individuals remember and how memories become influenced by later information and experiences.\n\n<img class=\"aligncenter size-full wp-image-129\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/image-of-a-humanoid-figure-standing-on-the-word-remember-while-scratching-its-head.png\" alt=\"image of a humanoid figure standing on the word remember while scratching its head\" width=\"235\" height=\"208\">\n<h4><strong>#3: Pre-\/Post- Intervention Comparison. <\/strong><\/h4>\nA wiser choice in terms of internal validity would be to directly compare data collected at the two points in time: pre-intervention and post-intervention. This <strong><em>pre-\/post-design<\/em><\/strong> remains a pre-experimental design because it lacks a comparison (control) group. Because the data are collected from the same individuals at two time points, this strategy is considered longitudinal in nature. This type of pre-\/post-intervention design allows us to directly identify change where observed differences on a specific outcome variable might be attributed to the intervention. A simple pre-\/post- study design could be diagrammed like this:\n\n<img class=\"aligncenter size-full wp-image-130\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes.png\" alt=\"O1 X and O2 representing intervention and outcomes\" width=\"319\" height=\"86\">\n\nHere we see an intervention X (perhaps our social work in-service training example), where data were still collected after the intervention (perhaps a knowledge and skills quiz). However, the investigators also collected the same information prior to the intervention. This allowed them to compare data for the two observation periods, pre- and post- intervention. See the arrow added to the diagram that shows this comparison:\n\n<img class=\"aligncenter size-full wp-image-131\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-with-O1-and-O2-connected-with-an-arrow.png\" alt=\"O1 X and O2 representing intervention and outcomes with O1 and O2 connected with an arrow\" width=\"305\" height=\"87\">\n\nWhile this pre-\/post- intervention design is stronger than a post-only design, it also is a bit more \u201ccostly\u201d to implement since there is an added data collection point. It imposes an additional burden on participants, and in some situations, it simply might not be possible to collect that pre-intervention data. This design strategy still suffers from the other two internal validity concerns: we do not know if any observed changes persisted over time, and we do not have the highest level of confidence that the changes observed can be attributed to the intervention itself.\n\n<img class=\"aligncenter size-full wp-image-132\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/The-words-Before-After-painted-on-blacktop.png\" alt=\"The words Before &amp; After painted on blacktop\" width=\"279\" height=\"185\">\n<h4><strong>#4: Pre-\/Post-\/Follow-Up Comparison. <\/strong><\/h4>\nInvestigators can improve on the pre-\/post- study design by adding a follow-up observation. This allows them to determine whether any changes observed between the pre- and post- conditions persisted or disappeared over time. While investigators may be delighted to observe a meaningful change between the pre- and post- intervention periods, if these changes do not last over time, then intervention efforts and resources may have been wasted. This <strong><em>pre-\/post-\/follow-up design<\/em><\/strong> remains in the pre-experimental category, and would be diagrammed like this:\n\n<img class=\"aligncenter size-full wp-image-133\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-O2-and-O3-representing-intervention-and-outcomes.png\" alt=\"O1 X O2 and O3 representing intervention and outcomes\" width=\"307\" height=\"87\">\n\nHere we see the intervention (X), with pre-intervention data, post-intervention data, and follow-up data being collected (O<sub>1<\/sub>, O<sub>2<\/sub>, and O<sub>3<\/sub>). As an example, Wade-Mdivanian, Anderson-Butcher, Newman, and Ruderman (2016) explored the impact of delivering a preventive intervention about alcohol, tobacco and other drugs in the context of a positive youth development program called Youth to Youth International. The outcome variables of interest were the youth leaders\u2019 knowledge, attitudes, self-efficacy, and leadership before the program, at the program\u2019s conclusion, and six months after completing the program. The authors concluded that positive changes in knowledge and self-efficacy outcomes were observed in the pre-\/post- comparison, though no significant change was observed in attitudes about alcohol, tobacco and other drugs; these differences persisted at the six-month follow-up. See the pre-\/post-\/follow-up design diagrammed with comparison arrows added:\n\n<img class=\"aligncenter size-full wp-image-134\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-O2-and-O3-representing-intervention-and-outcomes-with-O1-and-O2-connected-with-an-arrow-and-O2-and-O3-connected-with-an-arrow.png\" alt=\"O1 X O2 and O3 representing intervention and outcomes with O1 and O2 connected with an arrow and O2 and O3 connected with an arrow\" width=\"305\" height=\"88\">\n\nThis design resolved two of the three internal validity challenges, although at an increased cost of time, effort, and possibly other resources with the added data point. But an investigator would still lack confidence that observed changes were due to the intervention itself. Let\u2019s look at some design strategies that focus on that particular challenge.\n<h4><strong>#5: Comparison Groups.<\/strong><\/h4>\nIn our prior course we learned how to compare groups that differed on some characteristic, like gender for example. Comparison groups in intervention research allow us to compare groups where the difference lies in which intervention condition each received. By providing an experimental intervention to one group and not to the other group, investigator confidence increases about observed changes being related to the intervention. You may have heard this second group described as a <strong><em>control group<\/em><\/strong>. They are introduced into the study design to provide a benchmark for comparison with the status of the intervention group. The simplest form of a comparison group design, which is a quasi-experimental type of design, a <strong><em>post-only group design<\/em><\/strong> can be diagrammed as follows:\n\n<img class=\"aligncenter wp-image-135 size-full\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/X-and-O-representing-intervention-and-outcomes-and-a-second-row-with-just-an-O.png\" alt=\"the top row shows the group who received the training intervention (X) and the outcome was measured (O), and the bottom row shows the group without the training intervention (no X)\" width=\"237\" height=\"104\">\n\nConsider the possibility in our earlier example of evaluating a social work training intervention that the team decided to expand their post only design to include collecting data from a group of social workers who are going to get their training next month. On the same day that the data were collected from the trained group, the team collected data from the untrained social workers, as well. This situation is a post-only design where the top row shows the group who received the training intervention (X) and the outcome was measured (O), and the bottom row shows the group without the training intervention (no X was applied) also being measured at the same point in time as the intervention group (O). This remains a cross-sectional study because each individual was only observed once. The following diagram shows the arrow where investigators compared the two groups on the outcome variables (knowledge and skills quiz scores, using our training example). If the training intervention is responsible for the outcome, the team would see a significant difference in the outcome data when comparing the two groups; hopefully in the direction of the trained group having better scores.\n\n<img class=\"aligncenter size-full wp-image-136\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2018-12-28-at-9.10.18-AM.png\" alt=\"the top row shows the group who received the training intervention (X) and the outcome was measured (O), and the bottom row shows the group without the training intervention (no X) also being measured at the same point in time as the intervention group (O). \" width=\"241\" height=\"106\">\n\nWhile this design has helped boost investigator confidence that the outcomes observed with the intervention group are likely due to the intervention itself, this post-only design \u201ccosts\u201d more than a post-only single group (pre-experimental) study design. This post-only group design still suffers from a significant concern: how would an investigator know if the differences between the two groups appeared only after the intervention or could the differences always have existed, with or without the intervention? With this design, that possibility cannot be ignored. An investigator can only hope that the two groups were equivalent prior to the intervention. Two internal validity questions remain unanswered: did the outcome scores actually demonstrate a change resulting from intervention, and did any observed changes persist over time. Let\u2019s consider some other combination strategies that might help, even though they may \u201ccost\u201d considerably more to implement.\n<h4><strong>#6: Comparison Group Pre-\/Post- Design<\/strong>.<\/h4>\nA giant leap forward in managing internal validity concerns comes from combining the strategy of having both an intervention and a comparison group (which makes it quasi-experimental) with the strategy of collecting data both before and after the intervention (which makes it longitudinal). Now investigators are able to address more of the major validity concerns. This <em>comparison group pre-\/post- design <\/em>is diagrammed as follows:\n\n<img class=\"aligncenter size-full wp-image-137\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X.png\" alt=\"O1 X and O2 representing intervention and outcomes and a second row with O1 and O2 but no X\" width=\"322\" height=\"130\">\n\nWhat the investigators have done here is collect data from everyone in their study, both groups, prior to delivering the intervention to one group and not to the other group (control group). Then, they collected the same information at the post-intervention time period for everyone in their study, both groups. The power in this design can be seen in the following diagrams that include arrows for the kinds of longitudinal pre-\/post- comparisons that can be assessed, measuring change.\n\n<img class=\"aligncenter size-full wp-image-138\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X-with-arrows-connecting-O1-and-O2-on-each-row.png\" alt=\"O1 X and O2 representing intervention and outcomes and a second row with O1 and O2 but no X with arrows connecting O1 and O2 on each row\" width=\"323\" height=\"133\">\n\nThis is a pre-\/post- comparison, indicating if change occurred with the intervention group after the intervention. Investigators would hope the answer is \u201cyes\u201d if it is believed that the intervention should make a difference. Similarly, the investigators could compare the non-intervention group at the two observation points (the lower arrow). Hopefully, there would be no significant change without the intervention.\n\nYou might be wondering, \u201cWhy would there be change in the no intervention group when nothing has been done to make them change?\u201d Actually, with the passage of time, several possible explanatory events or processes could account for change.\n<ul>\n \t<li>The first is simple <strong><em>maturation<\/em><\/strong>. Particularly with young children, developmental change happens over relatively short periods of time even without intervention.<\/li>\n \t<li>Similarly, the passage of time might account for symptom improvement even without intervention. Consider, for example, the old adage that, \u201cif you treat a common cold it will take 7 to 10 days to get better; if you don\u2019t treat it, it will take a week to a week-and-a-half.\u201d Change without intervention is called <strong><em>spontaneous or natural change<\/em><\/strong>. Either way, there is change\u2014with or without intervention.<\/li>\n \t<li>Third, given time, individuals in the no intervention group might seek out and receive other interventions not part of the study that also can produce change. This could be as simple as getting help and support from friends or family members; or, seeking help and advice on the internet; or, enrolling in other informal or formal treatment programs at the same time.<\/li>\n<\/ul>\nThe benefit of combining the comparison groups and pre-\/post- designs continues to emerge as we examine the next diagram showing comparisons an investigator can consider:\n\n<img class=\"aligncenter size-full wp-image-139\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X-an-arrow-connects-O2-on-each-row-to-represent-a-comparison.png\" alt=\"\" width=\"273\" height=\"124\">\n\nThis group comparison of post-intervention results indicates whether there is a difference in outcomes between people who received the intervention and people who did not. Again, investigators would hope the answer is \u201cyes,\u201d and that the observed difference favors the intervention group. But with this design, an investigator can go even further in ruling out another possible explanation for outcome differences. Consider the power of this comparison:\n\n<img class=\"aligncenter size-full wp-image-140\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X-an-arrow-connects-O1-on-each-row-to-represent-a-comparison.png\" alt=\"\" width=\"280\" height=\"127\">\n\nBy comparing the two groups BEFORE the intervention took place, an investigator can hopefully rule out the possibility that post-intervention group differences were actually a reflection of pre-existing differences; differences that existed prior to the intervention. In this case, the investigator would hope to observe no significant differences in this comparison. This \u201cno differences\u201d result would boost confidence in the conclusion that any observed post-intervention differences were a function of the intervention itself (since there were no pre-existing differences). Not being able to rule out this possibility was one limitation of the post-only group comparison strategy discussed earlier.\n<h4><em>A Note About Non-Treatment, Placebo, and Treatment as Usual Comparison Groups<\/em>.<\/h4>\nWhat we have diagrammed above is a situation where the second group received no treatment at all. This, however, is problematic in three ways. First, the ethics of intentionally not serving individuals who are seeking help with a social work problem is concerning. Second, the scientific integrity of such studies tends to suffer because non-treatment \u201ccontrol\u201d groups tend to have high rates of study drop out (attrition) as those participants seek help elsewhere. Third, the results of these studies are coming under great scrutiny as the world of behavioral science has come to realize that any treatment is likely to be better than no treatment\u2014thus, study results where the tested intervention is significantly positive may be grossly over-interpreted. Compared to other treatments, what appears to be a fantastic innovation may be no better.\n\nMedical studies often include a comparison group who receives a \u201cfake\u201d or neutral form of a medication or other intervention. In medicine, a placebo is an inert \u201ctreatment\u201d with a substance that has no likely known effect on the condition being studies, such as a pill made of sugar, for example. The approximate equivalent in behavior al science is an intervention where clients are provided only with basic, factual information about the condition; nothing too empowering. In theory, neither the placebo medication nor the simple educational materials are expected to promote significant change. This is a slight variation on the <strong><em>non-treatment control<\/em><\/strong> condition. However, over 20 years of research provided evidence of a <strong><em>placebo effect<\/em><\/strong> that cannot be discounted. In one systematic review and meta-analysis study (Howick et al, 2016) the size of effect associated with placebos were no different or even larger than treatment effects. This placebo effect is, most likely, associated with the psychological principles of motivation and expectancies\u2014expecting something to work has a powerful impact on behavior and outcomes, particularly with regard to symptoms of nausea and pain (Cherry, 2018). Of course, this means that participants receiving the placebo believe they are (or could be) receiving the therapeutic intervention. Also interesting to note is that participants sometimes report negative side-effects with exposure to the placebo treatment.\n\nIn medication trials, the introduction of a placebo may allow investigators to impose a double-blind structure to the study. A <strong><em>double-blind study<\/em><\/strong> is one where neither the patient\/client nor the practitioner\/clinician knows if the person is receiving the test medication or the placebo condition. The double-blind structure is imposed as means of reducing practitioner bias in the study results from either patient or practitioner\/clinician beliefs about the experimental medication. However, it is difficult to disguise behavioral interventions from practitioners\u2014it is not as simple as creating a real-looking pill or making distilled water look like medicine.\n\nMore popular in intervention science today is the use of a <strong><em>treatment as usual condition (TAU)<\/em><\/strong>. In a TAU study, the comparison group receives the same treatment that would have been provided without the study being conducted. This resolves the ethical concerns of intentionally denying care to someone seeking help simply to fulfill demands of a research design. It also helps resolve the study integrity concerns mentioned earlier regarding non-treatment studies. But this design looks a little bit different from the diagram of a comparison group design. You will notice that there is a second intervention \u201cX\u201d symbol in the diagram and that the new X has a subscript notation (TAU) so you can tell the two intervention groups apart, X and X<sub>TAU<\/sub>.\n\n<img class=\"aligncenter size-full wp-image-141\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-and-with-an-X-that-represnts-treatment-as-usual-condition.png\" alt=\"O1 X and O2 representing intervention and outcomes and a second row with O1 and O2 and with an X that represnts treatment as usual condition\" width=\"325\" height=\"132\">\n<h4><strong>#7: Single-System Design<\/strong>.<\/h4>\nA commonly applied means of evaluating practice is to employ a quasi-experimental single-system design. This approach uniquely combines aspect of the case study with aspects of the pre-experimental pre-\/post-design. Like the case study, the data are collected for one single case at a time\u2014whether the element or unit of study is an individual, couple, family, or larger group, the data represent the behavior of that element over time. In that sense, the single-system design is longitudinal\u2014repeated measurements are drawn for the same element each time. It is a quasi-experimental design in that conditions are systematically varied, and outcomes measured for each variation. This approach to evaluating practice is often referred to as a single-subject design. However, the fact that the \u201csubject\u201d might be a larger system is lost in that label, hence a preference for the single-\u201csystem\u201d design label.\n\nThe single-system design is sufficiently unique in implementation that an entirely different notation system is applied. Instead of the previous Xs and Os, we will be using A\u2019s and B\u2019s (even C\u2019s and D\u2019s). The first major distinction is that instead of a pre-intervention measurement (what we called O<sub>1<\/sub>) we need a pre-intervention baseline. By definition, a line is the distance between two or more points, thus in order to be a baseline measurement, at least two and preferably at least 7 pre-intervention measurement points are utilized. For example, if the target of intervention with a family is that they spend more activity time together, a practitioner might have them maintain a daily calendar with the number of minutes spent in activity time is recorded for each of 7 days. This would be used as a baseline indication of the family\u2019s behavioral norm. Thus, since there are 7 measurement points replacing the single pre-intervention observation (O<sub>1<\/sub>), we designate this as A<sub>1<\/sub>, A<sub>2<\/sub>, A<sub>3<\/sub> and so forth to A<sub>7<\/sub> for days 1 through 7 during the baseline week--this might extend to A<sub>30<\/sub> for a month of baseline recording.\n\nNext, we continue to measure the target behavior during the period of intervention. Because the intervention period is different from the baseline period, we use the letter B to indicate the change. This corresponds to the single point in our group designs where we used the letter X to designate the intervention. Typically, in a pre-\/post- intervention study no data are collected during the intervention, X, which is another difference in single-system designs. Let\u2019s say that the social worker\u2019s intervention was to text the family members an assigned activity for each of 10 days, and they continued to record the number of minutes when they were engaged in family activity time each day. Because it is a series of measurement points, we use B<sub>1<\/sub>, B<sub>2<\/sub>, B<sub>3<\/sub> and so forth for the duration of the intervention, which is B<sub>10<\/sub> in this case. The next step is to remove the daily assignment text messages, giving them just one menu on the first of the next 7 days, from which the family is expected to pick an activity and continue to record time spent in family activity time each day. This would be a different form of intervention from the \u201cB\u201d condition, so it becomes C<sub>1<\/sub>, C<sub>2<\/sub>, C<sub>3<\/sub>, and so forth to the end, C<sub>7<\/sub>. Finally, the social worker no longer sends any cues, and the family simply records their daily family activity time for another week. This \u201cno intervention\u201d condition is the same as what happened at baseline before there was any intervention. So, the notation reverts back to A\u2019s, but this time it is A<sub>8<\/sub>, A<sub>9<\/sub>, A<sub>10<\/sub>, and so forth to the end of the observation period, A<sub>14<\/sub>. For this reason, single system design studies are often referred to as \u201cABA\u201d designs (initial baseline, an intervention line, and a post-intervention line) or, in our example an \"ABCA\" design since there was a second intervention after the \"B\" condition. This single-system design notation is different from the notation using Xs and Os because data are collected multiple times during each phase, rather than at each single point during pre-intervention, intervention, and post-intervention.\n\nThe data could be presented in a table format, but a visual graph depicts the trends in a more concise, communicative manner. What the practitioner is aiming to do with the family is to look at the pattern of behavior in as objective manner as possible, under the different manipulated conditions. Here is a graphical representation from one hypothetical family. As you can see, there is natural variation in the family\u2019s behavior that would be missed if we simply used a single weekly value for the pre- and post-intervention periods instead.\n\n<img class=\"aligncenter size-full wp-image-142\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Minutes-Daily-Familty-Activity-Time-Chart.png\" alt=\"Minutes Daily family Activity Time Chart\" width=\"874\" height=\"470\">\n\nTogether, the practitioner and the client family can discuss what they see in the data. For example, what makes Wednesday family activity time particularly difficult to implement and does it matter given the what happens on the other days? How does it feel as a family to have the greater activity days, and is that rewarding to them? What happens when the social worker is no longer prompting them to engage in activities and how can they sustain their gains over time without outside intervention? What new skills did they learn that support sustainability? What will be their cue that a \u201cbooster\u201d might be necessary? What the data allow is a clear evaluation of the intervention with this family, which is the main purpose of practice evaluation using the single-system design approach.\n\n<img class=\"aligncenter size-full wp-image-143\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/The-words-Before-During-and-After-painted-on-blacktop.png\" alt=\"The words Before During and After painted on blacktop\" width=\"289\" height=\"191\">\n<h4><strong>#8: Random Control Trial (RCT) Pre-\/Post- Design.<\/strong><\/h4>\nThe major difference between the comparison group pre-\/post- design just discussed and the <strong><em>random control trial (RCT)<\/em><\/strong> <em>with pre-\/post- design<\/em> is that investigators do not have to rely so much on hoping that the two comparison groups were initially equivalent: they randomly assign study participants to the two groups as an attempt to ensure that this is true. There is still no guarantee of initial group equivalence; this still needs to be assessed. But the investigators are less vulnerable to the bad luck of having the two comparison groups being initially different. This is what the RCT design looks like in a diagram:\n\n<img class=\"size-full wp-image-144 aligncenter\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/R-O1-X-and-O2-representing-intervention-a.png\" alt=\"R O1 X and O2 representing intervention and outcomes and a second row with O1 and O2 but no X an arrow connects O2 on each row to represent a comparison and R is connevted via an arrow to indicate randomization\" width=\"336\" height=\"146\">\n\nMany authors of research textbooks describe the RCT as the \u201cgold standard\u201d of intervention research design because it addresses so many possible internal validity concerns. It is, indeed, a powerful experimental design. However, it is important to recognize that this type of design comes at a relatively high \u201ccost\u201d compared to some of the others we have discussed. Because there are two comparison groups being compared, there are more study participant costs involved than the single group designs. Because there are at least two points in time when data are collected, there are more data collection and participant retention costs involved than the single time point post-only designs. And random assignment to experimental conditions or groups is not always feasible in real-world intervention situations. For example, it is difficult to apply random assignment to conditions where placement is determined by court order, making it difficult to use an RCT design for comparing a jail diversion program (experimental intervention) with incarceration (treatment as usual control group). For these practical reasons, programs often settle on group comparison designs without random assignment in their evaluation efforts.\n\n<img class=\"aligncenter size-full wp-image-145\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Pile-of-gold-bars.png\" alt=\"Pile of gold bars\" width=\"195\" height=\"147\">\n\n<div class=\"textbox textbox--sidebar textbox--key-takeaways\" style=\"min-width: 100%\"><header class=\"textbox__header\">\n<p class=\"textbox__title\">Back to Basics: Interpreting Statistics<\/p>\n\n<\/header>\n<div class=\"textbox__content\">\n\nRemembering how to interpret statistics in a research report. The investigators reported that for the intervention group, the mean reduction in AUDIT-12 scores was significantly greater for the SBI intervention group than for the TAU group on a one-way Anova: M=18.83 and 13.52 respectively, <em>F<\/em>(1,148)=6.34, <em>p<\/em>&lt;.01. This statement indicates the following:\n<ol>\n \t<li>The mean (M) for the innovative SBI intervention group =18.83<\/li>\n \t<li>The mean (<em>M<\/em>) for the treatment as usual (TAU) group =13.52.<\/li>\n \t<li>The computed <em>F <\/em>statistic in the analysis of variance (Anova) =6.34.<\/li>\n \t<li>Degrees of freedom for the Anova were 1 (2 groups -1) and 148 (149 cases - 1): <em>df<\/em>=(1, 148).<\/li>\n \t<li>The <em>F<\/em>-distribution test statistic at those degrees of freedom is significant at a level less than the criterion <em>p<\/em>&lt;.05.<\/li>\n \t<li>We reject the null hypothesis of no difference between the groups, concluding that a significant difference exists.<\/li>\n<\/ol>\n<\/div>\n<\/div>\n\nA slight variant on this study design was used to compare a screening and brief intervention (SBI) for substance misuse problems to the usual (TAU) condition among 729 women preparing for release from jail and measured again during early reentry to community living (Begun, Rose, &amp; LeBel, 2011). One-third of the women who had positive screen results for a potential substance use disorder were randomly assigned to the treatment as usual condition (X<sub>TAU<\/sub>), and two-thirds to the SBI experimental intervention condition (X) as diagrammed below. The investigators followed 149 women three months into post-release community reentry ( follow-up observation), and found that women receiving the innovative SBI intervention had better outcomes for drinking and drug use during early reentry (three months post-release): the mean difference in scores on the AUDIT-12 screening instrument prior to jail compared to reentry after release was more than 5 points (see Back to Basics Box for more details). The <em>random controlled trial with pre\/post\/follow-up study design<\/em> looked like this:\n\n<img class=\"aligncenter size-full wp-image-146\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/R-O1-X-O2-and-O3-representing-intervention-and-outcomes-and-a-second-row-with-R-O1-O2-and-O3-and-with-an-X-that-represnts-treatment-as-usual-condition.png\" alt=\"R O1 X O2 and O3 representing intervention and outcomes and a second row with R O1 O2 and O3 and with an X that represnts treatment as usual condition\" width=\"393\" height=\"130\">\n<h4><strong>#9: Random Control Trial (RCT) Comparing Two (or More) Interventions. <\/strong><\/h4>\nThe only difference between the design just discussed and the random control trial that compares two interventions is the indicator associated with the intervention symbol, X. The X<sub>1<\/sub> and X<sub>2<\/sub> refer to two different interventions, the same way the X<sub>1<\/sub> and X<sub>TAU<\/sub> reflected two different intervention conditions above. This kind of design is used to determine which of two interventions has greater effectiveness, especially if there is a considerable difference in cost of delivering them. The results can help with a cost-benefit comparison which program administrators and policy decision-makers use to help make decisions about which to fund. It is possible to compare more than two interventions by simply adding additional lines for each group (X<sub>3<\/sub>, X<sub>4<\/sub>, and so on). Each group added, however, also adds considerably to the \u201ccosts\u201d of conducting the study.\n\n<img class=\"aligncenter size-full wp-image-147\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/R-O1-X1-and-O2-representing-different-interventions.png\" alt=\"R O1 X1 and O2 representing different interventions. In the second tow, X is labelled X2 to represented a different intervention\" width=\"321\" height=\"129\">\n\nThe COMBINE Project was an historic example where multiple intervention approaches were compared for their effectiveness in treating alcohol dependence (see NIAAA Overview of COMBINE at <a href=\"https:\/\/pubs.niaaa.nih.gov\/publications\/combine\/overview.htm\">https:\/\/pubs.niaaa.nih.gov\/publications\/combine\/overview.htm<\/a>).\n\nTwo medications were being compared (acamprosate and naltrexone) along with medical management counseling for adherence to the medication protocol (MM), and medication with MM was compared to medication with MM plus a cognitive behavioral psychotherapy intervention (CBI). Clients were randomly assigned to 9 groups with approximately 153 clients per group:\n<p style=\"padding-left: 60px\">X<sub>1<\/sub>-acamprosate placebo + naltrexone placebo + MM (no CBI)<\/p>\n<p style=\"padding-left: 60px\">X<sub>2<\/sub>-acamprosate + naltrexone placebo + MM (no CBI)<\/p>\n<p style=\"padding-left: 60px\">X<sub>3<\/sub>-naltrexone + acamprosate placebo + MM (no CBI)<\/p>\n<p style=\"padding-left: 60px\">X<sub>4<\/sub>-acamprosate + naltrexone + MM (no CBI)<\/p>\n<p style=\"padding-left: 60px\">X<sub>5<\/sub>- acamprosate placebo + naltrexone placebo + MM + CBI<\/p>\n<p style=\"padding-left: 60px\">X<sub>6<\/sub>- acamprosate + naltrexone placebo + MM + CBI<\/p>\n<p style=\"padding-left: 60px\">X<sub>7<\/sub>- naltrexone + acamprosate placebo + MM + CBI<\/p>\n<p style=\"padding-left: 60px\">X<sub>8<\/sub>- acamprosate + naltrexone + MM + CBI<\/p>\n<p style=\"padding-left: 60px\">X<sub>9<\/sub>-no pills or MM, CBI only<\/p>\nThe COMBINE Project was an extremely \u201ccostly\u201d study to conduct given the number of study participants needed to meet the study\u2019s design demands--one reason why it was a collaboration across multiple study sites. The results included participants in all 9 groups showing at least some improvement as measured by a reduction in drinking--intervention is almost always an advantage over no intervention. The poorest results were observed for the last group (X<sub>9<\/sub>), those receiving the specialty cognitive behavioral intervention (CBI) alone, with no medication or placebo medication. Surprising was that best outcomes were observed for the group receiving CBI with both placebo medications and medication management (MM) counseling (X<sub>5<\/sub>)! The other group with best outcomes was the group receiving naltrexone with MM counseling but no CBI (X<sub>3<\/sub>). The investigative team concluded that pharmacotherapy combined with medication counseling can yield clinically significant alcohol treatment outcomes, and this can be delivered in primary care settings where specialty alcohol treatment is unavailable (Pettinati, Anton, &amp; Willenbring, 2006). They were surprised, also, by the lack of observed advantage to using the medication acamprosate because it had so much evidence support from studies conducted in European countries. This map of the design does not show the multiple follow-up observations conducted in the study.\n\n<img class=\"aligncenter size-full wp-image-148\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/R-01-X1-and-O2-in-each-row-with-X1-being-incremented-to-show-different-interventions.png\" alt=\"R 01 X1 and O2 in each row with X1 being incremented to show different interventions\" width=\"324\" height=\"248\">\n<h4><strong>#10: Comparison Group with a Delayed Intervention Condition. <\/strong><\/h4>\nOne way of overcoming the ethical dilemma of an intervention\/no intervention study design is to offer the intervention condition later for the second group. Not only does this more evenly distribute risks and benefits potentially experienced across both groups, it also makes sense scientifically and practically because it allows another set of comparison conditions to be longitudinally analyzed for a relatively smaller additional \u201ccost.\u201d Here is what this design strategy might look like using our study design notation:\n\n<img class=\"aligncenter size-full wp-image-149\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-O2-and-O3-representing-intervention-and-outcomes2.png\" alt=\"O1 X O2 and O3 representing intervention and outcomes. In row 1 X is between O1 and O2 and in row 2 it is between O2 and O3\" width=\"384\" height=\"132\">\n\nLook at all the information available through this design:\n<ul>\n \t<li>First is a simple pre-\/post- design (O<sub>1<\/sub> to O<sub>2<\/sub> with X), where investigators hope to see a difference;\n<img class=\"aligncenter size-full wp-image-150\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/1.png\" alt=\"\u2022 First is a simple pre-\/post- design (O1 to O2 with X), where investigators hope to see a difference; \" width=\"378\" height=\"110\"><\/li>\n \t<li>Second is the control group pre-\/post- comparison (O<sub>1<\/sub> to O<sub>2<\/sub> without intervention X), where investigators hope for no difference, or at least significantly less difference than for the group with the intervention early on;\n<img class=\"aligncenter size-full wp-image-151\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/2.png\" alt=\"\u2022 Second is the control group pre-\/post- comparison (O1 to O2 without intervention X), where investigators hope for no difference, or at least significantly less difference than for the group with the intervention early on; \" width=\"375\" height=\"127\"><\/li>\n \t<li>Third is the comparison of the group with the intervention (X) to the group without the intervention, after the intervention was delivered (O<sub>2<\/sub> for each group), where the investigators hope to see a significant difference between the two groups, and that the difference favors the group who received the intervention (X);\n<img class=\"aligncenter size-full wp-image-152\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/3.png\" alt=\"\u2022 Third is the comparison of the group with the intervention (X) to the group without the intervention, after the intervention was delivered (O2 for each group), where the investigators hope to see a significant difference between the two groups, and that the difference favors the group who received the intervention (X); \" width=\"374\" height=\"105\"><\/li>\n \t<li>Fourth is the post-\/follow-up comparison for the group with the intervention (O<sub>2<\/sub> to O<sub>3<\/sub>), where investigators hope that the changes seen on O<sub>2<\/sub> persisted at O<sub>3<\/sub> (or are even more improved, not that they declined over time;\n<img class=\"aligncenter size-full wp-image-153\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/4.png\" alt=\"\u2022 Fourth is the post-\/follow-up comparison for the group with the intervention (O2 to O3), where investigators hope that the changes seen on O2 persisted at O3 (or are even more improved, not that they declined over time;\" width=\"375\" height=\"104\"><\/li>\n \t<li>Fifth is the ability to replicate the results of the first group receiving the intervention with the results for the second group to receive the intervention (O<sub>2<\/sub> for the first group with O<sub>3<\/sub> for the second group\u2014actually, change from O<sub>1<\/sub> to O<sub>2<\/sub> for the first group and change from O<sub>1<\/sub> to O<sub>3<\/sub> for the second group). Ideally, the investigators would see similar results for the two groups\u2014those who received the intervention early on and those who received it a bit later;\n<img class=\"aligncenter size-full wp-image-154\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/5.png\" alt=\"\u2022 Fifth is the ability to replicate the results of the first group receiving the intervention with the results for the second group to receive the intervention (O2 for the first group with O3 for the second group\u2014actually, change from O1 to O2 for the first group and change from O1 to O3 for the second group). Ideally, the investigators would see similar results for the two groups\u2014those who received the intervention early on and those who received it a bit later\" width=\"383\" height=\"126\"><\/li>\n \t<li>Sixth is the ability to test the assumption that the two groups were similar prior to the study beginning\u2014that the differences observed between them were related to the intervention and not pre-existing group differences (O<sub>1<\/sub> for both groups), as well as differences immediately after intervention and between follow-up and immediately post-intervention.\n<img class=\"aligncenter size-full wp-image-155\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/6.png\" alt=\"\u2022 Sixth is the ability to test the assumption that the two groups were similar prior to the study beginning\u2014that the differences observed between them were related to the intervention and not pre-existing group differences (O1 for both groups), as well as differences immediately after intervention and between follow-up and immediately post-intervention\" width=\"372\" height=\"103\"><\/li>\n<\/ul>\nThis study design was used to examine the effectiveness of equine-assisted therapy for persons who have dementia (Dabelko-Schoeny, et al., 2014). The study engaged 16 participants with Alzheimer\u2019s Disease in activities with four horses (grooming, observing, interacting, leading, and photographing). Study participants were randomly assigned to two groups: one received the intervention immediately the other did not; however, this latter group received the intervention later, after the intervention was withdrawn from the first group. When not receiving the equine-assisted therapy, participants received the usual services. Thus, the investigators were able to make multiple types of comparisons and ensured that all participants had the opportunity to experience the novel intervention. The team reported that the equine-assisted therapy was significantly associated with lower rates of problematic\/disruptive behavior being exhibited by the participants in their nursing homes and higher levels of \u201cgood stress\u201d (sense of exhilaration or accomplishment) as measured by salivary cortisol levels.\n\nMajor disadvantages of this study design:\n<ul>\n \t<li>it may be that social work interventions are best delivered right away, when the need arises, and that their impact is diminished when delivered after a significant delay;<\/li>\n \t<li>the design is costlier to implement than either design alone might be, because it requires more participants and that they be retained over a longer period of time.<\/li>\n<\/ul>\n<img class=\"aligncenter size-full wp-image-156\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/large-collection-of-clocks-with-pastel-colored-faces.png\" alt=\"\" width=\"202\" height=\"135\">\n\n<h2><strong>Chapter Conclusion<\/strong><\/h2>\nAs you can see, social work professionals and investigators have numerous options available to them in planning how to study the impact of interventions and evaluate their practices, programs, and policies. Each option has pros and cons, advantages and disadvantages, costs and benefits that need to be weighed in making the study design decisions. New decision points arise with planning processes related to study measurement and participant inclusion. These are explored in our next chapters.\n\n<h2>Stop and Think<\/h2>\n<div style=\"float: left;min-height: 120px;width: 99%;margin-bottom: 10px;padding: 10px;background-color: #f1f7fe\">\n\n<img class=\"size-full wp-image-194 alignleft\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Stop_ThinkV2-150x150-1.png\" alt=\"Stop and Think\" width=\"150\" height=\"150\">Take a moment to complete the following activity.\n\n[h5p id=\"8\"]\n\n<\/div>","rendered":"<p>Previously, you reviewed different approaches to intervention and evaluation research and learned about a level of evidence (evidence hierarchy) model. These approaches and frameworks relate to how studies are designed to answer different types of research questions. The design strategies differ in the degree to which they address <strong><em>internal validity<\/em><\/strong> concerns\u2014the ability to conclude that changes or differences can be attributed to an intervention rather than to other causes. Designs for quantitative intervention research are the focus of this chapter.<\/p>\n<p>In this chapter, you learn about:<\/p>\n<ul>\n<li>pre-experimental, quasi-experimental, and experimental designs in relation to strength of evidence and internal validity concerns;<\/li>\n<li>how quantitative and single-system study designs are diagrammed;<\/li>\n<li>examples where different designs were used in social work intervention or evaluation research.<\/li>\n<\/ul>\n<h2><strong>Addressing Internal Validity Concerns via Study Design Strategies<\/strong><\/h2>\n<p>The study designs we examine in this chapter differ in terms of their capacity to address specific types of internal validity concerns. As a reminder of what you learned in our previous course, improving internal validity is about increasing investigator confidence that the outcomes observed in an experimental study are as believed, and are due to the experimental variables being studied. In the case of intervention research, the experimental variable being studied is the intervention.<\/p>\n<p>Three general internal validity challenges are important to consider addressing with intervention research.<\/p>\n<ul>\n<li>Were there actually changes that occurred with the intervention (comparing participants\u2019 status pre-intervention to their post-intervention status)?<\/li>\n<li>Did observed changes persist over time (comparing participants\u2019 post-intervention status to their status at some later post-intervention follow-up time)?<\/li>\n<li>Is the observed change most likely due to the intervention itself?<\/li>\n<\/ul>\n<p>Let\u2019s consider how these three questions might relate to the kind of study design choices that investigators make.<\/p>\n<h2><strong>Types of Intervention Research Designs<\/strong><\/h2>\n<p>The intervention study design options can be loosely categorized into three types: pre-experimental, quasi-experimental, and experimental. <strong><em>Pre-experimental designs<\/em><\/strong> do not include comparison (control) groups, only the one group of interest. <strong><em>Quasi-experimental designs<\/em><\/strong> do include comparison (control) groups, but individuals are not randomly assigned to these groups\u2014the groups are created naturally. <strong><em>Experimental designs<\/em><\/strong> include random assignment of study participants to the groups being compared in the study. These three types of designs differ in terms of their attempts to control as many alternative explanation factors as possible\u2014the extent to which they address internal validity concerns.<\/p>\n<h3><strong>Introducing short-hand for intervention study design diagrams<\/strong><strong>. <\/strong><\/h3>\n<p>It can become quite cumbersome to describe all the elements of an intervention study\u2019s design in words and complete sentences; sometimes it is easier to use a diagram instead. This type of short-hand quickly communicates important information about a study design. It can become somewhat confusing, like understanding how the diagram of a football play drawn on a chalkboard translates into reality.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-125\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2018\/12\/diagram-of-a-football-play-drawn-on-a-chalkboard.png\" alt=\"diagram of a football play drawn on a chalkboard\" width=\"281\" height=\"210\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2018\/12\/diagram-of-a-football-play-drawn-on-a-chalkboard.png 281w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2018\/12\/diagram-of-a-football-play-drawn-on-a-chalkboard-65x49.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2018\/12\/diagram-of-a-football-play-drawn-on-a-chalkboard-225x168.png 225w\" sizes=\"auto, (max-width: 281px) 100vw, 281px\" \/><\/p>\n<p>The first thing to know about this intervention study design short-hand is the symbols used and what each symbol means.<\/p>\n<ul>\n<li><strong>X<\/strong> is used to designate that an intervention is being administered.<\/li>\n<li><strong>[X] <\/strong>designates that an \u201cintervention\u201d or event naturally occurred, rather than one imposed by the investigator (for example, a change in policy, a natural disaster, or trauma-inducing crisis event).<\/li>\n<li><strong>O<\/strong> is used to designate that an observation is being made (data are collected).<\/li>\n<li><strong>Subscript numbers<\/strong> are used to designate which intervention or observation is relevant at that point. For example, X<sub>1<\/sub> might refer to the new intervention that is being tested and X<sub>2<\/sub> might refer to the second condition where the usual intervention is delivered. And, O<sub>1<\/sub> might refer to the first observation period (maybe before the intervention), O<sub>2<\/sub> to the second observation period (maybe after the intervention), and O<sub>3<\/sub> to a third observation period (maybe after a longer-term follow-up period).<\/li>\n<li><strong>R <\/strong>is used to designate that individual elements were randomly assigned to the different intervention conditions. (Important reminder: This <strong><em>random assignment<\/em><\/strong> is not about <strong><em>random selection<\/em><\/strong> of a sample to represent a population; it is about using a randomization strategy for placing participants into the different groups in the experimental design).<\/li>\n<\/ul>\n<h3><strong>Costs &amp; Benefits of Various Design Strategies.\u00a0 <\/strong><\/h3>\n<p>Before we get into discussing the specific strategies that might be adopted for intervention research, it is important to understand that every design has its advantages and its disadvantages. There is no such thing as a single, perfect design to which all intervention research should adhere. Investigators are faced with a set of choices that must be carefully weighed. As we explore their available options, one feature that will become apparent is that some designs are more \u201ccostly\u201d to implement than others. This \u201ccost\u201d term is being used broadly here: it is not simply a matter of dollars, though that is an important, practical consideration.<\/p>\n<ul>\n<li>First, a design may \u201ccost\u201d more in terms of more data collection points. There are significant costs associated with each time investigators have to collect data from participants\u2014time, space, effort, materials, reimbursement or incentive payments, data entry, and more. For this reason, longitudinal studies often are more costly than cross-sectional studies.<\/li>\n<li>Second, \u201ccosts\u201d increase with higher numbers of study participants. Greater study numbers \u201ccost\u201d dollars (e.g., advertising expenses, reimbursement or incentive payments, cost of materials used in the study, data entry), but also \u201ccost\u201d more in terms of a greater commitment in time and effort from study staff members, and in terms of greater numbers of persons being exposed to the potential risks associated with the intervention being studied.<\/li>\n<li>Third, some longitudinal study designs \u201ccost\u201d more in terms of the potential for higher rates of participant drop-out from the study over time. Each person who quits a study before it is completed increases the amount of wasted resources, since their data are incomplete (and possibly unusable), and that person may need to be replaced at duplicate cost.<\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-126\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/US-money-floating-around-in-the-air.png\" alt=\"US money floating around in the air\" width=\"325\" height=\"206\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/US-money-floating-around-in-the-air.png 325w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/US-money-floating-around-in-the-air-300x190.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/US-money-floating-around-in-the-air-65x41.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/US-money-floating-around-in-the-air-225x143.png 225w\" sizes=\"auto, (max-width: 325px) 100vw, 325px\" \/><\/p>\n<h2><strong>Ten Typical Evaluation\/Intervention Study Designs<\/strong><\/h2>\n<p>This section presents 10 general study designs that typically appear in intervention and evaluation research. These examples are presented in a general order of increasing ability to address internal validity concerns, but this is offset by increasing costs in resources, participant numbers, and participant burden to implement. Many variations on these general designs appear in the literature; these are 10 general strategies.<\/p>\n<h3><strong>#1: Case study. <\/strong><\/h3>\n<p>Original, novel, or new interventions are sometimes delivered under unusual or unique circumstances. In these instances, when very little is known about intervening under those circumstances, knowledge is extended by sharing a <strong><em>case study<\/em><\/strong>. Eventually, when several case studies can be reviewed together, a clearer picture might emerge about intervening around that condition. At that point, theories and interventions can more systematically be tested. Case studies are considered pre-experimental designs.<\/p>\n<p>An example happened with an adolescent named Jeanna Giese at Milwaukee\u2019s Children\u2019s Hospital of Wisconsin. In 2004, Jeanna was bitten by a bat and three weeks later was diagnosed with full-blown rabies when it was too late to administer a vaccine. At the time, no treatments were known to be successful for rabies once it has developed; the rabies vaccine only works before the disease symptoms develop. Until this case, full-blown rabies was reported to be 100% fatal. The hospital staff implemented an innovative, theory and evidence-informed treatment plan which became known as the \u201cMilwaukee Protocol.\u201d The case study design can be diagrammed the following way, where X represents the \u201cMilwaukee Protocol\u201d intervention and O represents the observed outcomes of the treatment delivered in this case.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-127\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/X-and-O-representing-intervention-and-outcomes.png\" alt=\"X and O representing intervention and outcomes\" width=\"189\" height=\"84\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/X-and-O-representing-intervention-and-outcomes.png 189w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/X-and-O-representing-intervention-and-outcomes-65x29.png 65w\" sizes=\"auto, (max-width: 189px) 100vw, 189px\" \/><\/p>\n<p>Jeanna Giese became the first person in the world known to survive full-blown rabies. The intervention team published the case study, and a handful of individuals around the world have been successfully treated with this protocol\u2014rabies continues to be a highly fatal, global concern. The Milwaukee Protocol is considered somewhat controversial, largely because so few others have survived full-blown rabies even with this intervention being administered; some authors argue that this single case was successful because of unique characteristics of the patient, not because of the intervention protocol\u2019s characteristics (Jackson, 2013).<\/p>\n<p>This argument reflects a major drawback of case studies, which are pre-experimental designs: the unique sample\/small sample size means that individual characteristics or differences can have a powerful influence on the outcomes observed. Thus, the internal validity problem is that the outcomes might be explained by some factors other than the intervention being studied. In addition, with a very small sample size the study results cannot be generalized to the larger population of individuals experiencing the targeted problem\u2014this is an <strong><em>external validity<\/em><\/strong> argument. (We call this an \u201cN of 1\u201d study, where the symbol N refers to the sample size.) The important message here is that case studies are the beginning of a knowledge building trajectory, they are not the end; they inform future research and, possibly, inform practice under circumstances where uncertainty is high with very new problems or solutions. And, just in case you are curious: although some permanent neurological consequences remained, Jeanna Giese completed a college education, was married in 2014, and in 2016 became the mother of twins.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-128\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Illustration-of-bats-flying.png\" alt=\"Illustration of bats flying\" width=\"377\" height=\"268\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Illustration-of-bats-flying.png 377w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Illustration-of-bats-flying-300x213.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Illustration-of-bats-flying-65x46.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Illustration-of-bats-flying-225x160.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Illustration-of-bats-flying-350x249.png 350w\" sizes=\"auto, (max-width: 377px) 100vw, 377px\" \/><\/p>\n<h4><strong>#2: Post-intervention only.<\/strong><\/h4>\n<p>Looking a great deal like the case study design is a simple pre-experimental design where the number of individuals providing data after an intervention is greater than the single or very small number in the case study design. For example, a social work continuing education training session (the intervention) might collect data from training participants at the end of the session to see what they learned during the training event. The trainers might ask participants to rate how much they learned about each topic covered in the training (nothing, a little, some, a lot, very much) or they might present participants with a quiz to test their post-training knowledge of content taught in the session. The <strong><em>post-only design<\/em><\/strong> diagram is the same as what we saw with the single case study; the only difference is that the sample size is greater\u2014it includes everyone who completed the evaluation form at the end of the training session rather than just a single case.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-127\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/X-and-O-representing-intervention-and-outcomes.png\" alt=\"X and O representing intervention and outcomes\" width=\"189\" height=\"84\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/X-and-O-representing-intervention-and-outcomes.png 189w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/X-and-O-representing-intervention-and-outcomes-65x29.png 65w\" sizes=\"auto, (max-width: 189px) 100vw, 189px\" \/><\/p>\n<p>The post-intervention only design is cross-sectional in nature (only one data collection point with each participant). This design strategy is extremely vulnerable to internal validity threats. The investigator does not know if the group\u2019s knowledge changed compared to before the training session: participants quizzed on knowledge may already have known the material before the training; or, a perception of how much they learned may not accurately depict how much they learned. The study design does not inform the investigators if the participants\u2019 learning persisted over time after completing the training. The investigators also do not have a high level of confidence that the training session was the most likely cause of any changes observed\u2014they cannot rule out other possible explanations.<\/p>\n<p>In response to the internal validity threat concerning ability to detect change with an intervention, an investigator might ask study participants to compare themselves before and after the intervention took place. That would still be a simple post- only design because there is only one time point for data collection: post-intervention. This kind of retrospective approach is vulnerable to bias because it relies on an individual\u2019s ability to accurately recall the past and make a valid comparison to the present, a comparison that hopefully is not influenced by their present state-of-mind. It helps to remember what you learned from SWK 3401 about the unreliability of the information individuals remember and how memories become influenced by later information and experiences.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-129\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/image-of-a-humanoid-figure-standing-on-the-word-remember-while-scratching-its-head.png\" alt=\"image of a humanoid figure standing on the word remember while scratching its head\" width=\"235\" height=\"208\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/image-of-a-humanoid-figure-standing-on-the-word-remember-while-scratching-its-head.png 235w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/image-of-a-humanoid-figure-standing-on-the-word-remember-while-scratching-its-head-65x58.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/image-of-a-humanoid-figure-standing-on-the-word-remember-while-scratching-its-head-225x199.png 225w\" sizes=\"auto, (max-width: 235px) 100vw, 235px\" \/><\/p>\n<h4><strong>#3: Pre-\/Post- Intervention Comparison. <\/strong><\/h4>\n<p>A wiser choice in terms of internal validity would be to directly compare data collected at the two points in time: pre-intervention and post-intervention. This <strong><em>pre-\/post-design<\/em><\/strong> remains a pre-experimental design because it lacks a comparison (control) group. Because the data are collected from the same individuals at two time points, this strategy is considered longitudinal in nature. This type of pre-\/post-intervention design allows us to directly identify change where observed differences on a specific outcome variable might be attributed to the intervention. A simple pre-\/post- study design could be diagrammed like this:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-130\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes.png\" alt=\"O1 X and O2 representing intervention and outcomes\" width=\"319\" height=\"86\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes.png 319w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-300x81.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-65x18.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-225x61.png 225w\" sizes=\"auto, (max-width: 319px) 100vw, 319px\" \/><\/p>\n<p>Here we see an intervention X (perhaps our social work in-service training example), where data were still collected after the intervention (perhaps a knowledge and skills quiz). However, the investigators also collected the same information prior to the intervention. This allowed them to compare data for the two observation periods, pre- and post- intervention. See the arrow added to the diagram that shows this comparison:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-131\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-with-O1-and-O2-connected-with-an-arrow.png\" alt=\"O1 X and O2 representing intervention and outcomes with O1 and O2 connected with an arrow\" width=\"305\" height=\"87\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-with-O1-and-O2-connected-with-an-arrow.png 305w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-with-O1-and-O2-connected-with-an-arrow-300x86.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-with-O1-and-O2-connected-with-an-arrow-65x19.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-with-O1-and-O2-connected-with-an-arrow-225x64.png 225w\" sizes=\"auto, (max-width: 305px) 100vw, 305px\" \/><\/p>\n<p>While this pre-\/post- intervention design is stronger than a post-only design, it also is a bit more \u201ccostly\u201d to implement since there is an added data collection point. It imposes an additional burden on participants, and in some situations, it simply might not be possible to collect that pre-intervention data. This design strategy still suffers from the other two internal validity concerns: we do not know if any observed changes persisted over time, and we do not have the highest level of confidence that the changes observed can be attributed to the intervention itself.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-132\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/The-words-Before-After-painted-on-blacktop.png\" alt=\"The words Before &amp; After painted on blacktop\" width=\"279\" height=\"185\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/The-words-Before-After-painted-on-blacktop.png 279w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/The-words-Before-After-painted-on-blacktop-65x43.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/The-words-Before-After-painted-on-blacktop-225x149.png 225w\" sizes=\"auto, (max-width: 279px) 100vw, 279px\" \/><\/p>\n<h4><strong>#4: Pre-\/Post-\/Follow-Up Comparison. <\/strong><\/h4>\n<p>Investigators can improve on the pre-\/post- study design by adding a follow-up observation. This allows them to determine whether any changes observed between the pre- and post- conditions persisted or disappeared over time. While investigators may be delighted to observe a meaningful change between the pre- and post- intervention periods, if these changes do not last over time, then intervention efforts and resources may have been wasted. This <strong><em>pre-\/post-\/follow-up design<\/em><\/strong> remains in the pre-experimental category, and would be diagrammed like this:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-133\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-O2-and-O3-representing-intervention-and-outcomes.png\" alt=\"O1 X O2 and O3 representing intervention and outcomes\" width=\"307\" height=\"87\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-O2-and-O3-representing-intervention-and-outcomes.png 307w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-O2-and-O3-representing-intervention-and-outcomes-300x85.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-O2-and-O3-representing-intervention-and-outcomes-65x18.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-O2-and-O3-representing-intervention-and-outcomes-225x64.png 225w\" sizes=\"auto, (max-width: 307px) 100vw, 307px\" \/><\/p>\n<p>Here we see the intervention (X), with pre-intervention data, post-intervention data, and follow-up data being collected (O<sub>1<\/sub>, O<sub>2<\/sub>, and O<sub>3<\/sub>). As an example, Wade-Mdivanian, Anderson-Butcher, Newman, and Ruderman (2016) explored the impact of delivering a preventive intervention about alcohol, tobacco and other drugs in the context of a positive youth development program called Youth to Youth International. The outcome variables of interest were the youth leaders\u2019 knowledge, attitudes, self-efficacy, and leadership before the program, at the program\u2019s conclusion, and six months after completing the program. The authors concluded that positive changes in knowledge and self-efficacy outcomes were observed in the pre-\/post- comparison, though no significant change was observed in attitudes about alcohol, tobacco and other drugs; these differences persisted at the six-month follow-up. See the pre-\/post-\/follow-up design diagrammed with comparison arrows added:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-134\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-O2-and-O3-representing-intervention-and-outcomes-with-O1-and-O2-connected-with-an-arrow-and-O2-and-O3-connected-with-an-arrow.png\" alt=\"O1 X O2 and O3 representing intervention and outcomes with O1 and O2 connected with an arrow and O2 and O3 connected with an arrow\" width=\"305\" height=\"88\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-O2-and-O3-representing-intervention-and-outcomes-with-O1-and-O2-connected-with-an-arrow-and-O2-and-O3-connected-with-an-arrow.png 305w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-O2-and-O3-representing-intervention-and-outcomes-with-O1-and-O2-connected-with-an-arrow-and-O2-and-O3-connected-with-an-arrow-300x87.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-O2-and-O3-representing-intervention-and-outcomes-with-O1-and-O2-connected-with-an-arrow-and-O2-and-O3-connected-with-an-arrow-65x19.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-O2-and-O3-representing-intervention-and-outcomes-with-O1-and-O2-connected-with-an-arrow-and-O2-and-O3-connected-with-an-arrow-225x65.png 225w\" sizes=\"auto, (max-width: 305px) 100vw, 305px\" \/><\/p>\n<p>This design resolved two of the three internal validity challenges, although at an increased cost of time, effort, and possibly other resources with the added data point. But an investigator would still lack confidence that observed changes were due to the intervention itself. Let\u2019s look at some design strategies that focus on that particular challenge.<\/p>\n<h4><strong>#5: Comparison Groups.<\/strong><\/h4>\n<p>In our prior course we learned how to compare groups that differed on some characteristic, like gender for example. Comparison groups in intervention research allow us to compare groups where the difference lies in which intervention condition each received. By providing an experimental intervention to one group and not to the other group, investigator confidence increases about observed changes being related to the intervention. You may have heard this second group described as a <strong><em>control group<\/em><\/strong>. They are introduced into the study design to provide a benchmark for comparison with the status of the intervention group. The simplest form of a comparison group design, which is a quasi-experimental type of design, a <strong><em>post-only group design<\/em><\/strong> can be diagrammed as follows:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-135 size-full\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/X-and-O-representing-intervention-and-outcomes-and-a-second-row-with-just-an-O.png\" alt=\"the top row shows the group who received the training intervention (X) and the outcome was measured (O), and the bottom row shows the group without the training intervention (no X)\" width=\"237\" height=\"104\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/X-and-O-representing-intervention-and-outcomes-and-a-second-row-with-just-an-O.png 237w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/X-and-O-representing-intervention-and-outcomes-and-a-second-row-with-just-an-O-65x29.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/X-and-O-representing-intervention-and-outcomes-and-a-second-row-with-just-an-O-225x99.png 225w\" sizes=\"auto, (max-width: 237px) 100vw, 237px\" \/><\/p>\n<p>Consider the possibility in our earlier example of evaluating a social work training intervention that the team decided to expand their post only design to include collecting data from a group of social workers who are going to get their training next month. On the same day that the data were collected from the trained group, the team collected data from the untrained social workers, as well. This situation is a post-only design where the top row shows the group who received the training intervention (X) and the outcome was measured (O), and the bottom row shows the group without the training intervention (no X was applied) also being measured at the same point in time as the intervention group (O). This remains a cross-sectional study because each individual was only observed once. The following diagram shows the arrow where investigators compared the two groups on the outcome variables (knowledge and skills quiz scores, using our training example). If the training intervention is responsible for the outcome, the team would see a significant difference in the outcome data when comparing the two groups; hopefully in the direction of the trained group having better scores.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-136\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2018-12-28-at-9.10.18-AM.png\" alt=\"the top row shows the group who received the training intervention (X) and the outcome was measured (O), and the bottom row shows the group without the training intervention (no X) also being measured at the same point in time as the intervention group (O).\" width=\"241\" height=\"106\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2018-12-28-at-9.10.18-AM.png 241w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2018-12-28-at-9.10.18-AM-65x29.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2018-12-28-at-9.10.18-AM-225x99.png 225w\" sizes=\"auto, (max-width: 241px) 100vw, 241px\" \/><\/p>\n<p>While this design has helped boost investigator confidence that the outcomes observed with the intervention group are likely due to the intervention itself, this post-only design \u201ccosts\u201d more than a post-only single group (pre-experimental) study design. This post-only group design still suffers from a significant concern: how would an investigator know if the differences between the two groups appeared only after the intervention or could the differences always have existed, with or without the intervention? With this design, that possibility cannot be ignored. An investigator can only hope that the two groups were equivalent prior to the intervention. Two internal validity questions remain unanswered: did the outcome scores actually demonstrate a change resulting from intervention, and did any observed changes persist over time. Let\u2019s consider some other combination strategies that might help, even though they may \u201ccost\u201d considerably more to implement.<\/p>\n<h4><strong>#6: Comparison Group Pre-\/Post- Design<\/strong>.<\/h4>\n<p>A giant leap forward in managing internal validity concerns comes from combining the strategy of having both an intervention and a comparison group (which makes it quasi-experimental) with the strategy of collecting data both before and after the intervention (which makes it longitudinal). Now investigators are able to address more of the major validity concerns. This <em>comparison group pre-\/post- design <\/em>is diagrammed as follows:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-137\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X.png\" alt=\"O1 X and O2 representing intervention and outcomes and a second row with O1 and O2 but no X\" width=\"322\" height=\"130\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X.png 322w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X-300x121.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X-65x26.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X-225x91.png 225w\" sizes=\"auto, (max-width: 322px) 100vw, 322px\" \/><\/p>\n<p>What the investigators have done here is collect data from everyone in their study, both groups, prior to delivering the intervention to one group and not to the other group (control group). Then, they collected the same information at the post-intervention time period for everyone in their study, both groups. The power in this design can be seen in the following diagrams that include arrows for the kinds of longitudinal pre-\/post- comparisons that can be assessed, measuring change.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-138\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X-with-arrows-connecting-O1-and-O2-on-each-row.png\" alt=\"O1 X and O2 representing intervention and outcomes and a second row with O1 and O2 but no X with arrows connecting O1 and O2 on each row\" width=\"323\" height=\"133\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X-with-arrows-connecting-O1-and-O2-on-each-row.png 323w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X-with-arrows-connecting-O1-and-O2-on-each-row-300x124.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X-with-arrows-connecting-O1-and-O2-on-each-row-65x27.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X-with-arrows-connecting-O1-and-O2-on-each-row-225x93.png 225w\" sizes=\"auto, (max-width: 323px) 100vw, 323px\" \/><\/p>\n<p>This is a pre-\/post- comparison, indicating if change occurred with the intervention group after the intervention. Investigators would hope the answer is \u201cyes\u201d if it is believed that the intervention should make a difference. Similarly, the investigators could compare the non-intervention group at the two observation points (the lower arrow). Hopefully, there would be no significant change without the intervention.<\/p>\n<p>You might be wondering, \u201cWhy would there be change in the no intervention group when nothing has been done to make them change?\u201d Actually, with the passage of time, several possible explanatory events or processes could account for change.<\/p>\n<ul>\n<li>The first is simple <strong><em>maturation<\/em><\/strong>. Particularly with young children, developmental change happens over relatively short periods of time even without intervention.<\/li>\n<li>Similarly, the passage of time might account for symptom improvement even without intervention. Consider, for example, the old adage that, \u201cif you treat a common cold it will take 7 to 10 days to get better; if you don\u2019t treat it, it will take a week to a week-and-a-half.\u201d Change without intervention is called <strong><em>spontaneous or natural change<\/em><\/strong>. Either way, there is change\u2014with or without intervention.<\/li>\n<li>Third, given time, individuals in the no intervention group might seek out and receive other interventions not part of the study that also can produce change. This could be as simple as getting help and support from friends or family members; or, seeking help and advice on the internet; or, enrolling in other informal or formal treatment programs at the same time.<\/li>\n<\/ul>\n<p>The benefit of combining the comparison groups and pre-\/post- designs continues to emerge as we examine the next diagram showing comparisons an investigator can consider:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-139\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X-an-arrow-connects-O2-on-each-row-to-represent-a-comparison.png\" alt=\"\" width=\"273\" height=\"124\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X-an-arrow-connects-O2-on-each-row-to-represent-a-comparison.png 273w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X-an-arrow-connects-O2-on-each-row-to-represent-a-comparison-65x30.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X-an-arrow-connects-O2-on-each-row-to-represent-a-comparison-225x102.png 225w\" sizes=\"auto, (max-width: 273px) 100vw, 273px\" \/><\/p>\n<p>This group comparison of post-intervention results indicates whether there is a difference in outcomes between people who received the intervention and people who did not. Again, investigators would hope the answer is \u201cyes,\u201d and that the observed difference favors the intervention group. But with this design, an investigator can go even further in ruling out another possible explanation for outcome differences. Consider the power of this comparison:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-140\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X-an-arrow-connects-O1-on-each-row-to-represent-a-comparison.png\" alt=\"\" width=\"280\" height=\"127\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X-an-arrow-connects-O1-on-each-row-to-represent-a-comparison.png 280w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X-an-arrow-connects-O1-on-each-row-to-represent-a-comparison-65x29.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-but-no-X-an-arrow-connects-O1-on-each-row-to-represent-a-comparison-225x102.png 225w\" sizes=\"auto, (max-width: 280px) 100vw, 280px\" \/><\/p>\n<p>By comparing the two groups BEFORE the intervention took place, an investigator can hopefully rule out the possibility that post-intervention group differences were actually a reflection of pre-existing differences; differences that existed prior to the intervention. In this case, the investigator would hope to observe no significant differences in this comparison. This \u201cno differences\u201d result would boost confidence in the conclusion that any observed post-intervention differences were a function of the intervention itself (since there were no pre-existing differences). Not being able to rule out this possibility was one limitation of the post-only group comparison strategy discussed earlier.<\/p>\n<h4><em>A Note About Non-Treatment, Placebo, and Treatment as Usual Comparison Groups<\/em>.<\/h4>\n<p>What we have diagrammed above is a situation where the second group received no treatment at all. This, however, is problematic in three ways. First, the ethics of intentionally not serving individuals who are seeking help with a social work problem is concerning. Second, the scientific integrity of such studies tends to suffer because non-treatment \u201ccontrol\u201d groups tend to have high rates of study drop out (attrition) as those participants seek help elsewhere. Third, the results of these studies are coming under great scrutiny as the world of behavioral science has come to realize that any treatment is likely to be better than no treatment\u2014thus, study results where the tested intervention is significantly positive may be grossly over-interpreted. Compared to other treatments, what appears to be a fantastic innovation may be no better.<\/p>\n<p>Medical studies often include a comparison group who receives a \u201cfake\u201d or neutral form of a medication or other intervention. In medicine, a placebo is an inert \u201ctreatment\u201d with a substance that has no likely known effect on the condition being studies, such as a pill made of sugar, for example. The approximate equivalent in behavior al science is an intervention where clients are provided only with basic, factual information about the condition; nothing too empowering. In theory, neither the placebo medication nor the simple educational materials are expected to promote significant change. This is a slight variation on the <strong><em>non-treatment control<\/em><\/strong> condition. However, over 20 years of research provided evidence of a <strong><em>placebo effect<\/em><\/strong> that cannot be discounted. In one systematic review and meta-analysis study (Howick et al, 2016) the size of effect associated with placebos were no different or even larger than treatment effects. This placebo effect is, most likely, associated with the psychological principles of motivation and expectancies\u2014expecting something to work has a powerful impact on behavior and outcomes, particularly with regard to symptoms of nausea and pain (Cherry, 2018). Of course, this means that participants receiving the placebo believe they are (or could be) receiving the therapeutic intervention. Also interesting to note is that participants sometimes report negative side-effects with exposure to the placebo treatment.<\/p>\n<p>In medication trials, the introduction of a placebo may allow investigators to impose a double-blind structure to the study. A <strong><em>double-blind study<\/em><\/strong> is one where neither the patient\/client nor the practitioner\/clinician knows if the person is receiving the test medication or the placebo condition. The double-blind structure is imposed as means of reducing practitioner bias in the study results from either patient or practitioner\/clinician beliefs about the experimental medication. However, it is difficult to disguise behavioral interventions from practitioners\u2014it is not as simple as creating a real-looking pill or making distilled water look like medicine.<\/p>\n<p>More popular in intervention science today is the use of a <strong><em>treatment as usual condition (TAU)<\/em><\/strong>. In a TAU study, the comparison group receives the same treatment that would have been provided without the study being conducted. This resolves the ethical concerns of intentionally denying care to someone seeking help simply to fulfill demands of a research design. It also helps resolve the study integrity concerns mentioned earlier regarding non-treatment studies. But this design looks a little bit different from the diagram of a comparison group design. You will notice that there is a second intervention \u201cX\u201d symbol in the diagram and that the new X has a subscript notation (TAU) so you can tell the two intervention groups apart, X and X<sub>TAU<\/sub>.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-141\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-and-with-an-X-that-represnts-treatment-as-usual-condition.png\" alt=\"O1 X and O2 representing intervention and outcomes and a second row with O1 and O2 and with an X that represnts treatment as usual condition\" width=\"325\" height=\"132\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-and-with-an-X-that-represnts-treatment-as-usual-condition.png 325w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-and-with-an-X-that-represnts-treatment-as-usual-condition-300x122.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-and-with-an-X-that-represnts-treatment-as-usual-condition-65x26.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-and-O2-representing-intervention-and-outcomes-and-a-second-row-with-O1-and-O2-and-with-an-X-that-represnts-treatment-as-usual-condition-225x91.png 225w\" sizes=\"auto, (max-width: 325px) 100vw, 325px\" \/><\/p>\n<h4><strong>#7: Single-System Design<\/strong>.<\/h4>\n<p>A commonly applied means of evaluating practice is to employ a quasi-experimental single-system design. This approach uniquely combines aspect of the case study with aspects of the pre-experimental pre-\/post-design. Like the case study, the data are collected for one single case at a time\u2014whether the element or unit of study is an individual, couple, family, or larger group, the data represent the behavior of that element over time. In that sense, the single-system design is longitudinal\u2014repeated measurements are drawn for the same element each time. It is a quasi-experimental design in that conditions are systematically varied, and outcomes measured for each variation. This approach to evaluating practice is often referred to as a single-subject design. However, the fact that the \u201csubject\u201d might be a larger system is lost in that label, hence a preference for the single-\u201csystem\u201d design label.<\/p>\n<p>The single-system design is sufficiently unique in implementation that an entirely different notation system is applied. Instead of the previous Xs and Os, we will be using A\u2019s and B\u2019s (even C\u2019s and D\u2019s). The first major distinction is that instead of a pre-intervention measurement (what we called O<sub>1<\/sub>) we need a pre-intervention baseline. By definition, a line is the distance between two or more points, thus in order to be a baseline measurement, at least two and preferably at least 7 pre-intervention measurement points are utilized. For example, if the target of intervention with a family is that they spend more activity time together, a practitioner might have them maintain a daily calendar with the number of minutes spent in activity time is recorded for each of 7 days. This would be used as a baseline indication of the family\u2019s behavioral norm. Thus, since there are 7 measurement points replacing the single pre-intervention observation (O<sub>1<\/sub>), we designate this as A<sub>1<\/sub>, A<sub>2<\/sub>, A<sub>3<\/sub> and so forth to A<sub>7<\/sub> for days 1 through 7 during the baseline week&#8211;this might extend to A<sub>30<\/sub> for a month of baseline recording.<\/p>\n<p>Next, we continue to measure the target behavior during the period of intervention. Because the intervention period is different from the baseline period, we use the letter B to indicate the change. This corresponds to the single point in our group designs where we used the letter X to designate the intervention. Typically, in a pre-\/post- intervention study no data are collected during the intervention, X, which is another difference in single-system designs. Let\u2019s say that the social worker\u2019s intervention was to text the family members an assigned activity for each of 10 days, and they continued to record the number of minutes when they were engaged in family activity time each day. Because it is a series of measurement points, we use B<sub>1<\/sub>, B<sub>2<\/sub>, B<sub>3<\/sub> and so forth for the duration of the intervention, which is B<sub>10<\/sub> in this case. The next step is to remove the daily assignment text messages, giving them just one menu on the first of the next 7 days, from which the family is expected to pick an activity and continue to record time spent in family activity time each day. This would be a different form of intervention from the \u201cB\u201d condition, so it becomes C<sub>1<\/sub>, C<sub>2<\/sub>, C<sub>3<\/sub>, and so forth to the end, C<sub>7<\/sub>. Finally, the social worker no longer sends any cues, and the family simply records their daily family activity time for another week. This \u201cno intervention\u201d condition is the same as what happened at baseline before there was any intervention. So, the notation reverts back to A\u2019s, but this time it is A<sub>8<\/sub>, A<sub>9<\/sub>, A<sub>10<\/sub>, and so forth to the end of the observation period, A<sub>14<\/sub>. For this reason, single system design studies are often referred to as \u201cABA\u201d designs (initial baseline, an intervention line, and a post-intervention line) or, in our example an &#8220;ABCA&#8221; design since there was a second intervention after the &#8220;B&#8221; condition. This single-system design notation is different from the notation using Xs and Os because data are collected multiple times during each phase, rather than at each single point during pre-intervention, intervention, and post-intervention.<\/p>\n<p>The data could be presented in a table format, but a visual graph depicts the trends in a more concise, communicative manner. What the practitioner is aiming to do with the family is to look at the pattern of behavior in as objective manner as possible, under the different manipulated conditions. Here is a graphical representation from one hypothetical family. As you can see, there is natural variation in the family\u2019s behavior that would be missed if we simply used a single weekly value for the pre- and post-intervention periods instead.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-142\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Minutes-Daily-Familty-Activity-Time-Chart.png\" alt=\"Minutes Daily family Activity Time Chart\" width=\"874\" height=\"470\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Minutes-Daily-Familty-Activity-Time-Chart.png 874w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Minutes-Daily-Familty-Activity-Time-Chart-300x161.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Minutes-Daily-Familty-Activity-Time-Chart-768x413.png 768w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Minutes-Daily-Familty-Activity-Time-Chart-65x35.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Minutes-Daily-Familty-Activity-Time-Chart-225x121.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Minutes-Daily-Familty-Activity-Time-Chart-350x188.png 350w\" sizes=\"auto, (max-width: 874px) 100vw, 874px\" \/><\/p>\n<p>Together, the practitioner and the client family can discuss what they see in the data. For example, what makes Wednesday family activity time particularly difficult to implement and does it matter given the what happens on the other days? How does it feel as a family to have the greater activity days, and is that rewarding to them? What happens when the social worker is no longer prompting them to engage in activities and how can they sustain their gains over time without outside intervention? What new skills did they learn that support sustainability? What will be their cue that a \u201cbooster\u201d might be necessary? What the data allow is a clear evaluation of the intervention with this family, which is the main purpose of practice evaluation using the single-system design approach.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-143\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/The-words-Before-During-and-After-painted-on-blacktop.png\" alt=\"The words Before During and After painted on blacktop\" width=\"289\" height=\"191\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/The-words-Before-During-and-After-painted-on-blacktop.png 289w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/The-words-Before-During-and-After-painted-on-blacktop-65x43.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/The-words-Before-During-and-After-painted-on-blacktop-225x149.png 225w\" sizes=\"auto, (max-width: 289px) 100vw, 289px\" \/><\/p>\n<h4><strong>#8: Random Control Trial (RCT) Pre-\/Post- Design.<\/strong><\/h4>\n<p>The major difference between the comparison group pre-\/post- design just discussed and the <strong><em>random control trial (RCT)<\/em><\/strong> <em>with pre-\/post- design<\/em> is that investigators do not have to rely so much on hoping that the two comparison groups were initially equivalent: they randomly assign study participants to the two groups as an attempt to ensure that this is true. There is still no guarantee of initial group equivalence; this still needs to be assessed. But the investigators are less vulnerable to the bad luck of having the two comparison groups being initially different. This is what the RCT design looks like in a diagram:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-144 aligncenter\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/R-O1-X-and-O2-representing-intervention-a.png\" alt=\"R O1 X and O2 representing intervention and outcomes and a second row with O1 and O2 but no X an arrow connects O2 on each row to represent a comparison and R is connevted via an arrow to indicate randomization\" width=\"336\" height=\"146\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/R-O1-X-and-O2-representing-intervention-a.png 336w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/R-O1-X-and-O2-representing-intervention-a-300x130.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/R-O1-X-and-O2-representing-intervention-a-65x28.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/R-O1-X-and-O2-representing-intervention-a-225x98.png 225w\" sizes=\"auto, (max-width: 336px) 100vw, 336px\" \/><\/p>\n<p>Many authors of research textbooks describe the RCT as the \u201cgold standard\u201d of intervention research design because it addresses so many possible internal validity concerns. It is, indeed, a powerful experimental design. However, it is important to recognize that this type of design comes at a relatively high \u201ccost\u201d compared to some of the others we have discussed. Because there are two comparison groups being compared, there are more study participant costs involved than the single group designs. Because there are at least two points in time when data are collected, there are more data collection and participant retention costs involved than the single time point post-only designs. And random assignment to experimental conditions or groups is not always feasible in real-world intervention situations. For example, it is difficult to apply random assignment to conditions where placement is determined by court order, making it difficult to use an RCT design for comparing a jail diversion program (experimental intervention) with incarceration (treatment as usual control group). For these practical reasons, programs often settle on group comparison designs without random assignment in their evaluation efforts.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-145\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Pile-of-gold-bars.png\" alt=\"Pile of gold bars\" width=\"195\" height=\"147\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Pile-of-gold-bars.png 195w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Pile-of-gold-bars-65x49.png 65w\" sizes=\"auto, (max-width: 195px) 100vw, 195px\" \/><\/p>\n<div class=\"textbox textbox--sidebar textbox--key-takeaways\" style=\"min-width: 100%\">\n<header class=\"textbox__header\">\n<p class=\"textbox__title\">Back to Basics: Interpreting Statistics<\/p>\n<\/header>\n<div class=\"textbox__content\">\n<p>Remembering how to interpret statistics in a research report. The investigators reported that for the intervention group, the mean reduction in AUDIT-12 scores was significantly greater for the SBI intervention group than for the TAU group on a one-way Anova: M=18.83 and 13.52 respectively, <em>F<\/em>(1,148)=6.34, <em>p<\/em>&lt;.01. This statement indicates the following:<\/p>\n<ol>\n<li>The mean (M) for the innovative SBI intervention group =18.83<\/li>\n<li>The mean (<em>M<\/em>) for the treatment as usual (TAU) group =13.52.<\/li>\n<li>The computed <em>F <\/em>statistic in the analysis of variance (Anova) =6.34.<\/li>\n<li>Degrees of freedom for the Anova were 1 (2 groups -1) and 148 (149 cases &#8211; 1): <em>df<\/em>=(1, 148).<\/li>\n<li>The <em>F<\/em>-distribution test statistic at those degrees of freedom is significant at a level less than the criterion <em>p<\/em>&lt;.05.<\/li>\n<li>We reject the null hypothesis of no difference between the groups, concluding that a significant difference exists.<\/li>\n<\/ol>\n<\/div>\n<\/div>\n<p>A slight variant on this study design was used to compare a screening and brief intervention (SBI) for substance misuse problems to the usual (TAU) condition among 729 women preparing for release from jail and measured again during early reentry to community living (Begun, Rose, &amp; LeBel, 2011). One-third of the women who had positive screen results for a potential substance use disorder were randomly assigned to the treatment as usual condition (X<sub>TAU<\/sub>), and two-thirds to the SBI experimental intervention condition (X) as diagrammed below. The investigators followed 149 women three months into post-release community reentry ( follow-up observation), and found that women receiving the innovative SBI intervention had better outcomes for drinking and drug use during early reentry (three months post-release): the mean difference in scores on the AUDIT-12 screening instrument prior to jail compared to reentry after release was more than 5 points (see Back to Basics Box for more details). The <em>random controlled trial with pre\/post\/follow-up study design<\/em> looked like this:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-146\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/R-O1-X-O2-and-O3-representing-intervention-and-outcomes-and-a-second-row-with-R-O1-O2-and-O3-and-with-an-X-that-represnts-treatment-as-usual-condition.png\" alt=\"R O1 X O2 and O3 representing intervention and outcomes and a second row with R O1 O2 and O3 and with an X that represnts treatment as usual condition\" width=\"393\" height=\"130\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/R-O1-X-O2-and-O3-representing-intervention-and-outcomes-and-a-second-row-with-R-O1-O2-and-O3-and-with-an-X-that-represnts-treatment-as-usual-condition.png 393w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/R-O1-X-O2-and-O3-representing-intervention-and-outcomes-and-a-second-row-with-R-O1-O2-and-O3-and-with-an-X-that-represnts-treatment-as-usual-condition-300x99.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/R-O1-X-O2-and-O3-representing-intervention-and-outcomes-and-a-second-row-with-R-O1-O2-and-O3-and-with-an-X-that-represnts-treatment-as-usual-condition-65x22.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/R-O1-X-O2-and-O3-representing-intervention-and-outcomes-and-a-second-row-with-R-O1-O2-and-O3-and-with-an-X-that-represnts-treatment-as-usual-condition-225x74.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/R-O1-X-O2-and-O3-representing-intervention-and-outcomes-and-a-second-row-with-R-O1-O2-and-O3-and-with-an-X-that-represnts-treatment-as-usual-condition-350x116.png 350w\" sizes=\"auto, (max-width: 393px) 100vw, 393px\" \/><\/p>\n<h4><strong>#9: Random Control Trial (RCT) Comparing Two (or More) Interventions. <\/strong><\/h4>\n<p>The only difference between the design just discussed and the random control trial that compares two interventions is the indicator associated with the intervention symbol, X. The X<sub>1<\/sub> and X<sub>2<\/sub> refer to two different interventions, the same way the X<sub>1<\/sub> and X<sub>TAU<\/sub> reflected two different intervention conditions above. This kind of design is used to determine which of two interventions has greater effectiveness, especially if there is a considerable difference in cost of delivering them. The results can help with a cost-benefit comparison which program administrators and policy decision-makers use to help make decisions about which to fund. It is possible to compare more than two interventions by simply adding additional lines for each group (X<sub>3<\/sub>, X<sub>4<\/sub>, and so on). Each group added, however, also adds considerably to the \u201ccosts\u201d of conducting the study.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-147\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/R-O1-X1-and-O2-representing-different-interventions.png\" alt=\"R O1 X1 and O2 representing different interventions. In the second tow, X is labelled X2 to represented a different intervention\" width=\"321\" height=\"129\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/R-O1-X1-and-O2-representing-different-interventions.png 321w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/R-O1-X1-and-O2-representing-different-interventions-300x121.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/R-O1-X1-and-O2-representing-different-interventions-65x26.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/R-O1-X1-and-O2-representing-different-interventions-225x90.png 225w\" sizes=\"auto, (max-width: 321px) 100vw, 321px\" \/><\/p>\n<p>The COMBINE Project was an historic example where multiple intervention approaches were compared for their effectiveness in treating alcohol dependence (see NIAAA Overview of COMBINE at <a href=\"https:\/\/pubs.niaaa.nih.gov\/publications\/combine\/overview.htm\">https:\/\/pubs.niaaa.nih.gov\/publications\/combine\/overview.htm<\/a>).<\/p>\n<p>Two medications were being compared (acamprosate and naltrexone) along with medical management counseling for adherence to the medication protocol (MM), and medication with MM was compared to medication with MM plus a cognitive behavioral psychotherapy intervention (CBI). Clients were randomly assigned to 9 groups with approximately 153 clients per group:<\/p>\n<p style=\"padding-left: 60px\">X<sub>1<\/sub>-acamprosate placebo + naltrexone placebo + MM (no CBI)<\/p>\n<p style=\"padding-left: 60px\">X<sub>2<\/sub>-acamprosate + naltrexone placebo + MM (no CBI)<\/p>\n<p style=\"padding-left: 60px\">X<sub>3<\/sub>-naltrexone + acamprosate placebo + MM (no CBI)<\/p>\n<p style=\"padding-left: 60px\">X<sub>4<\/sub>-acamprosate + naltrexone + MM (no CBI)<\/p>\n<p style=\"padding-left: 60px\">X<sub>5<\/sub>&#8211; acamprosate placebo + naltrexone placebo + MM + CBI<\/p>\n<p style=\"padding-left: 60px\">X<sub>6<\/sub>&#8211; acamprosate + naltrexone placebo + MM + CBI<\/p>\n<p style=\"padding-left: 60px\">X<sub>7<\/sub>&#8211; naltrexone + acamprosate placebo + MM + CBI<\/p>\n<p style=\"padding-left: 60px\">X<sub>8<\/sub>&#8211; acamprosate + naltrexone + MM + CBI<\/p>\n<p style=\"padding-left: 60px\">X<sub>9<\/sub>-no pills or MM, CBI only<\/p>\n<p>The COMBINE Project was an extremely \u201ccostly\u201d study to conduct given the number of study participants needed to meet the study\u2019s design demands&#8211;one reason why it was a collaboration across multiple study sites. The results included participants in all 9 groups showing at least some improvement as measured by a reduction in drinking&#8211;intervention is almost always an advantage over no intervention. The poorest results were observed for the last group (X<sub>9<\/sub>), those receiving the specialty cognitive behavioral intervention (CBI) alone, with no medication or placebo medication. Surprising was that best outcomes were observed for the group receiving CBI with both placebo medications and medication management (MM) counseling (X<sub>5<\/sub>)! The other group with best outcomes was the group receiving naltrexone with MM counseling but no CBI (X<sub>3<\/sub>). The investigative team concluded that pharmacotherapy combined with medication counseling can yield clinically significant alcohol treatment outcomes, and this can be delivered in primary care settings where specialty alcohol treatment is unavailable (Pettinati, Anton, &amp; Willenbring, 2006). They were surprised, also, by the lack of observed advantage to using the medication acamprosate because it had so much evidence support from studies conducted in European countries. This map of the design does not show the multiple follow-up observations conducted in the study.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-148\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/R-01-X1-and-O2-in-each-row-with-X1-being-incremented-to-show-different-interventions.png\" alt=\"R 01 X1 and O2 in each row with X1 being incremented to show different interventions\" width=\"324\" height=\"248\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/R-01-X1-and-O2-in-each-row-with-X1-being-incremented-to-show-different-interventions.png 324w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/R-01-X1-and-O2-in-each-row-with-X1-being-incremented-to-show-different-interventions-300x230.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/R-01-X1-and-O2-in-each-row-with-X1-being-incremented-to-show-different-interventions-65x50.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/R-01-X1-and-O2-in-each-row-with-X1-being-incremented-to-show-different-interventions-225x172.png 225w\" sizes=\"auto, (max-width: 324px) 100vw, 324px\" \/><\/p>\n<h4><strong>#10: Comparison Group with a Delayed Intervention Condition. <\/strong><\/h4>\n<p>One way of overcoming the ethical dilemma of an intervention\/no intervention study design is to offer the intervention condition later for the second group. Not only does this more evenly distribute risks and benefits potentially experienced across both groups, it also makes sense scientifically and practically because it allows another set of comparison conditions to be longitudinally analyzed for a relatively smaller additional \u201ccost.\u201d Here is what this design strategy might look like using our study design notation:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-149\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-O2-and-O3-representing-intervention-and-outcomes2.png\" alt=\"O1 X O2 and O3 representing intervention and outcomes. In row 1 X is between O1 and O2 and in row 2 it is between O2 and O3\" width=\"384\" height=\"132\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-O2-and-O3-representing-intervention-and-outcomes2.png 384w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-O2-and-O3-representing-intervention-and-outcomes2-300x103.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-O2-and-O3-representing-intervention-and-outcomes2-65x22.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-O2-and-O3-representing-intervention-and-outcomes2-225x77.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/O1-X-O2-and-O3-representing-intervention-and-outcomes2-350x120.png 350w\" sizes=\"auto, (max-width: 384px) 100vw, 384px\" \/><\/p>\n<p>Look at all the information available through this design:<\/p>\n<ul>\n<li>First is a simple pre-\/post- design (O<sub>1<\/sub> to O<sub>2<\/sub> with X), where investigators hope to see a difference;<br \/>\n<img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-150\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/1.png\" alt=\"\u2022 First is a simple pre-\/post- design (O1 to O2 with X), where investigators hope to see a difference;\" width=\"378\" height=\"110\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/1.png 378w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/1-300x87.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/1-65x19.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/1-225x65.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/1-350x102.png 350w\" sizes=\"auto, (max-width: 378px) 100vw, 378px\" \/><\/li>\n<li>Second is the control group pre-\/post- comparison (O<sub>1<\/sub> to O<sub>2<\/sub> without intervention X), where investigators hope for no difference, or at least significantly less difference than for the group with the intervention early on;<br \/>\n<img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-151\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/2.png\" alt=\"\u2022 Second is the control group pre-\/post- comparison (O1 to O2 without intervention X), where investigators hope for no difference, or at least significantly less difference than for the group with the intervention early on;\" width=\"375\" height=\"127\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/2.png 375w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/2-300x102.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/2-65x22.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/2-225x76.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/2-350x119.png 350w\" sizes=\"auto, (max-width: 375px) 100vw, 375px\" \/><\/li>\n<li>Third is the comparison of the group with the intervention (X) to the group without the intervention, after the intervention was delivered (O<sub>2<\/sub> for each group), where the investigators hope to see a significant difference between the two groups, and that the difference favors the group who received the intervention (X);<br \/>\n<img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-152\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/3.png\" alt=\"\u2022 Third is the comparison of the group with the intervention (X) to the group without the intervention, after the intervention was delivered (O2 for each group), where the investigators hope to see a significant difference between the two groups, and that the difference favors the group who received the intervention (X);\" width=\"374\" height=\"105\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/3.png 374w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/3-300x84.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/3-65x18.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/3-225x63.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/3-350x98.png 350w\" sizes=\"auto, (max-width: 374px) 100vw, 374px\" \/><\/li>\n<li>Fourth is the post-\/follow-up comparison for the group with the intervention (O<sub>2<\/sub> to O<sub>3<\/sub>), where investigators hope that the changes seen on O<sub>2<\/sub> persisted at O<sub>3<\/sub> (or are even more improved, not that they declined over time;<br \/>\n<img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-153\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/4.png\" alt=\"\u2022 Fourth is the post-\/follow-up comparison for the group with the intervention (O2 to O3), where investigators hope that the changes seen on O2 persisted at O3 (or are even more improved, not that they declined over time;\" width=\"375\" height=\"104\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/4.png 375w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/4-300x83.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/4-65x18.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/4-225x62.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/4-350x97.png 350w\" sizes=\"auto, (max-width: 375px) 100vw, 375px\" \/><\/li>\n<li>Fifth is the ability to replicate the results of the first group receiving the intervention with the results for the second group to receive the intervention (O<sub>2<\/sub> for the first group with O<sub>3<\/sub> for the second group\u2014actually, change from O<sub>1<\/sub> to O<sub>2<\/sub> for the first group and change from O<sub>1<\/sub> to O<sub>3<\/sub> for the second group). Ideally, the investigators would see similar results for the two groups\u2014those who received the intervention early on and those who received it a bit later;<br \/>\n<img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-154\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/5.png\" alt=\"\u2022 Fifth is the ability to replicate the results of the first group receiving the intervention with the results for the second group to receive the intervention (O2 for the first group with O3 for the second group\u2014actually, change from O1 to O2 for the first group and change from O1 to O3 for the second group). Ideally, the investigators would see similar results for the two groups\u2014those who received the intervention early on and those who received it a bit later\" width=\"383\" height=\"126\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/5.png 383w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/5-300x99.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/5-65x21.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/5-225x74.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/5-350x115.png 350w\" sizes=\"auto, (max-width: 383px) 100vw, 383px\" \/><\/li>\n<li>Sixth is the ability to test the assumption that the two groups were similar prior to the study beginning\u2014that the differences observed between them were related to the intervention and not pre-existing group differences (O<sub>1<\/sub> for both groups), as well as differences immediately after intervention and between follow-up and immediately post-intervention.<br \/>\n<img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-155\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/6.png\" alt=\"\u2022 Sixth is the ability to test the assumption that the two groups were similar prior to the study beginning\u2014that the differences observed between them were related to the intervention and not pre-existing group differences (O1 for both groups), as well as differences immediately after intervention and between follow-up and immediately post-intervention\" width=\"372\" height=\"103\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/6.png 372w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/6-300x83.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/6-65x18.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/6-225x62.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/6-350x97.png 350w\" sizes=\"auto, (max-width: 372px) 100vw, 372px\" \/><\/li>\n<\/ul>\n<p>This study design was used to examine the effectiveness of equine-assisted therapy for persons who have dementia (Dabelko-Schoeny, et al., 2014). The study engaged 16 participants with Alzheimer\u2019s Disease in activities with four horses (grooming, observing, interacting, leading, and photographing). Study participants were randomly assigned to two groups: one received the intervention immediately the other did not; however, this latter group received the intervention later, after the intervention was withdrawn from the first group. When not receiving the equine-assisted therapy, participants received the usual services. Thus, the investigators were able to make multiple types of comparisons and ensured that all participants had the opportunity to experience the novel intervention. The team reported that the equine-assisted therapy was significantly associated with lower rates of problematic\/disruptive behavior being exhibited by the participants in their nursing homes and higher levels of \u201cgood stress\u201d (sense of exhilaration or accomplishment) as measured by salivary cortisol levels.<\/p>\n<p>Major disadvantages of this study design:<\/p>\n<ul>\n<li>it may be that social work interventions are best delivered right away, when the need arises, and that their impact is diminished when delivered after a significant delay;<\/li>\n<li>the design is costlier to implement than either design alone might be, because it requires more participants and that they be retained over a longer period of time.<\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-156\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/large-collection-of-clocks-with-pastel-colored-faces.png\" alt=\"\" width=\"202\" height=\"135\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/large-collection-of-clocks-with-pastel-colored-faces.png 202w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/large-collection-of-clocks-with-pastel-colored-faces-65x43.png 65w\" sizes=\"auto, (max-width: 202px) 100vw, 202px\" \/><\/p>\n<h2><strong>Chapter Conclusion<\/strong><\/h2>\n<p>As you can see, social work professionals and investigators have numerous options available to them in planning how to study the impact of interventions and evaluate their practices, programs, and policies. Each option has pros and cons, advantages and disadvantages, costs and benefits that need to be weighed in making the study design decisions. New decision points arise with planning processes related to study measurement and participant inclusion. These are explored in our next chapters.<\/p>\n<h2>Stop and Think<\/h2>\n<div style=\"float: left;min-height: 120px;width: 99%;margin-bottom: 10px;padding: 10px;background-color: #f1f7fe\">\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-194 alignleft\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Stop_ThinkV2-150x150-1.png\" alt=\"Stop and Think\" width=\"150\" height=\"150\" \/>Take a moment to complete the following activity.<\/p>\n<div class=\"h5p-iframe-wrapper\"><iframe id=\"h5p-iframe-8\" class=\"h5p-iframe\" data-content-id=\"8\" style=\"height:1px\" src=\"about:blank\" frameBorder=\"0\" scrolling=\"no\" title=\"Module 3 chapter 2\"><\/iframe><\/div>\n<\/div>\n","protected":false},"author":3,"menu_order":3,"template":"","meta":{"pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":[],"pb_section_license":""},"chapter-type":[],"contributor":[],"license":[],"class_list":["post-157","chapter","type-chapter","status-publish","hentry"],"part":110,"_links":{"self":[{"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/pressbooks\/v2\/chapters\/157","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/wp\/v2\/users\/3"}],"version-history":[{"count":1,"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/pressbooks\/v2\/chapters\/157\/revisions"}],"predecessor-version":[{"id":158,"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/pressbooks\/v2\/chapters\/157\/revisions\/158"}],"part":[{"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/pressbooks\/v2\/parts\/110"}],"metadata":[{"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/pressbooks\/v2\/chapters\/157\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/wp\/v2\/media?parent=157"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/pressbooks\/v2\/chapter-type?post=157"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/wp\/v2\/contributor?post=157"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/wp\/v2\/license?post=157"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}