{"id":236,"date":"2019-01-09T16:09:24","date_gmt":"2019-01-09T16:09:24","guid":{"rendered":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/chapter\/module-4-chapter-3\/"},"modified":"2024-08-19T12:19:11","modified_gmt":"2024-08-19T12:19:11","slug":"module-4-chapter-3","status":"publish","type":"chapter","link":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/chapter\/module-4-chapter-3\/","title":{"rendered":"Module 4 Chapter 3: Analysis of Single System Design Data"},"content":{"raw":"As you learned in the previous module, one approach to evaluating social work practice involves collecting data using a single system design. Many of the approaches we have studied are about what happens with different groups of participants. The single system design is sometimes called \u201cintrasubject\u201d research: the data are related to what happens within an individual system (Nugent, 2010). Single system designs generally involve collecting baseline data repeatedly for a period prior to implementing an intervention (the \u201cA\u201d phase) and collecting data during the intervention period (the \u201cB\u201d phase). A common variation on this design include collecting data during a period when the intervention is removed (another \u201cA\u201d phase), hence the ABA designation. Many other variations exist including adding and comparing alternative interventions (B<sub>1<\/sub>, B<sub>2<\/sub>, B<sub>3<\/sub> for example) and collecting data during each of those phases. The issue in this current chapter is what to do with the collected data, how to analyze it for meaning.\n\nIn this chapter you learn:\n<ul>\n \t<li>visual graphic approaches to analyzing single system design data;<\/li>\n \t<li>assessing changes between single system design phases in level, trend, variability, overlap, means, persistence, and latency of change;<\/li>\n \t<li>two statistical analysis approaches for complementing visual approaches (non-overlap of all pairs analysis and two-standard deviation band analysis).<\/li>\n<\/ul>\n<h2><strong>Graphing Single System Design Data<\/strong><\/h2>\nThe first step in analyzing single-system design data is the visual analysis. According to Engel and Schutt (2013), visual examination of the graphed data is \u201cthe most common method\u201d of analysis (p. 200). The purpose is to determine whether the target variable (outcome) changed between the baseline and intervention phases. Let\u2019s begin by examining the elements of the single system design graph.\n\nFirst, the phase of a single study design can be considered as the <strong><em>independent variable\u00a0<\/em><\/strong>(the one that is manipulated experimentally) and the measured outcome as the <strong><em>dependent variable<\/em><\/strong>. Social work interventions often have multiple goals or objectives\u2014perhaps addressing multiple behaviors, or a behavior and an attitude, or knowledge on several topics. For the purposes of simplifying the understanding single system design data, we will focus on data measuring only one outcome variable at a time.\n\nThe single system design graph has \u201ctime\u201d as the horizontal\u00a0<strong><em>(x) axis\u00a0<\/em><\/strong>and the frequency or number count of events on the vertical <strong><em>(y) axis<\/em><\/strong>. Clearly identified in the graph are transition points in the phases of the single system design\u2014when baseline ends and intervention begins, for example.\n<ul>\n \t<li><strong><em>x\u00a0<\/em><\/strong><strong>axis:\u00a0<\/strong>Time might be measured in minutes, hours, days, or weeks, depending on the natural frequency of the behavior targeted for change. For example, the frequency of facial and spontaneous vocalization tics might be measured in minutes or hours for a child experiencing Tourette\u2019s syndrome, while the frequency of a couple\u2019s arguments might be measured in days or weeks or the frequency of temptation to drink (alcohol) might be measured in hours or days.<\/li>\n \t<li><strong><em>y\u00a0<\/em><\/strong><strong>axis:\u00a0<\/strong>The scale used for the \u201ccount\u201d data running up the vertical axis is important to consider. A major factor involves the degree of variability in the values counted. If the lowest value is 0 and the highest value is in the hundreds, then it will be very difficult to discern differences of 5 or 15\u2014they will seem very close together in the scale. On the other hand, if the variability runs between 0 and 20, a difference of 5 or 15 will show up very well.<\/li>\n \t<li><strong>transition points:\u00a0<\/strong>One consistent approach is drawing a vertical line through the point in time when a transition occurs and labeling it. It is also possible to use color changes if the graph can be presented in a color format (some printers or copiers will lose this feature).<\/li>\n<\/ul>\nHere is an example of an ABAB single system design graph to consider (adapted from Prochaska, <a href=\"http:\/\/www.sjsu.edu\/people\/fred.prochaska\/courses\/ScWk240\/s1\/ScWk-240-Single-Case-Designs-Slides-Week-8.pdf\">www.sjsu.edu\/people\/fred.prochaska\/courses\/ScWk240\/s1\/ScWk-240-Single-Case-Designs-Slides-Week-8.pdf<\/a>).\n\n<img class=\"aligncenter size-full wp-image-225\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2019\/01\/Graph-of-Hypothetical-Outcome-of-ABAB-Design.png\" alt=\"Graph of Hypothetical Outcome of ABAB Design\" width=\"604\" height=\"545\">\n\nThe client and social worker together can see that both the level and variability of depression were high during the five weeks of baseline and there may have been an upward trend occurring at the point when intervention began (week 6). There was a noticeable downward trend in level of depression during most of the intervention period, with a plateau occurring around week 11. Without additional intervention during weeks 14 to 17, the client\u2019s level of depression did not worsen again, nor did it continue to improve\u2014it remained at the previous plateau level. With additional intervention, the level of depression again showed a downward trend until it leveled off at a low level during the final weeks of intervention.\n\nNote that it would be unusual to leave a person experiencing depression without intervention for 5 weeks of baseline data collection; however, this example was created as a hypothetical case. Also note that sometimes target behaviors show a degree of \u201cnatural\u201d improvement during baseline as a benefit of the screening and assessment process. For example, in a study of women experiencing alcohol dependency, investigators observed significant improvement in drinking behavior prior to the beginning of intervention, during assessment of their drinking problems:\n<blockquote><em>\u201cChanges in drinking frequency occurred at all four points in the pretreatment assessment process, resulting in 44% of the participants \u00a0 abstinent before the first session of treatment. A decrease in drinking quantity across the assessment period also was found\u201d (Epstein et al., 2005).\u00a0 <\/em><\/blockquote>\nThe authors concluded several points from these observations:\n<ul>\n \t<li>the assessment process itself might be therapeutic for some participants\u2014significant changes began with seeking treatment, before treatment started;<\/li>\n \t<li>interpreting intervention outcome results should be informed by this fact\u2014interventions should be evaluated in comparison to initial intake\/screening levels rather than assessment process data to detect the full impact of the entire experience of assessment and intervention combined;<\/li>\n \t<li>intervention outcome results should be informed by the observation that women who showed assessment phase improvement also showed the greatest outcomes during and following treatment (12-month follow-up).<\/li>\n<\/ul>\n<h2><strong>Assessing Change<\/strong><\/h2>\nAccording to Nugent\u2019s (2010) guide for analyzing single system design data, visual methods allow evaluators to detect several important types of changes or contrasts in the data for different study phases. Let\u2019s explore what these different kinds of information suggested by Nugent (2010) offer in understanding and evaluating social work interventions.\n<h3><strong>Level<\/strong><\/h3>\nPlotting the \u201clevels\u201d data is simply a matter of identifying the point in a graph where the \u201c<em>y\u201d<\/em>value for each \u201c<em>x<\/em>\u201d time point is located. This graph shows where the \u201cpoint\u201d for week 12 at a value of 40 would go on an ABA (baseline, intervention, remove intervention) single system design graph. Note that it is critically important that the units of time be of equal intervals. It would not work to have the baseline phase measured in days and the intervention phase in weeks, for example.\n\n<img class=\"aligncenter size-full wp-image-226\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/ABA-single-system-design-graph.png\" alt=\"ABA single system design graph\" width=\"670\" height=\"366\">\n\nLet\u2019s work with the concrete example concerning a client\u2019s depression levels in an ABAB single system design. Here, in week 1 the level was 5, in week 2 the level was 5.5, and so forth on a 10-point scale (0 to 9 values). This chart shows the data from the previous hypothetical depression levels example where you can see the levels reported for each of the 4 phases:\n<ul>\n \t<li>A<sub>1<\/sub>-weeks 1-5 (first baseline phase), levels range 5.0-6.0<\/li>\n \t<li>B<sub>1<\/sub>-weeks 6-13 (first intervention phase), levels range 2.5-4.5<\/li>\n \t<li>A<sub>2<\/sub>-weeks 14-17(removal of intervention phase), levels range 2.5-2.5<\/li>\n \t<li>B<sub>2<\/sub>-weeks 18-24 (second intervention phase), levels range 0.5-2.0<\/li>\n<\/ul>\n<img class=\"aligncenter size-full wp-image-227\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/hypothetical-ABAB-single-system-design-levels.png\" alt=\"hypothetical ABAB single system design levels\" width=\"899\" height=\"525\">\n<div class=\"textbox textbox--learning-objectives\"><header class=\"textbox__header\">\n<p class=\"textbox__title\">Interactive Excel Workbook Activities<\/p>\n\n<\/header>\n<div class=\"textbox__content\">\n\nComplete the following Workbook Activity:\n<ul>\n \t<li><a href=\"https:\/\/ohiostate.pressbooks.pub\/swk3401workbook\/chapter\/swk-3402-4-3-1-graphing-single-systems-design-data\/\">SWK 3402.4-3.1 Graphing Single Systems Design Data<\/a><\/li>\n<\/ul>\n<\/div>\n<\/div>\n<h3><strong>Trend<\/strong><\/h3>\nTrends are less about amount or level of change and more about the rate of change. Rate of change is indicated by the slope of the line between points in a graph, slope being the angle of the line. A \u201cflat\u201d line which runs horizontally, parallel to the x axis, indicates no change. The greater the angle, the steeper the slope, indicating greater degrees of change between points in time. Two methods for determining how the line should be placed on the graph are the statistical computation of the slope for a line connecting the data points and the \u201cNugent\u201d method (Engel &amp; Schutt, 2013). The statistical method called ordinary least squares computes the \u201cbest fitting\u201d line that has the shortest total of distances from each data point to the line. A simpler method resulting in similar conclusions is to simply draw a line between the first and last data points in each phase\u2014this is the \u201cNugent\u201d method (described by Engel &amp; Schutt, 2013, referring to the author of Nugent, 2010).\n\nLooking at data from our previous hypothetical depression levels data, we can see a couple of trends represented as orange arrow lines created using the Nugent method:\n\n<img class=\"aligncenter size-full wp-image-228\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.48.59-AM.png\" alt=\"hypothetical ABAB single system design trends\" width=\"707\" height=\"416\">\n\nAs you can see, the baseline trend was in an undesired direction, the trend during the two intervention phases was in a desired direction, and two \u201cflat\u201d constant periods indicated no change.\n<h3>Variability<\/h3>\nFluctuations in the dependent variable may not be consistent across phases of a study. For example, the baseline period might be characterized by wildly fluctuating values and the intervention might help tame the variability, creating greater stability in the behavior of concern. See, for example the differences in the initial baseline (A<sub>1<\/sub>) phase compared to the post-intervention phase without intervention (A<sub>2<\/sub>). Variability looks different from a unidirectional trend in that it involves up-and-down fluctuations.\n\n<img class=\"aligncenter size-full wp-image-229\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.51.17-AM.png\" alt=\"Hypothetical ABAB single system design: variability\" width=\"710\" height=\"422\">\n<h3><strong>Overlap<\/strong><\/h3>\nThe concept of overlap has to do with the range of levels observed in different phases. This is depicted by drawing lines through the top and bottom range values observed in each phase, then seeing if the ranges overlap or are separated. The greater the width of overlap area, the less evident the degree of change; smaller areas of overlap meant change is more evident (Nugent, 2010). Here is what overlap mapping looks like in our hypothetical levels of depression example. There is no overlap between the baseline range and any of the other phases; some overlap between the first intervention phase and the removal of intervention phase; and, no overlap between the second intervention phase and any of the other phases.\n\n<img class=\"aligncenter size-full wp-image-230\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.52.15-AM.png\" alt=\"hypothetical ABAB single system design: overlap\" width=\"708\" height=\"408\">\n<h3><strong>Means &amp; medians<\/strong><\/h3>\nOne strategy for detecting differences in data from different phases of the single system design study is to compute the mean and\/or median of the values in each phase, then draw a line through the phase at each of those points. From our prior coursework, you know how to compute the mean (average) and the median (the 50<sup>th<\/sup> percentile, where \u00bd are above and \u00bd are below that value). Excel software can compute both of these values in very short order.\n\nIn our depression levels example, this is what the values would be:\n\n<img class=\"aligncenter size-full wp-image-231\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.53.22-AM.png\" alt=\"depression levels table\" width=\"419\" height=\"112\">\n\nHere is an example of what the graph might look like with the mean lines drawn in, using the data from our hypothetical depression levels ABAB design example. As you can see, there is progressive improvement in levels of depression across the 4 phases of the study design. Later we will look at how to analyze these differences for statistical significance.\n\n<img class=\"aligncenter size-full wp-image-232\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.53.55-AM-1.png\" alt=\"hypothetical ABAB single system design means\" width=\"698\" height=\"413\">\n\nOne thing to keep in mind, based on what you know about how \u201cextreme\u201d values influence the mean, is that the median might be a better tool to rely on when there exists a great deal of variability in the range of values (Engel &amp; Schutt, 2013). Also Keep in mind that clinical significance is of great importance in analyzing results of the intervention: a difference from 5.4 to 0.5 could be of great clinical significance in an individual\u2019s experience of depression, depending on what the 0-9 scale implies!\u00a0Engel and Schutt (2013) suggest three means of determining clinical significance in these situations:\n<ul>\n \t<li><em>Establish criteria:<\/em>prior to engaging in the evaluation effort, determine what might be relevant criteria for success. \u201cIf the intervention reaches that point, then the change is meaningful\u201d (Engel &amp; Schutt, 2013, p. 202). The criteria could be defined in terms of a specified degree, amount, or level of change. These values should be informed by the literature concerning the measure used.<\/li>\n \t<li><em>Use established cut point scores:<\/em>if the outcome is measured using clinical tools with established norms and\/or cut point scores, determine \u201cwhether the intervention has reduced the problem to a level below a clinical cut-off score\u201d (Engel &amp; Schutt, 2013, p. 202). Thus, in determining clinical significance, the outcome (dependent) variable becomes a dichotomous categorical variable\u2014above or below the clinical cut point.<\/li>\n \t<li><em>Weigh costs and benefits:<\/em>in this case, the goal is to determine whether the efforts to produce change are resulting in sufficient change to \u201cbe worth the cost and effort to produce the improvement\u201d observed (Engel &amp; Schutt, 2013, p. 204).<\/li>\n<\/ul>\n<h3><strong>Latency\/immediacy of change<\/strong><\/h3>\nOne other dimension that Nugent (2010) advises considering has to do with how long it takes before change is evident. In some cases, <strong><em>latency<\/em><\/strong>is short\u2014the beginning of change attributable to intervention are almost immediately observed. In other instances, latency might be longer\u2014it takes a while before the impact is beginning to be observed in the data. For example, changes in knowledge about a topic might happen quickly with intervention but changes in attitudes, values, and beliefs may take longer to appear. This is particularly true of complex behaviors. In our example, changes with intervention seemed to take a relatively short time since we observed change beginning in the week of intervention implementation each time\u2014latency was short.\n<h2><strong>Statistical Analysis of Single System Design Data<\/strong><\/h2>\nGraphs are quite useful in helping social work professionals and their clients interpret single system design data. However, precision interpretations are supported by statistical analysis of these data. Specifically, when there appear to be changes between phases, how do we know if the observed changes are meaningful? A variety of statistical approaches are described in the literature. Two approaches are presented in this chapter, based on three criteria to consider in selecting analytic approaches adapted from six criteria described by Manalov et al (2016). The statistical approach selected for analysis should:\n<ol>\n \t<li>be simple to compute and interpret without a high degree of statistical expertise and present a reduced likelihood of misinterpretation;<\/li>\n \t<li>complement (rather than duplicate) visual analysis, especially when trend and variability patterns complicate visual analysis, providing different information;<\/li>\n \t<li>be free from assumptions of data independence (longitudinally) or homoscedasticity (equal variance).<\/li>\n<\/ol>\nThus, the two approaches introduced in this chapter are NAP analysis (non-overlap of all pairs) and the two-standard deviation band approach.\n<h3><strong>Approach #1:\u00a0 Non-overlap of all pairs (NAP) analysis. <\/strong><\/h3>\nThis approach is essentially a non-parametric form of analysis based on probability: the Mann-Whitney U-test. <strong><em>Nonoverlap of all pair (NAP) analysis\u00a0<\/em><\/strong>is about pairs of observations for two different phases in a single system study being in the desired direction. The \u201call pair\u201d aspect in the title of the approach means that every observation in the time 2 data is paired to each of the observations in the time 1 data. This means that the number of pairs compared is the number of observations in time 1 multiplied by the number of observations in time 2. The probability default in this analysis is that no difference exists between two phases of single system data being compared. If \u201cimprovement\u201d means that the observed frequency or amount decreases (such as number of arguments, need for time out, cigarettes smoked, or anxiety scores), then improvement means the \u201ctime 1\u201d data will be greater than the \u201ctime 2\u201d data more often. The null hypothesis is that there is no difference in the number of times that the paired values reflect improvement compared to the number that reflect no improvement.\n\nTo make easier to understand, imagine that we wish to compare the baseline (A<sub>1<\/sub>) to the intervention phase (B) in our depression levels example. In this case we had 5 baseline observations and 8 intervention phase observations:\n<table style=\"border-collapse: collapse;width: 100%\" border=\"0\">\n<thead>\n<tr>\n<td style=\"width: 33.333333333333336%\">A<sub>1<\/sub> phase<\/td>\n<td style=\"width: 33.333333333333336%\">B phase<\/td>\n<td style=\"width: 33.333333333333336%\">A<sub>2<\/sub> phase<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td style=\"width: 33.333333333333336%\">5<\/td>\n<td style=\"width: 33.333333333333336%\">4.5<\/td>\n<td style=\"width: 33.333333333333336%\">2<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 33.333333333333336%\">5.5<\/td>\n<td style=\"width: 33.333333333333336%\">4<\/td>\n<td style=\"width: 33.333333333333336%\">1.5<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 33.333333333333336%\">5<\/td>\n<td style=\"width: 33.333333333333336%\">3.5<\/td>\n<td style=\"width: 33.333333333333336%\">1<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 33.333333333333336%\">5.5<\/td>\n<td style=\"width: 33.333333333333336%\">3<\/td>\n<td style=\"width: 33.333333333333336%\">0.5<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 33.333333333333336%\">6<\/td>\n<td style=\"width: 33.333333333333336%\">3<\/td>\n<td style=\"width: 33.333333333333336%\">0.5<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 33.333333333333336%\"><\/td>\n<td style=\"width: 33.333333333333336%\">2.5<\/td>\n<td style=\"width: 33.333333333333336%\">0.5<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 33.333333333333336%\"><\/td>\n<td style=\"width: 33.333333333333336%\">2.5<\/td>\n<td style=\"width: 33.333333333333336%\">0.5<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 33.333333333333336%\"><\/td>\n<td style=\"width: 33.333333333333336%\">2.5<\/td>\n<td style=\"width: 33.333333333333336%\"><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\nThus, we have a total of 40 pairs of data (5 x 8=40). This can be hand-calculated for a small data set using a somewhat complex formula to compute the probability. Or, the data can be entered in 2 columns in a calculator program at <a href=\"http:\/\/www.singlecaseresearch.or\/calculatros\/nap\">www.singlecaseresearch.org\/calculators\/nap<\/a>\u00a0(nap is an abbreviation for nonoverlap of all pairs; the calculator is attributed to Vannest, Parker, Gonen, &amp; Adiguzel, 2016). In this example, it would look like this after clicking on \u201ccontrast\u201d for the comparison of A vs B phases:\n\n<img class=\"aligncenter size-full wp-image-233\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/single-case-researcg-nap-calculator-example.png\" alt=\"single case researcg nap calculator example\" width=\"918\" height=\"541\">\n\nThe <em>p<\/em>-value for the <em>Z\u00a0<\/em>test-statistic (-2.93) for our 40 pairs was 0.0034; our <em>p<\/em>&lt;.05 allows us to reject the null hypothesis of no difference. In other words, there appears to be a statistically significant improvement between the intervention (B) phase and the baseline (A<sub>1<\/sub>) phase.\n\nSimilarly, if we wish to compare the intervention (B) phase to the post-intervention (A<sub>2<\/sub>) phase, we have a total of 56 possible pairs (8 x 7=56). Entering the data into the NAP calculator program we obtain the following result:\n\n<img class=\"aligncenter size-full wp-image-234\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/single-case-researcg-nap-calculator-example-results.png\" alt=\"single case researcg nap calculator example results\" width=\"958\" height=\"566\">\n\nOnce again, we observe that <em>p<\/em>&lt;.05 (<em>Z\u00a0<\/em>statistic for our 56 pairs is -3.24, <em>p<\/em>=0.0012). Therefore, we can again reject the null hypothesis of no difference and conclude that the changes continued in a downward direction during the post-intervention period.\n<h3><strong>Approach #2: Two-standard deviation band method<\/strong><\/h3>\nThis second approach is also valued the simplicity of computations involved (Orme &amp; Cox, 2001). It is called the <strong><em>two-standard deviation band method\u00a0<\/em><\/strong>because it involves computing the mean and standard deviation from the baseline phase data, drawing the \u201cband\u201d representing two standard deviations (2 <em>SD<\/em>s) around the baseline mean onto the graph, and determining where the intervention phase data points fall in comparison to that \u201cband.\u201d The band includes values that would be 2 standard deviations (<em>SD<\/em>s) above and 2 standard deviations below the mean for the phase. Another name for this \u201cbanded\u201d graph is a Shewhart chart (Orme &amp; Cox, 2001).\n\nA rule of thumb offered by Gottman and Leiblum (1974) is that if at least two consecutive intervention phase data points fall outside the band, a meaningful change was observed. The logic for this rule of thumb is that the probability of this happening by chance is less than the criterion of <em>p<\/em>&lt;.05 (Nourbakhsh &amp; Ottenbacher, 1994). Here is what it would look like for the hypothetical depression levels example with which we have been working throughout this chapter.\n\nFirst, compute the mean and standard deviation for the baseline (A<sub>1<\/sub>) phase data. Then, compute the values for the mean \u00b1 <em>SD<\/em> (mean plus SD and mean minus <em>SD<\/em>). Using Excel, we find the following:\n<p style=\"padding-left: 30px\"><em>M<\/em>=5.4<\/p>\n<p style=\"padding-left: 30px\"><em>SD<\/em>=0.42<\/p>\n<p style=\"padding-left: 30px\">2 <em>SD<\/em> band=(4.98, 5.82)<\/p>\nDrawn onto the original levels graph, the two-standard deviation band graph would appear as:\n\n<img class=\"aligncenter size-full wp-image-235\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-11.00.59-AM.png\" alt=\"hypothetical ABAB single system design levels\" width=\"701\" height=\"414\">\n\nAs you can see, more than two (in fact, all) of the intervention phase data points fall outside of the two-standard deviation band width. The conclusion would be that a significant degree of change has occurred between these two phases, baseline (A<sub>1<\/sub>) and intervention (B). To see this in action, work the exercise presented in your Excel Workbook.\n<div class=\"textbox textbox--learning-objectives\"><header class=\"textbox__header\">\n<p class=\"textbox__title\">Interactive Excel Workbook Activities<\/p>\n\n<\/header>\n<div class=\"textbox__content\">\n\nComplete the following Workbook Activity:\n<ul>\n \t<li><a href=\"https:\/\/ohiostate.pressbooks.pub\/swk3401workbook\/chapter\/swk-3402-4-3-2-analyzing-single-systems-design-data\/\">SWK 3402.4-3.2 Analyzing Single Systems Design Data<\/a><\/li>\n<\/ul>\n<\/div>\n<\/div>\n<h2>Stop and Think<\/h2>\n<div style=\"float: left;min-height: 120px;width: 99%;margin-bottom: 10px;padding: 10px;background-color: #f1f7fe\">\n\n<img class=\"size-full wp-image-194 alignleft\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Stop_ThinkV2-150x150-1.png\" alt=\"Stop and Think\" width=\"150\" height=\"150\">Take a moment to complete the following activity.\n\n[h5p id=\"13\"]\n\n<\/div>\n<h2><strong>Chapter Summary<\/strong><\/h2>\nThis chapter concerned approaches to analyzing data collected through single system design efforts to evaluate intervention outcomes. First, you learned about the visual and graphing approaches. You learned how social work professionals and clients can use levels, trends, variability, overlap, means, medians, persistence, and latency information in interpreting their outcomes data. In addition, you learned two statistical analytic approaches that might be used to determine the significance of observed changes in single system design data. The non-overlap of all pairs (NAP) analysis utilizes a form of nonparametic analytic logic to determine if a null hypothesis of no meaningful change should be rejected\u2014the test statistic was a <em>Z\u00a0<\/em>score (the calculation and distribution of which we have not previously studied), and the criterion of <em>p<\/em>&lt;.05 remained useful to informing the decision. Another easily computed statistic you learned about was the two-standard deviation band analysis which entails calculating the mean and standard deviation for baseline data, then computing the range of values encompassing two standard deviations above and below that mean. The decision concerning meaningful levels of change is based on whether there are at least two consecutive data points in the intervention phase that fall outside of the calculated two standard deviations band. This can be visualized on a graph, but does not need to be\u2014it can be determined based on reviewing a table of values once the band width has been computed. This chapter concludes our module on analyzing data for understanding social work interventions.\n\n&nbsp;","rendered":"<p>As you learned in the previous module, one approach to evaluating social work practice involves collecting data using a single system design. Many of the approaches we have studied are about what happens with different groups of participants. The single system design is sometimes called \u201cintrasubject\u201d research: the data are related to what happens within an individual system (Nugent, 2010). Single system designs generally involve collecting baseline data repeatedly for a period prior to implementing an intervention (the \u201cA\u201d phase) and collecting data during the intervention period (the \u201cB\u201d phase). A common variation on this design include collecting data during a period when the intervention is removed (another \u201cA\u201d phase), hence the ABA designation. Many other variations exist including adding and comparing alternative interventions (B<sub>1<\/sub>, B<sub>2<\/sub>, B<sub>3<\/sub> for example) and collecting data during each of those phases. The issue in this current chapter is what to do with the collected data, how to analyze it for meaning.<\/p>\n<p>In this chapter you learn:<\/p>\n<ul>\n<li>visual graphic approaches to analyzing single system design data;<\/li>\n<li>assessing changes between single system design phases in level, trend, variability, overlap, means, persistence, and latency of change;<\/li>\n<li>two statistical analysis approaches for complementing visual approaches (non-overlap of all pairs analysis and two-standard deviation band analysis).<\/li>\n<\/ul>\n<h2><strong>Graphing Single System Design Data<\/strong><\/h2>\n<p>The first step in analyzing single-system design data is the visual analysis. According to Engel and Schutt (2013), visual examination of the graphed data is \u201cthe most common method\u201d of analysis (p. 200). The purpose is to determine whether the target variable (outcome) changed between the baseline and intervention phases. Let\u2019s begin by examining the elements of the single system design graph.<\/p>\n<p>First, the phase of a single study design can be considered as the <strong><em>independent variable\u00a0<\/em><\/strong>(the one that is manipulated experimentally) and the measured outcome as the <strong><em>dependent variable<\/em><\/strong>. Social work interventions often have multiple goals or objectives\u2014perhaps addressing multiple behaviors, or a behavior and an attitude, or knowledge on several topics. For the purposes of simplifying the understanding single system design data, we will focus on data measuring only one outcome variable at a time.<\/p>\n<p>The single system design graph has \u201ctime\u201d as the horizontal\u00a0<strong><em>(x) axis\u00a0<\/em><\/strong>and the frequency or number count of events on the vertical <strong><em>(y) axis<\/em><\/strong>. Clearly identified in the graph are transition points in the phases of the single system design\u2014when baseline ends and intervention begins, for example.<\/p>\n<ul>\n<li><strong><em>x\u00a0<\/em><\/strong><strong>axis:\u00a0<\/strong>Time might be measured in minutes, hours, days, or weeks, depending on the natural frequency of the behavior targeted for change. For example, the frequency of facial and spontaneous vocalization tics might be measured in minutes or hours for a child experiencing Tourette\u2019s syndrome, while the frequency of a couple\u2019s arguments might be measured in days or weeks or the frequency of temptation to drink (alcohol) might be measured in hours or days.<\/li>\n<li><strong><em>y\u00a0<\/em><\/strong><strong>axis:\u00a0<\/strong>The scale used for the \u201ccount\u201d data running up the vertical axis is important to consider. A major factor involves the degree of variability in the values counted. If the lowest value is 0 and the highest value is in the hundreds, then it will be very difficult to discern differences of 5 or 15\u2014they will seem very close together in the scale. On the other hand, if the variability runs between 0 and 20, a difference of 5 or 15 will show up very well.<\/li>\n<li><strong>transition points:\u00a0<\/strong>One consistent approach is drawing a vertical line through the point in time when a transition occurs and labeling it. It is also possible to use color changes if the graph can be presented in a color format (some printers or copiers will lose this feature).<\/li>\n<\/ul>\n<p>Here is an example of an ABAB single system design graph to consider (adapted from Prochaska, <a href=\"http:\/\/www.sjsu.edu\/people\/fred.prochaska\/courses\/ScWk240\/s1\/ScWk-240-Single-Case-Designs-Slides-Week-8.pdf\">www.sjsu.edu\/people\/fred.prochaska\/courses\/ScWk240\/s1\/ScWk-240-Single-Case-Designs-Slides-Week-8.pdf<\/a>).<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-225\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2019\/01\/Graph-of-Hypothetical-Outcome-of-ABAB-Design.png\" alt=\"Graph of Hypothetical Outcome of ABAB Design\" width=\"604\" height=\"545\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2019\/01\/Graph-of-Hypothetical-Outcome-of-ABAB-Design.png 604w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2019\/01\/Graph-of-Hypothetical-Outcome-of-ABAB-Design-300x271.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2019\/01\/Graph-of-Hypothetical-Outcome-of-ABAB-Design-65x59.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2019\/01\/Graph-of-Hypothetical-Outcome-of-ABAB-Design-225x203.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2019\/01\/Graph-of-Hypothetical-Outcome-of-ABAB-Design-350x316.png 350w\" sizes=\"auto, (max-width: 604px) 100vw, 604px\" \/><\/p>\n<p>The client and social worker together can see that both the level and variability of depression were high during the five weeks of baseline and there may have been an upward trend occurring at the point when intervention began (week 6). There was a noticeable downward trend in level of depression during most of the intervention period, with a plateau occurring around week 11. Without additional intervention during weeks 14 to 17, the client\u2019s level of depression did not worsen again, nor did it continue to improve\u2014it remained at the previous plateau level. With additional intervention, the level of depression again showed a downward trend until it leveled off at a low level during the final weeks of intervention.<\/p>\n<p>Note that it would be unusual to leave a person experiencing depression without intervention for 5 weeks of baseline data collection; however, this example was created as a hypothetical case. Also note that sometimes target behaviors show a degree of \u201cnatural\u201d improvement during baseline as a benefit of the screening and assessment process. For example, in a study of women experiencing alcohol dependency, investigators observed significant improvement in drinking behavior prior to the beginning of intervention, during assessment of their drinking problems:<\/p>\n<blockquote><p><em>\u201cChanges in drinking frequency occurred at all four points in the pretreatment assessment process, resulting in 44% of the participants \u00a0 abstinent before the first session of treatment. A decrease in drinking quantity across the assessment period also was found\u201d (Epstein et al., 2005).\u00a0 <\/em><\/p><\/blockquote>\n<p>The authors concluded several points from these observations:<\/p>\n<ul>\n<li>the assessment process itself might be therapeutic for some participants\u2014significant changes began with seeking treatment, before treatment started;<\/li>\n<li>interpreting intervention outcome results should be informed by this fact\u2014interventions should be evaluated in comparison to initial intake\/screening levels rather than assessment process data to detect the full impact of the entire experience of assessment and intervention combined;<\/li>\n<li>intervention outcome results should be informed by the observation that women who showed assessment phase improvement also showed the greatest outcomes during and following treatment (12-month follow-up).<\/li>\n<\/ul>\n<h2><strong>Assessing Change<\/strong><\/h2>\n<p>According to Nugent\u2019s (2010) guide for analyzing single system design data, visual methods allow evaluators to detect several important types of changes or contrasts in the data for different study phases. Let\u2019s explore what these different kinds of information suggested by Nugent (2010) offer in understanding and evaluating social work interventions.<\/p>\n<h3><strong>Level<\/strong><\/h3>\n<p>Plotting the \u201clevels\u201d data is simply a matter of identifying the point in a graph where the \u201c<em>y\u201d<\/em>value for each \u201c<em>x<\/em>\u201d time point is located. This graph shows where the \u201cpoint\u201d for week 12 at a value of 40 would go on an ABA (baseline, intervention, remove intervention) single system design graph. Note that it is critically important that the units of time be of equal intervals. It would not work to have the baseline phase measured in days and the intervention phase in weeks, for example.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-226\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/ABA-single-system-design-graph.png\" alt=\"ABA single system design graph\" width=\"670\" height=\"366\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/ABA-single-system-design-graph.png 670w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/ABA-single-system-design-graph-300x164.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/ABA-single-system-design-graph-65x36.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/ABA-single-system-design-graph-225x123.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/ABA-single-system-design-graph-350x191.png 350w\" sizes=\"auto, (max-width: 670px) 100vw, 670px\" \/><\/p>\n<p>Let\u2019s work with the concrete example concerning a client\u2019s depression levels in an ABAB single system design. Here, in week 1 the level was 5, in week 2 the level was 5.5, and so forth on a 10-point scale (0 to 9 values). This chart shows the data from the previous hypothetical depression levels example where you can see the levels reported for each of the 4 phases:<\/p>\n<ul>\n<li>A<sub>1<\/sub>-weeks 1-5 (first baseline phase), levels range 5.0-6.0<\/li>\n<li>B<sub>1<\/sub>-weeks 6-13 (first intervention phase), levels range 2.5-4.5<\/li>\n<li>A<sub>2<\/sub>-weeks 14-17(removal of intervention phase), levels range 2.5-2.5<\/li>\n<li>B<sub>2<\/sub>-weeks 18-24 (second intervention phase), levels range 0.5-2.0<\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-227\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/hypothetical-ABAB-single-system-design-levels.png\" alt=\"hypothetical ABAB single system design levels\" width=\"899\" height=\"525\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/hypothetical-ABAB-single-system-design-levels.png 899w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/hypothetical-ABAB-single-system-design-levels-300x175.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/hypothetical-ABAB-single-system-design-levels-768x448.png 768w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/hypothetical-ABAB-single-system-design-levels-65x38.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/hypothetical-ABAB-single-system-design-levels-225x131.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/hypothetical-ABAB-single-system-design-levels-350x204.png 350w\" sizes=\"auto, (max-width: 899px) 100vw, 899px\" \/><\/p>\n<div class=\"textbox textbox--learning-objectives\">\n<header class=\"textbox__header\">\n<p class=\"textbox__title\">Interactive Excel Workbook Activities<\/p>\n<\/header>\n<div class=\"textbox__content\">\n<p>Complete the following Workbook Activity:<\/p>\n<ul>\n<li><a href=\"https:\/\/ohiostate.pressbooks.pub\/swk3401workbook\/chapter\/swk-3402-4-3-1-graphing-single-systems-design-data\/\">SWK 3402.4-3.1 Graphing Single Systems Design Data<\/a><\/li>\n<\/ul>\n<\/div>\n<\/div>\n<h3><strong>Trend<\/strong><\/h3>\n<p>Trends are less about amount or level of change and more about the rate of change. Rate of change is indicated by the slope of the line between points in a graph, slope being the angle of the line. A \u201cflat\u201d line which runs horizontally, parallel to the x axis, indicates no change. The greater the angle, the steeper the slope, indicating greater degrees of change between points in time. Two methods for determining how the line should be placed on the graph are the statistical computation of the slope for a line connecting the data points and the \u201cNugent\u201d method (Engel &amp; Schutt, 2013). The statistical method called ordinary least squares computes the \u201cbest fitting\u201d line that has the shortest total of distances from each data point to the line. A simpler method resulting in similar conclusions is to simply draw a line between the first and last data points in each phase\u2014this is the \u201cNugent\u201d method (described by Engel &amp; Schutt, 2013, referring to the author of Nugent, 2010).<\/p>\n<p>Looking at data from our previous hypothetical depression levels data, we can see a couple of trends represented as orange arrow lines created using the Nugent method:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-228\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.48.59-AM.png\" alt=\"hypothetical ABAB single system design trends\" width=\"707\" height=\"416\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.48.59-AM.png 707w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.48.59-AM-300x177.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.48.59-AM-65x38.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.48.59-AM-225x132.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.48.59-AM-350x206.png 350w\" sizes=\"auto, (max-width: 707px) 100vw, 707px\" \/><\/p>\n<p>As you can see, the baseline trend was in an undesired direction, the trend during the two intervention phases was in a desired direction, and two \u201cflat\u201d constant periods indicated no change.<\/p>\n<h3>Variability<\/h3>\n<p>Fluctuations in the dependent variable may not be consistent across phases of a study. For example, the baseline period might be characterized by wildly fluctuating values and the intervention might help tame the variability, creating greater stability in the behavior of concern. See, for example the differences in the initial baseline (A<sub>1<\/sub>) phase compared to the post-intervention phase without intervention (A<sub>2<\/sub>). Variability looks different from a unidirectional trend in that it involves up-and-down fluctuations.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-229\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.51.17-AM.png\" alt=\"Hypothetical ABAB single system design: variability\" width=\"710\" height=\"422\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.51.17-AM.png 710w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.51.17-AM-300x178.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.51.17-AM-65x39.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.51.17-AM-225x134.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.51.17-AM-350x208.png 350w\" sizes=\"auto, (max-width: 710px) 100vw, 710px\" \/><\/p>\n<h3><strong>Overlap<\/strong><\/h3>\n<p>The concept of overlap has to do with the range of levels observed in different phases. This is depicted by drawing lines through the top and bottom range values observed in each phase, then seeing if the ranges overlap or are separated. The greater the width of overlap area, the less evident the degree of change; smaller areas of overlap meant change is more evident (Nugent, 2010). Here is what overlap mapping looks like in our hypothetical levels of depression example. There is no overlap between the baseline range and any of the other phases; some overlap between the first intervention phase and the removal of intervention phase; and, no overlap between the second intervention phase and any of the other phases.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-230\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.52.15-AM.png\" alt=\"hypothetical ABAB single system design: overlap\" width=\"708\" height=\"408\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.52.15-AM.png 708w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.52.15-AM-300x173.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.52.15-AM-65x37.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.52.15-AM-225x130.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.52.15-AM-350x202.png 350w\" sizes=\"auto, (max-width: 708px) 100vw, 708px\" \/><\/p>\n<h3><strong>Means &amp; medians<\/strong><\/h3>\n<p>One strategy for detecting differences in data from different phases of the single system design study is to compute the mean and\/or median of the values in each phase, then draw a line through the phase at each of those points. From our prior coursework, you know how to compute the mean (average) and the median (the 50<sup>th<\/sup> percentile, where \u00bd are above and \u00bd are below that value). Excel software can compute both of these values in very short order.<\/p>\n<p>In our depression levels example, this is what the values would be:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-231\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.53.22-AM.png\" alt=\"depression levels table\" width=\"419\" height=\"112\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.53.22-AM.png 419w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.53.22-AM-300x80.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.53.22-AM-65x17.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.53.22-AM-225x60.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.53.22-AM-350x94.png 350w\" sizes=\"auto, (max-width: 419px) 100vw, 419px\" \/><\/p>\n<p>Here is an example of what the graph might look like with the mean lines drawn in, using the data from our hypothetical depression levels ABAB design example. As you can see, there is progressive improvement in levels of depression across the 4 phases of the study design. Later we will look at how to analyze these differences for statistical significance.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-232\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.53.55-AM-1.png\" alt=\"hypothetical ABAB single system design means\" width=\"698\" height=\"413\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.53.55-AM-1.png 698w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.53.55-AM-1-300x178.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.53.55-AM-1-65x38.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.53.55-AM-1-225x133.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-10.53.55-AM-1-350x207.png 350w\" sizes=\"auto, (max-width: 698px) 100vw, 698px\" \/><\/p>\n<p>One thing to keep in mind, based on what you know about how \u201cextreme\u201d values influence the mean, is that the median might be a better tool to rely on when there exists a great deal of variability in the range of values (Engel &amp; Schutt, 2013). Also Keep in mind that clinical significance is of great importance in analyzing results of the intervention: a difference from 5.4 to 0.5 could be of great clinical significance in an individual\u2019s experience of depression, depending on what the 0-9 scale implies!\u00a0Engel and Schutt (2013) suggest three means of determining clinical significance in these situations:<\/p>\n<ul>\n<li><em>Establish criteria:<\/em>prior to engaging in the evaluation effort, determine what might be relevant criteria for success. \u201cIf the intervention reaches that point, then the change is meaningful\u201d (Engel &amp; Schutt, 2013, p. 202). The criteria could be defined in terms of a specified degree, amount, or level of change. These values should be informed by the literature concerning the measure used.<\/li>\n<li><em>Use established cut point scores:<\/em>if the outcome is measured using clinical tools with established norms and\/or cut point scores, determine \u201cwhether the intervention has reduced the problem to a level below a clinical cut-off score\u201d (Engel &amp; Schutt, 2013, p. 202). Thus, in determining clinical significance, the outcome (dependent) variable becomes a dichotomous categorical variable\u2014above or below the clinical cut point.<\/li>\n<li><em>Weigh costs and benefits:<\/em>in this case, the goal is to determine whether the efforts to produce change are resulting in sufficient change to \u201cbe worth the cost and effort to produce the improvement\u201d observed (Engel &amp; Schutt, 2013, p. 204).<\/li>\n<\/ul>\n<h3><strong>Latency\/immediacy of change<\/strong><\/h3>\n<p>One other dimension that Nugent (2010) advises considering has to do with how long it takes before change is evident. In some cases, <strong><em>latency<\/em><\/strong>is short\u2014the beginning of change attributable to intervention are almost immediately observed. In other instances, latency might be longer\u2014it takes a while before the impact is beginning to be observed in the data. For example, changes in knowledge about a topic might happen quickly with intervention but changes in attitudes, values, and beliefs may take longer to appear. This is particularly true of complex behaviors. In our example, changes with intervention seemed to take a relatively short time since we observed change beginning in the week of intervention implementation each time\u2014latency was short.<\/p>\n<h2><strong>Statistical Analysis of Single System Design Data<\/strong><\/h2>\n<p>Graphs are quite useful in helping social work professionals and their clients interpret single system design data. However, precision interpretations are supported by statistical analysis of these data. Specifically, when there appear to be changes between phases, how do we know if the observed changes are meaningful? A variety of statistical approaches are described in the literature. Two approaches are presented in this chapter, based on three criteria to consider in selecting analytic approaches adapted from six criteria described by Manalov et al (2016). The statistical approach selected for analysis should:<\/p>\n<ol>\n<li>be simple to compute and interpret without a high degree of statistical expertise and present a reduced likelihood of misinterpretation;<\/li>\n<li>complement (rather than duplicate) visual analysis, especially when trend and variability patterns complicate visual analysis, providing different information;<\/li>\n<li>be free from assumptions of data independence (longitudinally) or homoscedasticity (equal variance).<\/li>\n<\/ol>\n<p>Thus, the two approaches introduced in this chapter are NAP analysis (non-overlap of all pairs) and the two-standard deviation band approach.<\/p>\n<h3><strong>Approach #1:\u00a0 Non-overlap of all pairs (NAP) analysis. <\/strong><\/h3>\n<p>This approach is essentially a non-parametric form of analysis based on probability: the Mann-Whitney U-test. <strong><em>Nonoverlap of all pair (NAP) analysis\u00a0<\/em><\/strong>is about pairs of observations for two different phases in a single system study being in the desired direction. The \u201call pair\u201d aspect in the title of the approach means that every observation in the time 2 data is paired to each of the observations in the time 1 data. This means that the number of pairs compared is the number of observations in time 1 multiplied by the number of observations in time 2. The probability default in this analysis is that no difference exists between two phases of single system data being compared. If \u201cimprovement\u201d means that the observed frequency or amount decreases (such as number of arguments, need for time out, cigarettes smoked, or anxiety scores), then improvement means the \u201ctime 1\u201d data will be greater than the \u201ctime 2\u201d data more often. The null hypothesis is that there is no difference in the number of times that the paired values reflect improvement compared to the number that reflect no improvement.<\/p>\n<p>To make easier to understand, imagine that we wish to compare the baseline (A<sub>1<\/sub>) to the intervention phase (B) in our depression levels example. In this case we had 5 baseline observations and 8 intervention phase observations:<\/p>\n<table style=\"border-collapse: collapse;width: 100%\">\n<thead>\n<tr>\n<td style=\"width: 33.333333333333336%\">A<sub>1<\/sub> phase<\/td>\n<td style=\"width: 33.333333333333336%\">B phase<\/td>\n<td style=\"width: 33.333333333333336%\">A<sub>2<\/sub> phase<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td style=\"width: 33.333333333333336%\">5<\/td>\n<td style=\"width: 33.333333333333336%\">4.5<\/td>\n<td style=\"width: 33.333333333333336%\">2<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 33.333333333333336%\">5.5<\/td>\n<td style=\"width: 33.333333333333336%\">4<\/td>\n<td style=\"width: 33.333333333333336%\">1.5<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 33.333333333333336%\">5<\/td>\n<td style=\"width: 33.333333333333336%\">3.5<\/td>\n<td style=\"width: 33.333333333333336%\">1<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 33.333333333333336%\">5.5<\/td>\n<td style=\"width: 33.333333333333336%\">3<\/td>\n<td style=\"width: 33.333333333333336%\">0.5<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 33.333333333333336%\">6<\/td>\n<td style=\"width: 33.333333333333336%\">3<\/td>\n<td style=\"width: 33.333333333333336%\">0.5<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 33.333333333333336%\"><\/td>\n<td style=\"width: 33.333333333333336%\">2.5<\/td>\n<td style=\"width: 33.333333333333336%\">0.5<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 33.333333333333336%\"><\/td>\n<td style=\"width: 33.333333333333336%\">2.5<\/td>\n<td style=\"width: 33.333333333333336%\">0.5<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 33.333333333333336%\"><\/td>\n<td style=\"width: 33.333333333333336%\">2.5<\/td>\n<td style=\"width: 33.333333333333336%\"><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Thus, we have a total of 40 pairs of data (5 x 8=40). This can be hand-calculated for a small data set using a somewhat complex formula to compute the probability. Or, the data can be entered in 2 columns in a calculator program at <a href=\"http:\/\/www.singlecaseresearch.or\/calculatros\/nap\">www.singlecaseresearch.org\/calculators\/nap<\/a>\u00a0(nap is an abbreviation for nonoverlap of all pairs; the calculator is attributed to Vannest, Parker, Gonen, &amp; Adiguzel, 2016). In this example, it would look like this after clicking on \u201ccontrast\u201d for the comparison of A vs B phases:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-233\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/single-case-researcg-nap-calculator-example.png\" alt=\"single case researcg nap calculator example\" width=\"918\" height=\"541\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/single-case-researcg-nap-calculator-example.png 918w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/single-case-researcg-nap-calculator-example-300x177.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/single-case-researcg-nap-calculator-example-768x453.png 768w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/single-case-researcg-nap-calculator-example-65x38.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/single-case-researcg-nap-calculator-example-225x133.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/single-case-researcg-nap-calculator-example-350x206.png 350w\" sizes=\"auto, (max-width: 918px) 100vw, 918px\" \/><\/p>\n<p>The <em>p<\/em>-value for the <em>Z\u00a0<\/em>test-statistic (-2.93) for our 40 pairs was 0.0034; our <em>p<\/em>&lt;.05 allows us to reject the null hypothesis of no difference. In other words, there appears to be a statistically significant improvement between the intervention (B) phase and the baseline (A<sub>1<\/sub>) phase.<\/p>\n<p>Similarly, if we wish to compare the intervention (B) phase to the post-intervention (A<sub>2<\/sub>) phase, we have a total of 56 possible pairs (8 x 7=56). Entering the data into the NAP calculator program we obtain the following result:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-234\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/single-case-researcg-nap-calculator-example-results.png\" alt=\"single case researcg nap calculator example results\" width=\"958\" height=\"566\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/single-case-researcg-nap-calculator-example-results.png 958w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/single-case-researcg-nap-calculator-example-results-300x177.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/single-case-researcg-nap-calculator-example-results-768x454.png 768w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/single-case-researcg-nap-calculator-example-results-65x38.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/single-case-researcg-nap-calculator-example-results-225x133.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/single-case-researcg-nap-calculator-example-results-350x207.png 350w\" sizes=\"auto, (max-width: 958px) 100vw, 958px\" \/><\/p>\n<p>Once again, we observe that <em>p<\/em>&lt;.05 (<em>Z\u00a0<\/em>statistic for our 56 pairs is -3.24, <em>p<\/em>=0.0012). Therefore, we can again reject the null hypothesis of no difference and conclude that the changes continued in a downward direction during the post-intervention period.<\/p>\n<h3><strong>Approach #2: Two-standard deviation band method<\/strong><\/h3>\n<p>This second approach is also valued the simplicity of computations involved (Orme &amp; Cox, 2001). It is called the <strong><em>two-standard deviation band method\u00a0<\/em><\/strong>because it involves computing the mean and standard deviation from the baseline phase data, drawing the \u201cband\u201d representing two standard deviations (2 <em>SD<\/em>s) around the baseline mean onto the graph, and determining where the intervention phase data points fall in comparison to that \u201cband.\u201d The band includes values that would be 2 standard deviations (<em>SD<\/em>s) above and 2 standard deviations below the mean for the phase. Another name for this \u201cbanded\u201d graph is a Shewhart chart (Orme &amp; Cox, 2001).<\/p>\n<p>A rule of thumb offered by Gottman and Leiblum (1974) is that if at least two consecutive intervention phase data points fall outside the band, a meaningful change was observed. The logic for this rule of thumb is that the probability of this happening by chance is less than the criterion of <em>p<\/em>&lt;.05 (Nourbakhsh &amp; Ottenbacher, 1994). Here is what it would look like for the hypothetical depression levels example with which we have been working throughout this chapter.<\/p>\n<p>First, compute the mean and standard deviation for the baseline (A<sub>1<\/sub>) phase data. Then, compute the values for the mean \u00b1 <em>SD<\/em> (mean plus SD and mean minus <em>SD<\/em>). Using Excel, we find the following:<\/p>\n<p style=\"padding-left: 30px\"><em>M<\/em>=5.4<\/p>\n<p style=\"padding-left: 30px\"><em>SD<\/em>=0.42<\/p>\n<p style=\"padding-left: 30px\">2 <em>SD<\/em> band=(4.98, 5.82)<\/p>\n<p>Drawn onto the original levels graph, the two-standard deviation band graph would appear as:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-235\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-11.00.59-AM.png\" alt=\"hypothetical ABAB single system design levels\" width=\"701\" height=\"414\" srcset=\"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-11.00.59-AM.png 701w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-11.00.59-AM-300x177.png 300w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-11.00.59-AM-65x38.png 65w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-11.00.59-AM-225x133.png 225w, https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-content\/uploads\/sites\/185\/2024\/08\/Screen-Shot-2019-01-23-at-11.00.59-AM-350x207.png 350w\" sizes=\"auto, (max-width: 701px) 100vw, 701px\" \/><\/p>\n<p>As you can see, more than two (in fact, all) of the intervention phase data points fall outside of the two-standard deviation band width. The conclusion would be that a significant degree of change has occurred between these two phases, baseline (A<sub>1<\/sub>) and intervention (B). To see this in action, work the exercise presented in your Excel Workbook.<\/p>\n<div class=\"textbox textbox--learning-objectives\">\n<header class=\"textbox__header\">\n<p class=\"textbox__title\">Interactive Excel Workbook Activities<\/p>\n<\/header>\n<div class=\"textbox__content\">\n<p>Complete the following Workbook Activity:<\/p>\n<ul>\n<li><a href=\"https:\/\/ohiostate.pressbooks.pub\/swk3401workbook\/chapter\/swk-3402-4-3-2-analyzing-single-systems-design-data\/\">SWK 3402.4-3.2 Analyzing Single Systems Design Data<\/a><\/li>\n<\/ul>\n<\/div>\n<\/div>\n<h2>Stop and Think<\/h2>\n<div style=\"float: left;min-height: 120px;width: 99%;margin-bottom: 10px;padding: 10px;background-color: #f1f7fe\">\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-194 alignleft\" src=\"https:\/\/pressbooks.ulib.csuohio.edu\/projectmanagement2ndedition\/wp-content\/uploads\/sites\/185\/2024\/08\/Stop_ThinkV2-150x150-1.png\" alt=\"Stop and Think\" width=\"150\" height=\"150\" \/>Take a moment to complete the following activity.<\/p>\n<div class=\"h5p-iframe-wrapper\"><iframe id=\"h5p-iframe-13\" class=\"h5p-iframe\" data-content-id=\"13\" style=\"height:1px\" src=\"about:blank\" frameBorder=\"0\" scrolling=\"no\" title=\"3402 mod 4 ch 3\"><\/iframe><\/div>\n<\/div>\n<h2><strong>Chapter Summary<\/strong><\/h2>\n<p>This chapter concerned approaches to analyzing data collected through single system design efforts to evaluate intervention outcomes. First, you learned about the visual and graphing approaches. You learned how social work professionals and clients can use levels, trends, variability, overlap, means, medians, persistence, and latency information in interpreting their outcomes data. In addition, you learned two statistical analytic approaches that might be used to determine the significance of observed changes in single system design data. The non-overlap of all pairs (NAP) analysis utilizes a form of nonparametic analytic logic to determine if a null hypothesis of no meaningful change should be rejected\u2014the test statistic was a <em>Z\u00a0<\/em>score (the calculation and distribution of which we have not previously studied), and the criterion of <em>p<\/em>&lt;.05 remained useful to informing the decision. Another easily computed statistic you learned about was the two-standard deviation band analysis which entails calculating the mean and standard deviation for baseline data, then computing the range of values encompassing two standard deviations above and below that mean. The decision concerning meaningful levels of change is based on whether there are at least two consecutive data points in the intervention phase that fall outside of the calculated two standard deviations band. This can be visualized on a graph, but does not need to be\u2014it can be determined based on reviewing a table of values once the band width has been computed. This chapter concludes our module on analyzing data for understanding social work interventions.<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"author":3,"menu_order":4,"template":"","meta":{"pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":[],"pb_section_license":""},"chapter-type":[],"contributor":[],"license":[],"class_list":["post-236","chapter","type-chapter","status-publish","hentry"],"part":208,"_links":{"self":[{"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/pressbooks\/v2\/chapters\/236","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/wp\/v2\/users\/3"}],"version-history":[{"count":2,"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/pressbooks\/v2\/chapters\/236\/revisions"}],"predecessor-version":[{"id":265,"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/pressbooks\/v2\/chapters\/236\/revisions\/265"}],"part":[{"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/pressbooks\/v2\/parts\/208"}],"metadata":[{"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/pressbooks\/v2\/chapters\/236\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/wp\/v2\/media?parent=236"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/pressbooks\/v2\/chapter-type?post=236"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/wp\/v2\/contributor?post=236"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/pressbooks.ulib.csuohio.edu\/swk627\/wp-json\/wp\/v2\/license?post=236"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}