A best evidence synthesis of literacy instruction on the social adjustment of students with or at-risk for behavior disorders.

A best evidence synthesis of literacy instruction on the social adjustment of students with or at-risk for behavior disorders. Abstract The findings of a best-evidence synthesis of the collateral effectof literacy instruction on the social adjustment of students arereported. The goal of the synthesis was to extend the work of Wanzek,Vaughn, Kim, and Cavanaugh (2006) by (a) reviewing treatment-outcomesconducted using group design methodology; (b) focusing on a more definedset of outcome measures; and (c) analyzing outcomes using average effectsize estimates as a common metric. Results of the review suggest thateffective literacy instruction does not appear to have a collateraleffect on the social adjustment of children. Limitations and futuredirections are presented. Few would argue the importance of being a strong reader. Clearly,reading is a keystone skill that serves as the key to access other formsof learning as well as entertainment (Lyon, 1996). Unfortunately, manystudents struggle to acquire the requisite skills (e.g., phonemicawareness and decoding skills) to become strong readers. Gaps in earlyliteracy skills are troublesome given that these deficits tend tobroaden over time, becoming increasingly more resistant to interventionefforts (Adams, Treiman, & Pressley, 2000; Kazdin, 1993; Nelson,Benner, Lane, & Smith, 2004; O'Shaughnessy, Lane, Gresham,& Beebe-Frankenberger, 2003). For example, deficits in decodingexpand to include problems with fluency and, subsequently, comprehensionskills. Consequently, disparity between strong and weak readers tends tobecome more pronounced over time. Often stated as the "MatthewEffect", essentially rich readers become richer and poor readersbecome poorer (Adams, 1990; Stanovich, 1986). Parallel patterns are also observed in terms of behavioraldeficits. Students with antisocial behaviors are noted for persistentviolations of social norms and expectations (Walker, Ramsey, &Gresham, 2004). These behaviors manifest in a variety of ways includingverbal and physical aggression, coercion; noncompliance, as well as lowlevels of academic engagement (Anderson, Kutash, & Duchnowski, 2001;Mattison, Hooper, & Glassberg, 2002). Not surprisingly, thesebehavior patterns impede their ability to successfully negotiate socialinteractions with their teachers and peers, and also disrupt theinstructional environment for teachers and students (Lane, Fletcher,Carter, Dejud, & DeLorenzo, 2007). Like reading deficits, problemsin social adjustment tend to broaden over time, becoming less amenableto intervention efforts. For example, verbal aggression and low levelnoncompliance in the early elementary years may expand to includephysical aggression and more extreme noncompliance during earlyadolescence (Wagner, Blackorby, Cameto, & Newman, 1993). Some students, such as those with and at-risk for emotional orbehavioral disorders (EBD; Walker et al., 2004), have deficits in bothreading and socio-behavioral domains. Given the multitude of negativeconsequences characteristic for students with EBD (e.g., school failure,impaired social relations, and propensity towards criminality), it isimperative that evidence-based interventions be implemented to meettheir multiple needs (Lane, 2007). However, questions arise regardinghow best to support students who struggle in both academic andsocio-behavioral domains. Researchers have posed a number of hypothetical models to elucidatethe relation between academic underachievement - particularly in thearea of reading - and behavioral problems (Berger, Yule, & Rutter,1975; Hinshaw, 1992; Richards, Symons, Greene, & Szuszkrewiz, 1995).One model suggests that academic deficits lead to behavioral problems;namely, students may act out to avoid aversive tasks (Hinshaw, 1992). Asecond model suggests that behavioral problems lead to academic deficits(Berger et al., 1975). A third model suggests a transactional model,with a recursive nature between these two domains (Richards et al.,1995). A fourth model suggests that other variables such ashyperactivity-impulsivity-inattention (HIA) or cognitive behavior maymoderate this relation (Rapport, Scanlan, & Denney, 1999). Researchers have begun to explore these models, with somesuggesting that improved academic outcomes are associated withcollateral effects on behavior (Ayllon & Roberts, 1974; Coie &Krehbiel, 1984; Lane, O'Shaughnessy, Lambros, Gresham, &Beebe-Franken-berger, 2001; Lane, Wehby, Menzies, Gregg, Doukas, &Munton, 2002; Lane et al., 2007). Directly connected to the purpose ofthis paper, Wanzek, Vaughn, Kim, and Cavanaugh (2006) conductedsynthesis of the literature to examine the impact of readingintervention on wide range of social outcomes for elementary-agestudents with reading difficulties. They identified 27 studies publishedbetween 1975 and 2002. Results of their review revealed small, positiveeffects for several outcomes. In brief, interactive, group-based readinginterventions appeared to produce the strongest social outcomes.However, this review focused on rather broad range of social outcomes(e.g., motivation, self-concept, peer interactions, and attitudemeasure). Furthermore, effect sizes were not combined across outcomemeasures; instead, only individual effect sizes were examined. Theintent of the current review was to target the collateral effects (i.e.,causal relationship) of literacy instruction on the social adjustment ofstudents by (a) reviewing treatment-outcomes conducted using groupdesign methodology; (b) focusing on a more defined set of outcomemeasures; and (c) analyzing outcomes using average effect-size estimatesas a common metric. Conceptual Framework Our conceptual framework for this review article is a best-evidencesynthesis of the collateral effects of literacy instruction on thesocial adjustment of students. The review does not report on the effectsof literacy interventions on attitudes toward reading; peerrelationships; engagement and motivation to read; or other nonsocialadjustment outcomes (see Wanzek et al., 2006 for a review of many ofthese variables). Studies focusing on these variables are excluded notbecause they are unimportant but because our primary interest was on thecausal collateral effects of literacy instruction on the socialadjustment of students. The review also does not report the findings ofsingle-case studies of students' classroom deportment duringliteracy instruction activities. Single-case studies are excluded notbecause they are unimportant but because there is still considerabledebate surrounding the use of and interpretation of effect sizes (a keycomponent of our analysis plan) to examine the magnitude of changepresent in single case studies (Parker et al., 2005) and inability tocombine effect sizes from single-case and experimental studies. Weacknowledge the limitations of excluding single-case studies and believethat a distinct review of the literature focusing on single-case studieswould be important to undertake. The results of such a review could thenbe contrasted with those obtained in the present review. Additionally,each of the studies included in this review contain valuable informationbeyond what we have reported in this review. We encourage the reader toread the original studies to obtain a deeper understanding of all of thefindings from each study. Review Method A best-evidence synthesis approach (Slavin, 1986) was used toidentify the collateral effects of literacy instruction on the socialadjustment of students. A best-evidence synthesis approach combines thequantification of effect sizes and systematic study selection proceduresof meta-analytic syntheses (Cooper & Hedges, 1994; Glass, McGraw,& Smith, 1981; Hedges & Olkin, 1985; Lipsey & Wilson, 2001)with the attention to individual studies and methodological andsubstantive issues typical of narrative reviews. The goal is tosummarize the "best evidence" in a field, the studies highestin internal and external validity, using well-specified and defended apriori inclusion criteria, and use effect-size data as an adjunct to adiscussion of the literature being reviewed. Our intention is to inform parents, educators, researchers, andother professionals about the current state of evidence on the extent towhich effective literacy instruction improves the social adjustment ofstudents. This information can serve to guide current interventionefforts for students with BD (i.e., necessity of directly targetingliteracy and behavior difficulties), including the development tieredand response to intervention models being implemented in schools.Because the relatively small number of studies precluded the search formoderator variables such as type of dependent measure or study quality(e.g., treatment fidelity), we did not apply statistical techniques(e.g., regression analyses) designed to identify moderator variables.Rather, we used average effect-size estimates as a common metric withwhich to help the reader to discern the magnitude and relativecollateral effects of literacy instruction on the social adjustment ofstudents. Researchers have used such quantitative techniques in similarcases in which the depth and breadth of the program of researchprecludes the analyses of moderator variables (e.g., Dimino, Taylor,& Gersten, 1995; Nelson, Benner, & Gonzalez, 2003; Rosenshine& Meister, 1994). Literature Search Procedures A comprehensive literature search was carried out in an attempt tolocate every study that met the specified inclusion criteria. Thisincluded electronic searches of the educational data-bases (ERIC, EBSCO,PsycINFO, Dissertation Abstracts, JSTOR), Web-based repositories(Google, International Reading Association), and the review of theliterature on the effects of literacy interventions on a host of socialoutcomes for students with learning disabilities conducted by Wanzek andcolleagues (2006). A wide range of search terms were used in theelectronic searches (i.e., reading difficulties, remedial reading, read,beginning reading, reading readiness, emergent literacy, earlyinterventions, phonological awareness instruction, phonemic awareness,at-risk, disadvantaged, behavioral disorders, emotional disturbance,problem behavior, behavior difficulties). Hand searchers of ExceptionalChildren, Journal of Learning Disabilities and Practice, InternationalJournal of Disabilities, journal of Educational Research, BehavioralDisorders, Journal of Emotional and Behavioral Disorders, journal ofSpecial Education, Literacy Research and Instruction, Reading Researchand Instruction, Reading and Writing Quarterly, Journal of ReadingBehavior, Journal of Literacy Research, Reading Research Quarterly,Scientific Studies in Reading, and Remedial and Special Education werealso conducted. We initially read the titles and abstracts of theidentified articles to identify studies that potentially met the seveninclusion criteria. From these articles, full text reviews wereconducted by two independent reviewers to determine if they met theinclusion criteria. Inclusion Criteria Inclusion criteria were chosen to ensure, to the extent possible,that the studies utilized reflected causal research regarding thecollateral effects of literacy interventions on the social adjustment ofstudents. Studies included in the meta-analysis had to meet seveninclusion criteria. 1. Participants included students with or at-risk for readingand/or behavior problems. 2. The studies compared students taught using a specified literacyintervention or program to those in a control group using an alternativeprogram or standard methods. 3. The independent variable targeted one or more of the fiveliteracy areas identified by the National Reading Pane! (2000; i.e.,phonological awareness, phonics, fluency, comprehension, vocabulary). 4. Studies were randomized control trials or quasi-experimentaldesigns that had comparison group(s) with demonstrated equivalence. 5. Dependent measures addressed both literacy (e.g., changes inphonological awareness) and social adjustment (e.g., social skills).Outcomes in other areas (e.g., receptive language skills, attitudestoward reading) were not considered in the studies reviewed. 6. Reports had to include quantitative information (e.g., means andassociated standard deviations, F statistic) necessary to compute atleast one effect size estimator each for a literacy and socialadjustment outcome. 7. The article had to be available in English. The study could havebeen conducted in any country. Upon analysis of the complete studies, four articles meet theinclusion criteria. These four articles presented information on thecollateral effects of literacy instruction on the social adjustment ofstudents. Effect Sizes We made several decisions during the literature review processbased on commonly used meta-analytic literature review guidelines toreduce redundancy or overweights of estimates from interdependence inthe research samples or measures (Cooper & Hedges, 1994; Glass etal., 1981; Hedges & Olkin, 1985; Lipsey & Wilson, 2001). First,we reviewed the studies to ensure independence in the samples. Each ofthe studies represented independent samples. Second, in calculations of effect sizes, Hedges's g wasweighted by sample size according to the procedures recommended byCooper and Hedges (1994). Estimates of Hedges's g were obtained bycomputing the difference between the experimental and control individualstudent mean (a) change scores divided by the pooled standard deviationor (b) posttest scores divided by the pooled standard deviation. Alleffect sizes were computed with the Comprehensive Meta-Analyses Version2 statistical package (Borenstein & Rothstein, 2005). Finally, single within-study average estimates of Hedge's gwere calculated for both literacy and social adjustment outcomes toreduce redundancy or overweights of estimates from the use of multipleinter-correlated measures within a given study. For social adjustment,for example, if researchers reported the outcomes for several sub-scalesfrom a social adjustment measure we calculated a single within-studyaverage Hedges's g effect size estimate for the effect of literacyinstruction on social adjustment. Note that we report the Hedges'sg for each individual literacy and social adjustment measure/subscale.We also calculated the overall average Hedge's g and associated 95%confidence intervals for literacy and social adjustment outcomes. Recorded Variables Two independent coders (first author, independent researchassistant) reviewed each study that met the inclusion criteria toidentify six categories of information. The coders used a coding form torecord the following information for each of the categories. 1. Research setting. Information was collected on the geographiclocation (i.e., Northeast, Midwest, South, West), size of community(i.e., small, medium, large), type of community (i.e., urban, rural),type of school (i.e., public, private and elementary, middle, highschool), grade configuration (e.g., K-6), percent free or reduced lunch,achievement level. 2. Participants, Information was collected on the number ofparticipants (i.e., total, per experimental group), selection criteria(i.e., reading, social adjustment), grade level, mean age, gender,race/ethnicity, disabilities, and free or reduced lunch status. 3. Research design. Information was collected on the type of design(i.e., randomized control, quasi-experimental with equating ofnon-equivalent groups) and level of randomization if applicable (i.e.,student, classroom, school). 4. Independent variable. Information was collected on the mode ofdelivery (i.e., large group, small group, one-to-one), literacy focus(i.e., phonological awareness, phonics, fluency, vocabulary,comprehension), intervention delivery (i.e., school staff, researchers),length of intervention (i.e., number of sessions/days), and treatmentfidelity (i.e., measured, use in analyses). 5. Dependent variables. Information collected included the type(s)of literacy measures (i.e., norm-referenced, curriculum-based,researcher designed), descriptions of literacy measures, type(s) ofsocial adjustment measures (i.e., norm-referenced, observational), andother non-literacy and non-social adjustment measures. 6. Outcomes. Information necessary to compute effect sizes werecollected. As noted above, this included change and posttest scores andassociated standard deviations. Inter-coder agreement was calculated overall and for each categoryof information by taking the number of agreements divided by agreementsplus disagreements times 100. The overall level of inter-coder agreementwas 97%. Inter-coder agreements for the setting, participants, researchdesign, independent variable, dependent variables, and outcomescategories were 90%, 100%, 100%, 88%, 89%, and 100%, respectively.Discrepancies were noted, discussed, and resolved by the coders. Results Description of Methods A total of four studies met our inclusion criteria (Jenkins,Jewell, O'Connor, Jenkins, & Troutner, 1994; Lane, 1999; Laneet al., 2007; Nelson, Stage, Epstein, & Pierce, 2005). Theinformation on the number, mean age, grade-level and effect sizestatistics for the meta-analysis are depicted in Table 2 and describedbelow. We detail specific information on the settings, participants,research designs, and independent variable(s), and dependent variablesin the remainder of this section. Settings Researchers generally did not provide a great deal of informationon the research settings. The studies were conducted in publicelementary schools located in the West (Jenkins et al., 1994; Lane,1999; Lane et al., 2007) and Midwest (Nelson et al., 2005). None of theresearchers provided information on the overall school achievementlevel. Only Jenkins and colleagues (1994) reported the overall free orreduced lunch rate (i.e., experimental school=47%, control school=24%). Participants. A total of 208 students contributed to this review.We did not include 13 students assigned to a social skills intervention(Lane, 1999) and 755 special (n=58) and general (n=697) educationstudents (Jenkins et al., 1994). In the latter case, three naturallyoccurring groups of students in the experimental and control schoolswere compared: at-risk (received federal- or state-funded remedialsupplemental literacy instruction), special education, and generaleducation. The at-risk group of students was formally considered in thisreview because they were similar to those participants included in theremaining studies. We felt it was important to reduce sample-irrelevantvariance given the small number of studies. The numbers of students fromthe respective experimental and control schools that were in theat-risk, special education, and general education groups were 51 and 44,23 and 35, and 248 and 449, respectively. One study included students in the K-6th grades (Jenkins et al.,1994). The three remaining studies were conducted with students inkindergarten (Nelson et al., 2005) and first grade (Lane, 1999; Lane etal., 2007). Systematic multi-step selection procedures were used toidentify participations in these three studies. For example, Lane andcolleagues (2007) used a multi-step nomination procedure to identifystudents with reading and social adjustment problems. First teacherswere asked to nominate up to 12 students (1st grade) who demonstratedexternalizing or internalizing behaviors and who performed below averagein reading ability. Second, teachers reviewed definitions, examples, andnon-examples of externalizing and internalizing behaviors derived byfrom the Systematic Screening for Behavior Disorders (SSBD; Walker &Severson, 1992) and nominated up to 12 students who best fit either ofthe behavioral descriptions. Third, teachers grouped all of the studentsin their classes into two groups: average to high reading abilities orbelow-average reading abilities. Finally, up to 10 students in eachclassroom who best met the description of externalizing or internalizingbehavior patterns and who fell in the below average reading group wereinvited participate in the study. Research designs. Three of the four studies reviewed here randomlyassigned selected students with reading and social adjustment problemsto experimental and control conditions (Lane, 1999; Lane et al., 2007;Nelson et al., 2005). In all three cases, researchers verified theequivalence of the experimental and control conditions. This isimportant because the total number of students assigned to theexperimental studies in each of these three studies was relatively small(see Table 2). For example, Lane and colleagues (2007) randomly assigned24 students to the treatment (n=13) and control (n=11) conditions. Thissuggests that the results of these three studies were not confounded bypre-treatment differences (i.e., sampling error). Pre-treatmentdifferences can affect the dependent variable, making it difficult toseparate the effects of the independent variable from the effects fromthe initial differences between the groups. A quasi-experimental designwas used in the remaining study (Jenkins et al., 1994). Two elementaryschools were selected for participation. Although the pre-treatmentequivalence of the students in the at-risk group was established, thelack of random assignment leaves this study open to selection bias. Ofcourse, the negative findings, which were consistent across all studiesreviewed here, regarding the collateral effects of literacy instructionon the social adjustment of students, at least in part, mutes selectionbias concerns. Independent variables. Brief descriptions of the literacyinstruction used in each study are presented in Table 1. Jenkins andcolleagues (1994) implemented a comprehensive literacy instructionprogram across all grades in two elementary schools for an entireacademic year. The core literacy program was supplemented withsmall-group, one-to-one, and cross-age tutoring instructionalactivities. Further, the comprehensive literacy instruction programtargeted the complete range of literacy skills. Researchers in the threeremaining studies targeted supplementary early literacy skillsinstruction (e.g., phonological awareness, letter names, letter-sounds;Lane, 1999, Lane et al., 2007, Nelson et al., 2005). In all cases, astandard protocol literacy intervention was used. For example, Nelsonand colleagues (2005) used Stepping Stones to Literacy (Nelson, Cooper,& Gonzalez, 2004). Stepping Stones to Literacy is a cohesive (i.e.,sequential scope and sequence of skills, explicit instruction practices)and intensive 25-lesson supplementary early literacy intervention foryoung children who are at-risk for developing reading difficulties.Stepping Stones to Literacy consists of one Lesson Book and a separatesection on serial rapid automatic naming (activities that providechildren practice making quick visual-verbal associations of known setsof colors, numbers, and/or letter names in a left-to-right format).Additional instructional activities are used in the initial 14instructional sessions to promote children's listeningcomprehension skills and understanding of the meanings ofsentences/short stories (i.e., listen for who is doing something andwhat they are doing). Additionally, the instructional formats are heldconstant across the 25 lessons and guide the educator through each ofthe instructional activities (i.e., soft scripted) and all of thenecessary instructional stimuli are included.Table 1Literacy Instruction for Qualifying StudiesStudy Instructional format DescriptionLane, et. Small group Phonological Awareness Training foral. (2007) instruction 3 days Reading (Torgesen & Bryant, 1994a) per week for 30 is a supplementary early reading minutes for 10-weeks curriculum that includes four activities: (a) rhyming, (b) blending, (c) segmenting, and (d) reading and spelling. Researchers developed 30 scripted lessons.Nelson, One-to-one Stepping Stones to Literacy (Nelson,et.al. instruction 5 days a Cooper, et al., 2004) is a(2005) per week for 25 supplemental early literacy minutes until 25 curriculum that includes four lessons were activities: (a) listening completed comprehension, (b) phonological awareness, (c) alphabet knowledge, and (d) rapid automatic naming.Lane, Small Group Phonological Awareness Training for(1999) Instruction Reading (Torgesen & Bryant, 1994a) See above description above. Researchers developed 25 scripted lessons.Jenkins, Large, small-group, Four components were comprised theet.al. one-to-one, and peer comprehensive reading program(1994) and cross-age implemented: (a) Cooperative instructional formats Integrated Reading and Composition were used across the (Stevens, Madden, Slavin, & Famish, entire academic year 1987) activities and materials were integrated with the Houghton-Mifflin reading series (Durr et al., 1989); (b) peer and cross-age tutoring (Jenkins & Jenkins, 1985), (c) supplementary synthetic phonics instruction (Engelmann & Bruner, 1988), and (d) in-class instructional support from specialists (Bean, Cooley, Eichelberger, Lazar, & Zigmond, 1989). Dependent measures. In all cases, researchers used a combination ofstandardized norm-referenced and curriculum-based measures to assesschanges in the literacy skills of students (see Table 2). For example,Lane and colleagues (2007) used a norm-referenced (i.e., Test ofPhonological Awareness, Torgesen & Bryant, 1994b) and acurriculum-based measure (i.e., Dynamic Indicators of Basic EarlyLiteracy Skills Nonsense Word Fluency probe; Good & Kaminski, 2002)to assess changes in the phonological awareness and word attack skillsof students, respectively. Additionally, with one exception (Lane etal., 2007),researchers used norm-referenced rating scales to assesschanges in the social adjustment of students (see Table 2). The ratingscales focused on students' emotional and social strengths (i.e.,Behavioral and Emotional Rating Scale, Epstein & Sharma, 1998) orschool related social skills and deportment (e.g., Walker-McConnellScale of Social Competence and School Adjustment; Walker &McConnell, 1988). Lane and colleagues (2007) used behavioralobservations to assess changes in the social adjustment of students.Table 2Descriptive Information and Effect Sizes for Qualifying Studies Effect size ReadingStudy (1) N/X Age/Grade Measure Effect Overall size by effect posttest sizeLane, et.al. Total = 24 DIBELS +.50(2007) Treatment = 13 Nonsense Word +.35 Control = 11 Fluen. Age = - TOPA +.64 Grade=lNelson, et.al. Total = 63 DIBELS +.99(2005) Treatment = 47 Letter Naming +1.16 Control = 16 Fluen. Age = 5.2 CTOPP Grade=K Phonological +.99 Aware. WRMT-R Word Attack +.97 Word +1.07 IdentificationLane (1999) Total = 26 TOPA +.79 +.46(b) Treatment = 13 WRMT-R Control = 13 Word Attack +.13 Age = 6.8 Grade=lJenkins et.al. Total = 95 MAT-6 +.22(1994) (c) Treatment = 51 Total Reading +.44 Control = 44 Comprehension +.02 Age = - Vocabulary +.48 Grade=K-6 BASS Reading -.06 Effect size Social adjustmentStudy (1) N/X Age/Grade Measure Effect Overall size by effect posttest sizeLane, et.al. Total = 24 Behavior -.14(2007) Treatment = 13 Observation Control = 11 Total Disruptive +.28 Age = - Negative Social -.55 Grade=l Int.Nelson, et.al. Total = 63 BERS -.52(2005) Treatment = 47 School Control = 16 Functioning -.34 Age = 5.2 Interpersonal -.58 Grade=K Intrapersonal -.63Lane (1999) Total = 26 SSRS +.68(b) Treatment = 13 Social Skills +.63 Control = 13 Problem Behavior +.74 Age = 6.8 Grade=lJenkins et.al. Total = 95 SSCSA -.35(1994) (c) Treatment = 51 Total Score -.29 Control = 44 Teacher-Preferred -.30 Age = - Peer-Preferred -.63 Grade=K-6Note. A- indicates that information was not reported. (1) All researchdesigns were randomized control trials with the exception of Jenkins,et.al, which was a quasi-experimental study in which the equivalence ofthe groups was established. (b) Results for treatment group thatreceived social skills intervention are not reported. BASS--BasicAcademic Skills Samples (Espin, Deno, Maruyama, & Cohen, 1989).(c) Only the results for the at-risk students in the treatment andcontrol are reported. BERS--Behavioral and Emotional Rating Scale(Epstein & Sharma, 1998). CTOPP--Comprehensive Test of PhonologicalProcessing (Wagner, Torgesen, & Rashorte, 1999). TOPA--Test ofPhonological Awareness (Torgesen & Bryant, 1994b). DIBELS--DynamicIndicators of Basic Early Literacy Skills (Good & Kaminski, 2002).MAT-6-Metropolitan Achievement Test: Sixth Edition (Prescott, Balow,Hogan, & Farr, 1984). SSCSA--Walker-McConnell Scale of SocialCompetence and School Adjustment (Walker & McConnell, 1988).SSRS-Social Skills Rating Scale (Gresham & Elliott, 1990).WRMT-R-Woodcock Reading Mastery Revised (Woodcock, 1998). Study Outcomes The single within-study average Hedges's g and those for theassociated individual literacy and social adjustment measures arepresented in Table 2. The overall average Hedge's g for theliteracy outcomes was +.49. The 95% confidence interval ranged from +.20to +.79. The within-study average Hedges's g ranged from .22(Jenkins et al., 1994) to .99 (Nelson et al., 2005). Homogeneityanalysis indicate that differences in the within-study averageHedges's g were not statistically significant (i.e., Q = 4.28, df.= 3, p = .24). This suggests that differences in the within-studyaverage effects sizes were not statistically significant. The overall average Hedge's g for the social adjustmentoutcomes was -.14. The 95% confidence interval ranged from +.16 to -.43.The within study average Hedge's g ranged from +.68 (Lane, 1999) to-.52 (Nelson et al., 2005). Homgeneity analysis indicate thatdifferences in the within-study average Hedges's g werestatistically significant (i.e., Q = 8.87, df. = 3, p = .03). Thissuggests that differences with-study average effect sizes werestatistically significant. Discussion In this review we extended the work of Wanzek and colleagues (2006)to further elucidate the relation between literacy instruction on thesocial adjustment of students. We conducted a best-evidence synthesisapproach, which involves quantification of effect sizes, systematicstudy selection procedures characteristics of meta-analytical syntheses(e.g., Cooper & Hedges, 1994; Lipsey & Wilson, 2001), as well asfeatures of narrative reviews (e.g., discussion of individual reviews).Whereas Wanzek et al. provided a more global review of the literature byreviewing group and single case design studies, reporting individualeffect sizes on a broad array of social outcome measures, we offer amore focused review. Specifically, we focus exclusively ontreatment-outcome studies, reporting average effect size estimates on amore narrowly defined set of outcome measures. A total of 4 studies met inclusion criteria specified in thisreview and were subsequently included in the analyses. The overalleffect size for literacy outcomes was +.49. The 95% confidence intervalranged from +.20 to +.79. The homgeneity analysis indicates that theeffects on literacy outcomes were consistent across the four studies.Thus, the interventions yielded consistent improvements in readingskills. These findings provide evidence that the four studies allow forcausal inferences regarding the collateral effects of literacyinstruction on the social adjustment of students. Overall, the results of the review suggest that literacyinstruction does not appear to have collateral positive effect on thesocial adjustment of students. The overall effect size for socialadjustment outcomes was -.14. However, it is important to note that the95% confidence interval ranged from +.16 to -.43 and the homogeneityanalysis indicates that the collateral effects of literacy instructionon the social adjustment of students varied across the four studies(range=+.68 to -.52). The average within-study effect size of Lane(1999) appeared to vary from the remaining three studies (Jenkins etal., 1994; Lane et al., 2007; Nelson et al., 2005). These findings were somewhat in contrast to the synthesis reportedby Coleman and Vaughn (2000) that revealed small, positive effects forseveral outcomes reported in the 27 studies reviewed. However, in termsof eight studies in the Vaughn and colleagues review that measured theeffects of reading interventions on behavioral outcomes, results weremixed. Namely, results appeared to vary according to how behavioraloutcomes were assessed. For example, when direct observation techniqueswere employed, small to moderate improvements in behavioral outcomes(other than social adjustment) were detected - in both group design andsingle case studies. However, in three studies using teacher reports tomeasure classroom deportment revealed no effects (e.g., Abikoff,Ganeles, Reiter, Blum, Foley, & Klein, 1988) or even negativeeffects (e.g., O'Shaughnessy & Swanson, 2000) on behavioraloutcomes. Limitations and Future Directions When interpreting these findings, we encourage readers to considerthe following limitations of the current review. First, it should benoted that due to the small numbers of studies identified, we were notable to conduct analyses to identify potential moderating (e.g., studentcharacteristics) and mediating (e.g., treatment fidelity) variables thatmight have influenced social adjustment outcomes. We encourageadditional inquiry so that the role of mediating and moderatingvariables in relation to social adjustment outcomes potentiallyassociated with reading interventions can be further explored.Additionally, as noted earlier, we a parallel synthesis of single-casestudies on the collateral effects of literacy instruction on the socialadjustment of students with or at risk of BD should be undertaken. Second, the studies included in this review, had limitedinformation regarding the sustainability of the outcomes. In someinstances (e.g., Jenkins et al. 1994), the post-intervention outcomeswere not favorable. Future studies are needed to examine both short-termand long-term effects of literacy instruction on the social adjustmentof students. It is possible that changes in social adjustment may take aperiod of time to accrue. Also, it is important to understand andidentify the characteristics of the students who do and do not sustainchanges over time. Third, the majority of the studies in this review focused on earlyliteracy skills using commercially available programs such as PATR(Lane, 1999; Lane et al., 2007) and Stepping Stones to Literacy (Nelsonet al., 2005), one study only the Jenkins et al. (1994) focused on moreadvanced skill sets. While it is certainly necessary and important tofocus on early literacy skills, when both academic and behavioraldeficits are more amenable to change (Kazdin, 1993; Lane, 2004), it isalso important to explore the relation between improved reading skillsand social adjustment for more advanced skill deficits and with olderstudents. Namely, it is possible that the nature of the relation betweenliteracy and social adjustment may be different as the skill deficitsbecome more broad in nature as students move across the K-12 continuum.For example, it may be that collateral effects on literacy instructionon social adjustment may be more pronounced if deficits in more pivotalareas (such as reading comprehension skills) are remediated. Fourth, researchers should conduct population based studies inwhich clear criteria are used to identify a specific population ofstudents rather than general populations. It may be that the patterns ofresponding vary as function of student characteristics (e.g., those withand at-risk for EBD). In the Jenkins et al. (1994) study the findingswere similar for general education and high incidence special educationstudents as evidenced by respective effect sizes of -.12 and -.11. Toexplore this study, it will be critical that researchers clearly specifyboth participant selection criteria and participant characteristics todraw accurate conclusions regarding the generaliziability of thefindings (Gersten et al., 2005) Fifth, it is necessary for additional studies to be conducted todetermine the extent to which various interventionists such as graduatestudents (Lane, 1999; Nelson et al., 1995), paraprofessionals (Lane etal., 2007), and existing school-site staff (Jenkins et al., 1994) areable to implement the interventions with sufficient fidelity to securethe desired changes in students' academic performance. Also, thetype of interventionist may influence potentially collateral effects onsocial adjustment. For example, if the interventionist is a school-siteemployee, it is possible that their presence on campus may serve as aprompt (a discriminative stimulus; Cooper, Heron, &: Heward, 2007)to utilize the strategies acquired during instruction beyond thetraining sessions. If so, then it is possible that maintenance effectsand collateral effects could be influenced by who delivered theinstruction. Finally, it is important to note that all but one study (Lane etal., 2007) used behavior rating scales to measure social behavioraladjustment. The difference in behavior measurement systems (e.g., directobservations versus behavior rating scales) may account for some of thedifferences in the magnitude of the changes presented (e.g., -.14 forLane et al., 2007 versus -.52 for Nelson et al., 2005). Namely, it ispossible that teacher ratings may have been influenced by the knowledgethat the students were participating in an intervention study in whichreading and behavioral performance was being assessed. Directobservation techniques are not subject to this potential confound, andmay be more sensitive to changes in specific behaviors. This may explainthe mixed results in the Lane et al. (2007) study that revealedimprovements in social interactions (as evidenced by a negative effectsize for social interactions, -.55) and declines in behavior (asevidenced by a positive effect size for disruptive behavior, +.28).However, direct observation techniques are more narrow in scope, onlycapturing a small window of students' socio-behavioral performance.In contrast, behavior rating scales such as the BERS capture a broaderrange of student behavior (e.g., school functioning, interpersonal, andintrapersonal skills). Summary Despite these limitations, findings of this best-evidence synthesisreview examining the collateral effects of literacy instruction on thesocial adjustment of students did not support the first hypotheticalmodel posed by Hinshaw (1992). Namely, results suggest that academicunderachievement does not lead to problem behaviors. However, resultsmust be interpreted in light of the limitations posed above, withparticular attention to the relation between outcomes and the method ofmeasurement used to assess behavioral performance as well as the limitedscope of statistical analyses able to be employed due to the smallnumber of studies meeting the inclusion criteria specified in thisreview. It is possible collateral effects of literacy instruction onstudents' social adjustment may exist. However, results of thisbest-evidence synthesis did not demonstrate such effects. We encouragefuture inquiry, addressing some the limitations identified in thisreview. Nevertheless, the findings of this paper can be used to guidepractice for those working with students experiencing both literacy andbehavior difficulties. The findings suggest that literacy and behaviorinterventions are necessary to improve the outcomes of students withEBD. Unfortunately, it does not appear that changes in literacy outcomeswill have a collateral effect on the social adjustment of thesestudents. Additionally, we would concur with scholars recommendationsthat current tiered prevent models designed to better meet the needs ofstudents with EBD should include encompass both the academic and socialdomains (Nelson, Duppong-Hurley, Synhorst, Epstein, & Stage, 2009;Stewart, Benner, Martella, & Marchand-Martella, 2007). References * Denotes articles included in the meta-analysis. Abikoff, H., Ganeles, D., Reiter, G., Blum, C., Foley, C., &Klein, R. G. (1988). Cognitive training in academically deficient ADDHboys receiving stimulant medication. Journal of Abnormal ChildPsychology, 16, 411-432. Adams, M. J. (1990). Beginning to reading: Thinking and learningabout print. Cambridge, MA: MIT Press. Adams, M. J., Treiman, R., & Pressley, M., (2000). Reading,writing, and literacy. In I. E. Sigel & K. A. Renninger (Eds.),Handbook of child psychology, child psychology in practice, vol. 4, (pp.275-355). New York: Wiley. Anderson, J. A., Kutash, K., & Duchnowski, A. J. (2001). Acomparison of the academic progress of students with EBD and studentswith LD. Journal of Emotional and Behavioral Disorders, 9, 106-115. Ayllon, T., & Roberts, M. (1974). Eliminating disciplineproblems by strengthening academic performance. Journal of AppliedBehavior Analysis, 7, 71-76. Bean, R. M., Cooley, W., Eichelberger, R. T., Lazar, M., &Zigmond, N. (1989). In-class remedial reading programs: Old wine in newbottles. Paper presented at the annual meeting of the AmericanEducational Research Association, San Francisco, CA. Berger, M., Yule, W., & Rutter, M. (1975). Attainment andadjustment in two geographical areas: II. The prevalence of specificreading retardation. British Journal of Psychiatry, 126, 510-519. Borenstein, M., & Rothstein H. (2005). Comprehensive MetaAnalysis, Version 2. Software available from Biostat, Inc., Englewood,NJ. Coie, J., & Krehbiel, G. (1984). Effects of academic tutoringon the social status of low-achieving, socially-rejected children. ChildDevelopment, 55, 1465-1478. Coleman, M., & Vaughn, S. (2000). Reading interventions forstudents with emotional/behavioral disorders. Behavioral Disorders, 25,93-104. Cooper, H. M., & Hedges, L. V. (Eds.). (1994). The handbook ofresearch synthesis. New York: The Russell Sage Foundation. Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Appliedbehavior analysis. Upper Saddle River, N.J.: Pearson Education. Dimino, J., Taylor, R. & Gersten, R. (1995). Synthesis of theresearch on story grammar as a means to increase comprehension. Readingand Writing Quarterly, 11(1), 53-72. Durr, W. K., Pikulski, J. J., Bean, R. M., Cooper, J. D., Glaser,N. A., Greenlaw, M. J., et al. (1989). Houghton-Mifflin reading. Boston:Houghton-Mifflin. Engelmann, S., & Bruner, E. (1988). Reading mastery I. Chicago:Science Research Associates. Epstein, M. H., & Sharma, J. M. (1998). Behavioral andEmotional Rating Scale: A strength-based approach to assessment. Austin,TX: PRO-ED. Espin, C., Deno, S. L., Maruyama, G., & Cohen, C. (1989). TheBasic Academic Skills Samples (BASS): An instrument for the screeningand identification of students at-risk for failure in regular educationclassrooms. Paper presented at the annual meeting of the AmericanEducational Research Association: San Francisco. Gersten, R., Fuchs, L. S., Compton, D., Coyne, M., Greenwood, C.,& Innocenti, M. S. (2005). Quality indicators for group experimentaland quasi-experimental research in special education. ExceptionalChildren, 71, 149-164. Glass, G. V., McGraw, B., & Smith, M. L. (1981). Meta-analysisin social research. Beverly Hills, CA: Sage. Good, R. H., & Kaminski, R. A. (Eds.). (2002). Dynamicindicators of basic early literacy skills (6th ed.). Eugene, OR:Institute for the Development of Educational Achievement. Available:http://dibels.uoregon.edu. Gresham, F. M., & Elliott, S. N. (1990). Social skills ratingsystem. Circle Pines, MN: American Guidance Service. Hedges, L. V., & Olkin, I. (1985). Statistical methods formeta-analysis. San Diego, CA: Academic Press. Hinshaw, S. P. (1992). Externalizing behavior problems and academicunderachievement in childhood and adolescence: Causal relationships andunderlying mechanisms. Psychological Bulletin, 111, 127-155. Jenkins, J. R., & Jenkins, L. M. (1985). Peer tutoring inelementary and secondary programs. Focus on Exceptional Children, 17,1-12. * Jenkins, J. R., Jewell, M., O'Connor, R. E., Jenkins, L. M.,& Troutner, N. M. (1994). Accommodations for individual differenceswithout classroom ability groups: An experiment in school restructuring.Exceptional Children, 60, 344-358. Kazdin, A. E. (1993). Treatment of conduct disorders: Progress anddirections in psychotherapy research. Development and Psychopathology,5, 277-310. * Lane, K. L. (1999). Young students at-risk for antisocialbehavior: The utility of academic and social skills interventions.Journal of Emotional and Behavioral Disorders, 7, 211-224. Lane, K. L. (2004). Academic instruction and tutoring interventionsfor students with emotional/behavioral disorders: 1990 to present. In R.B. Rutherford, M. M. Quinn, &. S. R. Mathur (Eds.), Handbook ofresearch in emotional and behavioral disorders (pp. 462-486). New York:Guilford Press. Lane, K. L. (2007). Identifying and supporting students at-risk foremotional and behavioral disorders with multi-level models: Data drivenapproaches to conducting secondary interventions with academic emphasis.Education and Treatment of Children, 30, 135-164. * Lane, K. L., Fletcher, T., Carter, E. W., Dejud, C., &DeLorenzo, J. (2007). Paraprofessional-led phonological awarenesstraining with youngsters at-risk for reading and behavior problems.Remedial and Special Education, 28, 266-276. Lane, K. L., O'Shaughnessy, T. E., Lambros, K. M., Gresham, F.M, & Beebe-Frankenberger, M. E. (2001). The efficacy of phonologicalawareness training with first-grade students who have behavior problemsand reading difficulties. Journal of Emotional and Behavioral Disorders,9, 219-231. Lane, K. L., Wehby, J. H., Menzies, H. M., Gregg, R. M., Doukas, G.L., & Munton, S. M. (2002). Early literacy instruction forfirst-grade students at-risk for antisocial behavior. Education andTreatment of Children, 25, 433-458. Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis.Thousand Oaks, CA: Sage. Lyon, G. R. (1996). Learning disabilities. In E. Marsh & R.Barkley (Eds.), Child psychopathology (pp. 390-434). New York: GuildfordPress. Mattison, R. E., Hooper, S. R., & Glassberg, L. A. (2002).Three-year course of learning disorders in special education studentsclassified as behavioral disorder. Journal of the American Academy ofChild & Adolescent Psychiatry, 41, 1454-1461. National Reading Panel. (2000). Teaching children to read: Anevidence-based assessment of the scientific research literature onreading and its implications for reading instruction. Washington, DC:National Institute of Child Health and Human Development. Nelson, J. R, Benner, G. J., & Gonzalez, J. (2003). Learnercharacteristics that influence the treatment effectiveness of earlyliteracy interventions: A meta-analytic review. Learning DisabilitiesResearch & Practice, 18, 255-267. Nelson, J. R., Benner, G. J., Lane, K., & Smith, B. W. (2004).An investigation of the academic achievement of K-12 students withemotional and behavioral disorders in public school settings.Exceptional Children, 71, 59-73. Nelson, J. R., Cooper, P., & Gonzalez, J. E. (2004). SteppingStones to Literacy. Denver, CO: Sopris West. Nelson, J. R., Duppong-Hurley, K., Synhorst, L., Epstein, M. H.,& Stage, S. (2009). The child outcomes of a three tier behaviormodel. Exceptional Children, 76, 7-30. * Nelson, J. R., Stage, S. A., Epstein, M. H., & Pierce, C. D.(2005). Effects of a prereading intervention on the literacy and socialskills of children. Exceptional Children, 72, 29-45. O'Shaughnessy, T. E., & Swanson, H. L. (2000). Acomparison of two reading interventions for children with readingdisabilities. Journal of Learning Disabilities, 33, 257-277. O'Shaughnessy, T. E., Lane, K. L., Gresham, F. M, &Beebe-Franken-berger, M. E. (2003). Children placed at-risk for learningand behavioral difficulties: Implementing a school-wide system of earlyidentification and prevention. Remedial and Special Education, 24,27-35. Parker, R. I., Brossart, D. F., Bannest, K. J., Long, J. R.,De-Alba, R. G., Baugh, F. G., et al. (2005). Effect sizes in single caseresearch: How large is large? School Psychology Review, 34, 116-132. Prescott, G. A., Balow, I. H., Hogan, T. P., & Farr, R. C.(1984). Metropolitan Achievement Tests (MAT-6). San Antonio, TX:Psychological Corp., Harcourt Brace Jovanovich. Rapport, M. D., Scanlan, S. W., & Denney, C. B. (1999).Attention-deficit/hyperactivity disorders and scholastic achievement: Amodel of dual developmental pathways. Journal of Child PsychologyPsychiatry, 40, 1169-1183. Richards, C. M., Symons, D. K., Greene, C. A., & Szuszkrewiz,T. A. (1995). The bi-directional relationship between achievement andexternalizing behavior problems of students with learning disabilities.Journal of Learning Disabilities, 28, 8-17. Rosenshine, B., & Meister, C. (1994). Reciprocal teaching: Areview of the research. Review of Educational Research, 64, 479-530. Slavin, R. E. (1986). Best-evidence synthesis: An alternative tometa-analytic and traditional reviews. Educational Researcher, 19, 5-11. Stanovich, K. E. (1986). Matthew effects in reading: Someconsequences of individual differences in the acquisition of literacy.Reading Research Quarterly, 21, 360-406. Stevens, R. J., Madden, N. A., Slavin, R. E., & Farnish, A. M.(1987). Cooperative integrated reading and composition: Two heldexperiments. Reading Research Quarterly, 22, 433-454. Stewart, R. M., Benner, G. J., Martella, R. C., &Marachand-Martella, N. E. (2007). Journal of Positive BehaviorInterventions, 19, 239-253. Torgesen, J. K., & Bryant, B. R. (1994a). PhonologicalAwareness Training for Reading. Austin, TX: PRO-ED. Torgesen, J, K., & Bryant, B. R. (1994b). Test of PhonologicalAwareness. Austin, TX: PRO-ED. Wagner, M. M., Blackorby, J., Cameto, R., & Newman, L. (1993).What makes a difference? Influences on post-school outcomes of youthwith disabilities. Menlo Park, CA: SRI International. Wagner, R. K., Torgesen, J. K., & Rashotte, C. A. (1999).Comprehensive Test of Phonological Processing (CTOPP). Longmont, CO:Sopris West. Walker, H. M., & McConnell, S. R. (1988). The Walker-McConnellScale of Social Competence and School Adjustment. Austin, TX: PRO-ED. Walker, H. M., & Severson, H. (1992). Systematic screening forbehavior disorders: Technical manual. Longmont, CO: Sopris West. Walker, H. M., Ramsey, E., & Gresham, F. M. (2004). Antisocialbehavior in school: Evidence-based practices (2nd ed.). Belmont, CA:Wadsworth. Wanzek, J., Vaughn, S., Kim, A., & Cavanaugh, C. (2006). Theeffects of reading interventions on social outcomes for elementarystudents with reading difficulties: A synthesis. Reading and WritingQuarterly, 22, 121-138. Woodcock, R. W. (1998). Woodcock Reading Mastery Tests--Revised.Circle Pines, MN: AGS. J. Ron Nelson University of Nebraska, Lincoln Kathleen L. Lane Vanderbilt University Gregory J. Benner University of Washington, Tacoma Ockjean Kim University of Nebraska, Lincoln Correspondence to J. Ron Nelson, University of Nebraska-Lincoln,Center for At-Risk Children's Services, Barkley Center, Departmentof Special Education and Communication Disorders, Lincoln, NE68583-0738; e-mail: rnelson8unl.edu.