- Who We Are
- What We Do
- Contact Us
- Current Events
Volume 24 Number 2
Edited by Melissa Maras and Joni Splett
Greetings from the School Intervention Interest Group!! The research-to-practice gap is a significant issue in the public school system. While many researchers follow the traditional research-centered approach when developing and disseminating evidence-based practices, others utilize participatory approaches driven by practice-based evidence to develop externally valid and sustainable innovations. In this issue of TCP, we are pleased to feature the work of such community-driven researchers. The article describes how these researchers used practice-based evidence to iteratively develop and evaluate a school-based mentoring program that is gaining empirical support and positive reviews from partnering public school districts.
Written by Samuel D. McQuillin, Brent W. Smith, and Brittany McLelland, University of Houston
The traditional approach to evidence-based practice assumes a) randomized controlled trials are capable of documenting effective interventions and b) the effects of these interventions are generalizable from research labs to routine practice in the field (Kratochwill, 2012). It has become increasingly clear the second assumption may be compromised by a variety of factors at the individual level (e.g. client variability) and systemic level (e.g. contextual differences) (Green, 2007).
A complementary supplement to the traditional evidence-based approach is practice-based research, a process in which practitioners contribute to knowledge generation by collaborating with researchers to inform the development or implementation of services provided in routine practice settings (Leeman & Sandelowski, 2012). Practice-based research has become increasingly important in contexts where there are large research-to-practice gaps, such as public school systems (Kratochwill, 2012). In this article, we provide an example of how practice-based evidence has transformed the development of a school-based mentoring program for middle school students by identifying and resolving three barriers associated with the implementation of school-based mentoring programs. We hope recognition of this approach to innovation will encourage practitioners to contribute to future development of school-based services, and encourage current program developers to include practice-based research in their efforts.
The AMPED Program. The University of Houston Academic Mentoring Program for Education and Development (AMPED) is a prevention-focused, school-based youth mentoring program for middle school students. It is the product of collaborative development and evaluation efforts of faculty, staff and students at the University of South Carolina, the University of Houston, and public schools in Columbia, SC, and Houston, TX. The history of the AMPED program can be viewed as a series of stages in which the program was iteratively developed based on barriers identified by researchers and practitioners.
Stage 1: The AMPED program was originally developed in response to a 2009 evaluation of a school-based mentoring program that indicated the program was unhelpful (at best) or harmful (at worst) to students ( McQuillin, Smith, & Strait, 2011). In response to these findings, a group of stakeholders systematically re-developed the mentoring program across two iterative cycles.
Stage 2: During the second stage we spent one year redeveloping the failed program. These redevelopment efforts were conducted in collaboration with mentors, key stakeholders, research faculty, and school staff. Core components of evidence-based interventions were strategically selected and implemented into a comprehensive training manual for mentors. The program was then implemented and re-evaluated in the fall of 2010 (McQuillin et. al, 2013).
Stage 3: We revised the program to enhance feasibility and acceptability over the next two years and made revisions to the training manual based on stakeholder feedback. Then, we conducted a small pilot (n=5 middle school students) in spring of 2013 and a re-evaluation in fall of 2013, including 38 middle school students and undergraduate mentors. We are currently preparing to analyze data and once again make indicated program revisions.
Barrier #1, Relationship Duration. One chronic issue that school-based mentoring programs face is the abbreviated mentoring relationship. Relationship duration seems to be a problem to the extent that relationships are prematurely terminated, which may cause disappointment, confusion, or feelings of rejection on behalf of the protégé who may expect a full academic year of mentoring (Grossman, Chan, Schwartz, & Rhodes, 2012). In our dialogue with mentors, program coordinators, and school staff, models of mentoring that have the flexibility to accommodate both lasting and short-term relationships would be more beneficial than models that rely on relationship duration as the primary thrust of the intervention.
Informed by these suggestions, we revised the AMPED program in Stage 2 to operate under 2-3 month “cycles” of mentoring, wherein protégés are informed they will receive a mentor over the course of the semester so as to not promote unrealistic expectations regarding the match length. We have preliminary support from the Stage 2 randomized evaluation that in the short term, this program helps students improve their grades and behavior (McQuillin et al., 2013). The local programs have also been retaining more long-term mentors because they are capable of “casting a wider net” for mentors that may be hesitant because of typical year-long commitments.
Barrier #2, School Priorities. Over the course of the AMPED development process, we have solicited the involvement of public school stakeholders to identify key proximal goals that are valued by schools. Two major themes that emerged are academic preparation and the reduction of school misbehavior, neither of which are common outcomes targeted by traditional school-based mentoring programs. To accommodate these proximal goals we adapted existing practices from motivational interviewing (Miller & Rollnick, 2013), an evidence-based brief counseling style, into a motivational enhancement curriculum used by mentors to help protégés identify and work towards specific goals related to their school behavior and academic pursuits. We also adapted practices from academic interventions related to organization, studying, and planning for tests that were originally developed for treating adolescents with Attention Deficit Hyperactive Disorder (Langberg, Epstein, Urbanowicz, Simon, & Graham, 2008). We chose these two interventions because of the documented evidence for their efficacy, brevity (i.e. deliverable within a single semester), and similar delivery modality to school-based mentoring. School stakeholders have since requested expansion of this service to new schools in our active districts, which we interpret as a reflection of increased acceptability and/or fit with school priorities
Barrier #3, Implementation Problems. After a cohort of 72 mentors completed the revised mentoring program in Stage 2, we identified two areas that warranted additional program refinement. The first challenge was some of the mentors, who were all paraprofessional college students, had difficulty maintaining consistency with the conversational style that we adapted from motivational interviewing. In Stage 3 a small group of mentors and school staff helped us revise the manual and supervision process to include examples of dialogue for common problems associated with core program procedures. The revised manual was received favorably in focus groups, and supervisors rated a majority of the cohort as engaging in conversation style consistent with the AMPED manual.
After conducting focus groups in Stage 3 with protégé’s and mentors, we found the manualized procedures had unintentionally limited some mentor’s interactions with protégés. This left less opportunity for informal interaction or “fun” activities, such as playing games or talking about sports. As a consequence of this realization, we have added prompts in the manual for “free time” and suggested examples of non-structured activities that the protégé may choose. We hope coordinators and supervisors can use these to promote a balance between the manualized procedures and non-structured, protégé-centered activities.
Conclusions and future directions. The AMPED program is an example of how researchers and practitioners can collaboratively work together to create innovative prevention/intervention programs that are catered to fit the unique real-world context of public school systems. After almost five years of systematic and iterative practice-based program development, the AMPED program has transformed a locally-popular, but ineffective, service delivery model into a program that is gaining empirical support and appears to be sustainable and feasible in public school systems. As described in this article, we addressed several barriers to implementation through collaboration with key stakeholders providing practice-based evidence. We suspect we will identify new barriers in the Stage 3 evaluation and anticipate collaboratively attempting to resolve these barriers using practice-based research, further exemplifying the utility of this approach.
Green, L. W. (2007). The prevention research centers as models of practice-based evidence two decades on. American Journal of Preventive Medicine, 33(1 Suppl), S6–8. doi:10.1016/j.amepre.2007.03.012
Grossman, J. B., Chan, C. S., Schwartz, S. E. O., & Rhodes, J. E. (2012). The test of time in school-based mentoring: The role of relationship duration and re-matching on academic outcomes. American Journal of Community Psychology, 49(1-2), 43–54. doi:10.1007/s10464-011-9435-0
Kratochwill, T. R., Hoagwood, K. E., Kazak, A. E., Weisz, J. R., Hood, K., Vargas, L. A., & Banez, G. A. (2012). Practice-Based Evidence for Children and Adolescents: Advancing the Research Agenda in Schools. School Psychology Review, 41(2).
Langberg, J. M., Epstein, J. N., Urbanowicz, C. M., Simon, J. O., & Graham, A. J. (2008). Efficacy of an organization skills intervention to improve the academic functioning of students with attention-deficit/hyperactivity disorder. School Psychology Quarterly, 23(3), 407–417. doi:10.1037/1045-3818.104.22.1687
Leeman, J., & Sandelowski, M. (2012). Practice-based evidence and qualitative inquiry. Journal of Nursing Scholarship, 44(2), 171–179. doi:10.1111/j.1547-5069.2012.01449.x
McQuillin, S. D., Terry, J. D., Strait, G. G., & Smith, B. H. (2013). Innovation in school-based mentoring: Matching the context, structure and goals of mentoring with evidence-based practices. Advances in School Mental Health Promotion, 6(4), 280–294. doi:10.1080/1754730X.2013.832009
McQuillin, S., Smith, B., & Strait, G. (2011). Randomized evaluation of a single semester transitional mentoring program for first year middle school students: A cautionary result for brief, school‐based mentoring programs. Journal of Community Psychology, 39(7), 844–859. doi:10.1002/jcop.20475
Miller, W. R., & Rollnick, S. (2013). Motivational interviewing: Helping people change. New York, NY: Guilford Press.