In one of our earlier Nuggets, we talked about evidence-based (EB) interventions and policies (Prevention Nugget #10) that have been rigorously investigated and found to be effective for use in the primary settings where prevention services are offered--family, school, workplace, community including entertainment venues and the media, and health care settings . We introduced registries as important resources for helping to select an EB intervention or strategy to meet the specific needs of your target populations. Of course, you usually reach this stage after assessing your community needs and resources and determining your priority focus and audience and are ready to find the intervention best suited to your goals and objectives.
So, how can prevention registries help you identify prevention interventions and policies that meet the needs of your community? What are they? Where are they? How do they differ? How do I use them?
Registries are not just lists of prevention programs. They include manualized and, in some cases, environmental prevention programming that:
Meet specific criteria of research evidence
Involve a review process using groups of independent raters.
Registries vary considerably in these criteria, in guidelines for the independent reviews, as well in the qualifications of the raters. Registries are available at the International level (e.g., Xchange Prevention Registry of the European Monitoring Centre for Drugs and Drug Addiction), National level (e.g., BluePrints, Crime Solutions), and at the state level (e.g., The California Evidence-Based Clearinghouse). The urls for these sites are provided below.
Mission and Criteria for Entry. Most registries include programs that meet the most rigorous criteria and those that do not meet all of these criteria but are still considered very effective. The following describes the mission of the registry and their criteria for program entry:
The mission of Blueprints for Healthy Youth Development is to provide a comprehensive registry of scientifically proven and scalable interventions that prevent or reduce the likelihood of antisocial behavior and promote a healthy course of youth development and adult maturity. They also advocate for evidence-based interventions. Blueprints rates interventions as either Promising, Model or Model Plus. The criteria for these categorizations are:
Promising interventions meet the following standards:
- Intervention specificity: The intervention description clearly identifies the intended outcome(s), whether specific risk and/or protective factors are targeted to produce this change, the population for which the intervention is intended, and how the components of the intervention work to produce this change.
- Evaluation quality: The evaluation trials produce valid and reliable findings. This requires a minimum of (a) one high-quality randomized control trial or (b) two high- quality quasi-experimental evaluations.
- Intervention impact: The preponderance of evidence from the high-quality evaluations indicates a significant positive change in intended outcomes that can be attributed to the intervention and there is no evidence of harmful effects.
- Dissemination readiness: The intervention is currently available for dissemination and has the necessary organizational capability, manuals, training, technical assistance and other support.
Model interventions meet the Promising intervention criteria PLUS:
- Replication: A minimum of (a) two high-quality randomized control trials or (b) one high-quality randomized control trial plus one high-quality quasi-experimental evaluation.
- Long-term follow-up: Positive intervention impact is sustained for a minimum of 12 months after the program intervention ends.
Model Plus interventions meet Promising and Model standards PLUS:
- Independent Replication: In at least one high-quality study demonstrating desired outcomes, authorship, data collection, and analysis has been conducted by a researcher who is neither a current or past member of the program developer’s research team and who has no financial interest in the program.
The goal of the California Evidence-Based Clearinghouse (CEBC) for Child Welfare is to provide a searchable database of programs that can be utilized by professionals that serve children and families involved with the child welfare system. The CEBC uses a different rating scale for programs and practices that are listed in the registry. This scale is from 1 to 5. A scientific rating of 1 represents a practice with the strongest research evidence and a 5 represents a concerning practice that appears to pose substantial risk to children and families. Some programs do not currently have strong enough research evidence to be rated on the Scientific Rating Scale and are classified as NR - (Not able to be Rated).
Well-Supported by Research Evidence
- At least 2 rigorous randomized controlled trials (RCTs) with non-overlapping analytic samples that were carried out in the usual care or practice settings have found the program to be superior to an appropriate comparison program on outcomes specified in the criteria for that particular topic area
- In at least one of these RCTs, the program has shown to have a sustained effect of at least one year beyond the end of treatment, when compared to a control group.
Supported by Research Evidence
- At least one rigorous RCT in a usual care or practice setting has found the program to be superior to an appropriate comparison program outcomes specified in the criteria for that particular topic area.
- In that RCT, the program has shown to have a sustained effect of at least six months beyond the end of treatment, when compared to a control group.
Promising Research Evidence
- At least one study utilizing some form of control (e.g., untreated group, placebo group, matched wait list) has done one of the following:
- established the program's benefit over the control on the outcomes specified in the criteria for that particular topic area,
- found it to be comparable on outcomes specified in the criteria for the topic area to a program rated 3 or higher on this rating scale in the same topic area,
OR
- found it to be superior on outcomes specified for that particular topic area to an appropriate comparison program. Promising Research Evidence
Evidence Fails to Demonstrate Effect
- Two or more RCTs with non-overlapping analytic samples that were carried out in usual care or practice settings have found that the program has not resulted in improved outcomes specified in the criteria for that particular topic area, when compared to usual care.
- The overall weight of evidence does not support the benefit of the program on the outcome specified in the criteria for that particular topic area.
Concerning Practice
- One or more of the following statements is true:
- if multiple outcome studies have been conducted, the overall weight of evidence suggests the program has a negative effect on the target population being served or on outcomes specified in the criteria for that particular topic area;
- there is case data suggesting a risk of harm that: a) was probably caused by the program and b) was severe and/or frequent;
OR
- there is a legal or empirical basis suggesting that, compared to its likely benefits, the program constitutes a risk of harm to those receiving it.
NR - Not Able to be Rated
- The program does not have any published, peer-reviewed study utilizing some form of control (e.g., untreated group, placebo group, matched wait list study) that has established the program's benefit over the control on outcomes specified in the criteria for that particular topic area or found it to be comparable to or better than an appropriate comparison program on outcomes specified in the criteria for the topic area.
- The research evidence for programs listed NR cannot be established at this time. This does not mean the programs are ineffective, but rather that there is not research evidence that meets the criteria for any other level on this rating scale.
The Xchange Prevention Registry of the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA is an online registry of thoroughly evaluated prevention interventions. The registry includes manualized interventions for which good European evaluation studies showed beneficial outcomes relating to substance use, as well as programmes for youth offending and bullying. The registry also includes programmes of US origin with Blueprints ratings, and local environmental prevention strategies (starting in 2020). The Xchange Prevention Registry uses different terms for its ratings. Only interventions that target substance-related/violence/delinquency outcomes are included in the registry.
Then:
Is the intervention still active or able to be used in Europe?
Is the intervention clearly defined (outcomes, target group, risk and protective factors, logic model or theoretical rationale)?
Is there at least one suitable evaluation study in Europe (RCT, Quasi-experimental design or interrupted time series) for this intervention?
Are the expected outcomes measured?
If the interventions meet these criteria they are then rated by independent raters using the following categories:
Beneficial: Interventions for which convincing, consistent and sustained effects for relevant outcomes are in favour of the intervention as found in two or more studies of excellent quality in Europe.
Likely to be beneficial: Interventions for which convincing and consistent effects for relevant outcomes are in favour of the intervention as found in at least one evaluation study of excellent quality in Europe.
Possibly beneficial: Interventions for which some effects for relevant outcomes are in favour of the intervention as found in at least one evaluation study of acceptable quality in Europe. An intervention ranked as ‘possibly beneficial’ is suitable for application in the context of more rigorous evaluations.
Additional studies recommended: Interventions for which concerns about evaluation quality or consistency of outcomes in Europe make it difficult to assess if they are effective or not, even if outcomes seem to be in favour of the intervention.
Unlikely to be beneficial: Interventions for which at least one evaluation of excellent quality in Europe shows convincing evidence of no or harmful effects on relevant outcomes.
Possibly harmful: Interventions for which some effects for relevant outcomes of the intervention are considered harmful, as found in at least one evaluation study of acceptable quality in Europe. An intervention ranked as 'possibly harmful' is unsuitable for application except within a framework of other priorities and with rigorous and strictly supervised evaluations.
Coverage and Searching. By and large there is overlap across the registries for many of the stronger prevention programs. The challenge is to find programs that fit the needs of your target population or community. The registries provide the tools to select programs by outcome, by the characteristics of the target population, by setting, or by risk and/or protective factor. Once you enter your information, the registries provide a description of the program and of the evaluation research and contact information to learn more about possible training and accessing materials to deliver the intervention.
Accessing Registries:
Blueprints for Healthy Youth Development. Blueprintsprograms.org
The California Evidence-Based Clearinghouse. https://www.cebc4cw.org/
Xchange Prevention Registry. https://www.emcdda.europa.eu/best-practice/xchange