Evidence Standards

The Best Practices Clearinghouse (BPC) Evidence Standards, developed jointly by TEA and project partners, Resources for Learning (RFL) and ICF International (ICF), provide an intuitive approach for understanding the evidence base of presented summaries. Because the BPC will be used by practitioners to determine which programs can be replicated in their districts, the goal of the BPC Evidence Standards is to help stakeholders understand the evidence associated with a practice. The BPC does not rank evidence; rather, the BPC provides the information needed to make effective decisions. The Evidence Standards are working guidelines and are modified as necessary to reflect feedback from the field and closer alignment with educator needs.

Based on how the sum of evidence is aligned with the BPC Evidence Standards, each summary will be designated as one of three BPC Evidence Types. The BPC Evidence Standards are defined here:

  • Relevance of Outcomes: Outcomes must be directly relevant to the intervention being considered. Although BPC staff will often determine the effectiveness of an intervention on multiple outcomes, only the most relevant outcomes will be identified. For example, the BPC will not report reading outcomes for a math intervention.
  • Strength of Outcome Measures: Outcomes must have a sufficient degree of validity to be considered. Standard TEA measures (e.g., TAKS performance or completion rates) will be considered valid. This criterion will become more difficult to apply to locally developed measures. As a general rule, the validity of a measure must be established, either through reporting its psychometric properties (e.g., internal consistency, inter-rater reliability) or through a demonstration at baseline that the measure produced roughly equivalent results between treatment and comparison groups matched on demographic and at least one other academic measure besides the outcome.
  • Eligible Designs: The BPC will accept a wide range of approaches to establishing supporting evidence, primarily use of trend data but also including randomized controlled trials, quasi-experimental studies, single case designs, and pre-post designs. Regardless of the design, the approach should establish some basis of comparison, either with an equivalent comparison group (e.g., through a randomized controlled trial or a quasi-experimental study) or using subjects as their own control (through trend data). In each method, with the exception of trend data, baseline equivalence needs to be established in order for the analysis to be considered valid. This can be achieved either through careful matching procedures, or, most often, by making statistical adjustments for pretest differences.
  • Stability of Results: The “Established Best Practice” evidence type designation requires that results be proven across time (i.e., at least three years of post-intervention data, with positive effects for more than half of the time points).
  • Literature Base: The BPC also provides standards for a strong evidence base as represented in the literature. By drawing upon previous research on similar practices, the BPC will be able to establish whether a particular intervention or strategy was judged effective in other settings. This evidence base will be particularly relevant in the “Emerging Practice” and “Theory-Based Practice” evidence designations.

BPC Evidence Types

The three BPC Evidence Type designations are designed to incorporate the Evidence Standards and provide a simple, intuitive summary of the evidence and the results. Each Evidence Type is described here: 

  • Established Best Practice: This quantitative evidence type includes validated data for at least three years that indicate a positive trend in outcomes within the school/district implementing the program or among students who are in the targeted group. The effectiveness of these practices is validated by standard TEA measures, TEA and local program staff reports, and, since 2010, statistical significance testing (state and peer campus/district comparisons, where appropriate).
  • Emerging Practice: This largely qualitative evidence type is for practices that are in the early stages of implementation and that have not had enough time to demonstrate a positive trend over three time periods. TEA and local staff reports and up to two years of data will be used to validate these types of research-based best practices.
  • Theory-Based Practice: Practices assigned this evidence type include those that staff identify as effective through observation and that are supported by a theoretical base (but not necessarily an empirical research base). This evidence type is designed to capture innovations in practice that do not easily lend themselves to quantitative measurement or that are viewed by practitioners and researchers alike as good practice but that have not yet been proven through empirical research.

Summary of Evidence Standards by BPC Evidence Type

  Established Best Practice Emerging Practice Theory-Based Practice

Relevance of Outcomes

Outcomes must be relevant to intervention

Outcomes must be relevant to intervention

Outcomes must be relevant to intervention

Strength of Outcome Measures

Quantitative outcomes with demonstrated validity

Limited quantitative outcomes may be included; outcomes may also be supported by staff observations of whether a practice works and a literature base

Outcomes can be more subjective

Eligible Designs

Trend data (three years), randomized controlled trial, quasi-experimental study, single case design, or pre-post (two years of post-implementation data) Evidence base will be established through a combination of primary observation by TEA and local staff, a well-established literature base, and trend data (up to two years) No quantitative data needed: evidence base will be established through a combination of primary observation (from school district staff) and a theoretical base

External Validity or Generalizability of Results

At least three years of data (two years post-implementation for pre-post data or three years of trend data) with positive results, standard TEA measures, TEA and local program staff reports, and since 2010, statistical significance testing (state and peer campus/district comparisons, where appropriate)

TEA and local staff reports, up to two years of trend data, and a well-established literature base

No external validity considerations needed

Literature Base

Literature base should support the practice’s strategies

Literature base should support the practice’s strategies

Innovative practices may be relatively new, so a quantitative or qualitative literature base is not required. These practices, however, must have a theoretical base established in published literature

Information from these Evidence Standards will be presented in each Best Practice Summary and will include, when possible and appropriate, statistical significance tests of comparisons with state and peer district/campus averages. This evidence and the contextual information outlined in each summary will provide a sufficient level of detail for users to make an assessment about whether to pursue local implementation of a featured intervention.