ATTACHMENT III
Summary of Public Comments and Agency Responses Related to Proposed Amendment to 19 TAC Chapter 97, Planning and Accountability, Subchapter AA, Accountability and Performance Monitoring, §97.1005, Performance-Based Monitoring Analysis System
Comment: The Texas Charter Schools Association (TCSA) recommended that the Performance-Based Monitoring Analysis System (PBMAS) offer greater flexibility for school districts and charter schools that are under the Alternative Education Accountability (AEA) system. The TCSA further stated that the performance level cut-points do not take into account the mission of the school or the population of students served by specific campuses.
Agency Response: The agency disagrees with the recommendation that the PBMAS offer greater flexibility for school districts and charter schools that were under the former AEA system. The PBMAS is designed to align with the requirements and priorities of state and federal programs that apply to all districts serving students in those programs, irrespective of district type or mission. The PBMAS includes components that acknowledge and effectively accommodate the diversity of the state's districts, including charter schools. In addition, the PBMAS annual development process includes a review of all performance level assignments based on longitudinal, publicly available performance data to ensure the PBMAS appropriately identifies districts with performance concerns. Any district (regardless of type or mission) currently performing at the lower end of any PBMAS indicator is likely to benefit from the implementation of improvement efforts given the strong alignment between the PBMAS indicators and the requirements and performance measures outlined in the Carl D. Perkins Act; the Individuals with Disabilities Education Act (IDEA); and Title I, Part A and Title I, Part C of the No Child Left Behind Act.
The agency disagrees that the PBMAS performance levels cut-points do not take into account the population of students served. The PBMAS was specifically developed to take into consideration the diversity that exists across districts, not only in terms of charter schools, but also between small and large districts, rural and urban districts, economically disadvantaged and non-economically disadvantaged districts, and districts with significant numbers of English language learners (ELLs) and districts with small numbers of ELLs. The diversity of the state's districts and charters is accommodated in the PBMAS through a variety of unique components, including performance level cut-points, special analysis, minimum size requirements, and required improvement. The PBMAS indicators are based not only on a performance level standard but also include several variations from that standard as well as differentiated performance level methodologies based on the size of the student group being evaluated. These components effectively allow the PBMAS to evaluate a very diverse set of districts while at the same time generously accounting for distinct circumstances that may exist for any given indicator or district. In addition, the PBMAS annual development process includes a review of all performance level assignments based on longitudinal, publicly available performance data to ensure the PBMAS appropriately accommodates the diversity of students and districts that exist in the state.
Comment: The TCSA commented that two PBMAS components (special analysis and minimum size requirements) require districts to have at least 30 students in the relevant segment of the student population, which presumably protects smaller campuses and districts; however, larger charter districts operating under the AEA system, which specifically target at-risk populations, have many more than 30 students populating certain indicators. Additionally, the commenter recommended implementing a percentage of total population minimum size criteria as opposed to the current absolute number minimum size requirement of 30 students.
Agency Response: The agency disagrees. The comment misconstrues that the special analysis process and minimum size requirements require districts to have at least 30 students in the relevant segment of the student population. The minimum size requirements and the special analysis process are components of the PBMAS that ensure a guiding principle of the PBMAS, maximum inclusion, is effectively implemented through the use of appropriate methods to analyze the performance of districts with both small and large numbers of students, including districts with fewer than 30 students in the relevant segment of the student population. The current minimum size requirements and the special analysis process effectively ensure an appropriate scope of the state's monitoring system, and the agency disagrees with the recommendation to implement a percentage of total population minimum size criteria as it would significantly, and inappropriately, reduce the number of districts evaluated.
Comment: The TCSA recommended applying filters for charters operating under the AEA to certain data sets before performance indicators are calculated, including the Recommended High School Program and Distinguished Achievement Program (RHSP/DAP) indicators for limited English proficient (LEP), Career and Technology Education (CTE), and Title I sub-groups and the annual dropout rate indicators for LEP, CTE, and Title I sub-groups.
Agency Response: The agency disagrees with the recommendation that filters be applied to certain data sets for charters operating under the former AEA system. The performance level cut-points implemented in PBMAS take into account the diversity that exists across districts, not only in terms of charter schools, but also between small and large districts, rural and urban districts, economically disadvantaged and non-economically disadvantaged districts, and districts with significant numbers of ELLs and districts with small numbers of ELLs.
The PBMAS' wide range of performance level cut-points allows for significant variation in performance across diverse districts and charters. Depending on the particular program area within PBMAS, between 50% and 97% of a district's students can earn a minimum high school diploma and still avoid the lowest performance level designation on the PBMAS RHSP/DAP indicators. Depending on the particular program area within PBMAS, districts with dropout rates that are nearly five times the state rate can still avoid the lowest performance level designation on the PBMAS annual dropout rate indicators. In addition, the PBMAS annual development process includes a review of all performance level assignments based on longitudinal, publicly available performance data to ensure the PBMAS appropriately identifies districts with performance concerns.
Statutory requirements (TEC, §39.057(a)(10)) state that the commissioner shall authorize special accreditation investigations to be conducted "when excessive numbers of students graduate under the minimum high school program." As such, a continued focus on low levels of RHSP/DAP diploma rates remains a state priority. Any district (including districts in the former AEA system) currently performing at the lower end of the PBMAS RHPS/DAP diploma indicator is likely to benefit from the implementation of improvement efforts given this statutory requirement.
Finally, under 19 TAC Chapter 100, Charters, Subchapter AA, Commissioner's Rules Concerning Open-Enrollment Charter Schools, Division 1, General Provisions, §100.1011(15), ". . .the terms of an open-enrollment charter include. . . (B) all applicable state and federal laws, rules, and regulations. . . (E) to the extent they are consistent with subparagraphs (A)-(D) of this paragraph, all statements, assurances, commitments, and/or representations made by the charter holder in writing in its application for charter, attachments, or related documents, or orally during a public meeting of the SBOE or any of its committees." Thus, to the extent that the content of a charter application conflicts with current law respecting graduation requirements, the current law controls over the charter application. This rule applies to all charter holders that accepted funding under TEC, §12.1071.
Comment: The TCSA commented that the special education representation indicator should be changed to the special education identification indicator, which would encourage all schools to appropriately identify special education students and thereby help the Texas Education Agency (TEA) readily identify particular schools that are "over-identifying" students for special education. The TCSA also stated the change would allow charters serving a larger percentage of special education students to continue to abide by those students' required accommodations and modifications without being unfairly sanctioned.
Agency Response: The agency disagrees with the recommendation. The PBMAS special education representation indicator is appropriately aligned with the IDEA authorizing statute and its corresponding federal performance measures. To the extent part of the comment addresses sanctions, those issues are outside of the scope of the current rule proposal; however, districts are required to adhere to all state and federal requirements pertaining to serving students with disabilities irrespective of districts' anticipated performance level assignment on any PBMAS indicator.
Comment: The TCSA commented that the TEA should increase each school system's opportunity to provide additional data or other inputs prior to TEA's assignment of a PBMAS intervention stage.
Agency Response: The agency cannot address this comment because intervention stages are outside of the scope of the current rule proposal.
Comment: The Texas Classroom Teachers Association (TCTA) commented that the Discretionary Disciplinary Alternative Education Program (DAEP) Placements, Discretionary Placements to In-School Suspension, and Discretionary Placements to Out-of-School Suspension indicators could likely have the unintended effect of encouraging more Admission, Review, and Dismissal (ARD) committee overrides of teacher removals of students from class in order to avoid punitive action from the state for exceeding these indicators' standards. The commenter requested that an additional 2012 PBMAS indicator be added ("ARD Committee Overrides of Teacher Removals") to mitigate this unintended effect.
Agency Response: In the absence of verifiable data to substantiate the alleged unintended effect, the agency disagrees with the recommendation to add the additional indicator to the PBMAS at this time. In addition, to ensure continued validity of the PBMAS, any future indicators must, among other considerations, be quantifiable, reliable, valid, and derivable from official agency data sources.
Comment: The TCTA recommended adding an indicator to the 2012 PBMAS that would evaluate whether districts have established a procedure, as required by 19 TAC §89.1075, by which a teacher can request assistance in implementing a special education student's Individualized Education Program (IEP).
Agency Response: The agency disagrees with the recommendation to add the additional indicator to the 2012 PBMAS. Indicators in the PBMAS are designed to evaluate student performance and program effectiveness. Issues of compliance with required district procedures are more appropriately evaluated as part of a program compliance review the agency may initiate when performance concerns are identified and/or under the responsibility given to the board of trustees under TEC, §7.028(b), for ensuring that the district or school complies with all applicable requirements of state educational programs.
Comment: The TCTA recommended adding an indicator to the PBMAS that would evaluate the number of complaints filed by staff (and other parties) against a district with TEA in order to integrate TEA's special education complaints system with the agency's monitoring system.
Agency Response: The agency disagrees with the recommendation to add an additional indicator to the PBMAS at this time, given the numbers are too small to integrate into a statewide monitoring system. To ensure continued validity of the PBMAS, any future indicators must have statewide applicability, cannot be overly limited by small numbers, and must have an expectation that a sufficient number of districts have viable numbers to be evaluated on the indicator. As reported in the most recent Annual Performance Report, 327 complaints were filed with only 150 reports issued. The number of complaints filed by staff and third parties is even smaller.
Comment: The TCTA recommended adding indicators to the 2012 PBMAS regarding circumstances meriting special accreditation investigations, as added by TEC, §39.057(a)(9)-(12): (9) when a significant pattern of decreased academic performance has developed as a result of the promotion in the preceding two school years of students who did not perform satisfactorily as determined by the commissioner under Section 39.0241(a) on assessment instruments administered under Section 39.023(a), (c), or (l); (10) when excessive numbers of students graduate under the minimum high school program; (11) when excessive numbers of students eligible to enroll fail to complete an Algebra II course or any other course determined by the commissioner as distinguishing between students participating in the recommended high school program from students participating in the minimum high school program; and (12) when resource allocation practices evaluated under Section 39.0821 indicate potential for significant improvement in resource allocation.
Agency Response: The agency agrees in part and disagrees in part. The agency agrees that the circumstances described under TEC, §39.057(a)(10), form the basis for appropriate PBMAS indicators. The RHSP/DAP indicators included in the PBMAS since 2004 identify districts with excessive numbers of students graduating under the minimum high school program since low rates of graduates with RHSP/DAP diplomas indicate high rates of graduates with the minimum high school diploma. The agency disagrees that the other circumstances described under (9) and (11) are appropriate indicators for the 2012 PBMAS. The state's new State of Texas Assessments of Academic Readiness (STAAR) system and new end-of-course graduation requirements are in their first year of implementation, and it is still too soon to consider whether the circumstances described under (9) and (11) form the basis for appropriate, reliable, and valid PBMAS indicators. As greater numbers of students are subject to these requirements and as more data are available for analysis, the agency will consider whether additional indicators are appropriate for the PBMAS. Finally, the circumstances described under (12) are calculated and evaluated by the comptroller in a system that is outside of the current rule proposal.
For more information, email rules@tea.state.tx.us.