Administrators’ Assessment Toolkit: Measuring Student Academic Performance

View the local program manager’s version of this toolkit.

 Background

Toolkit Administrators' Assessment Toolkit (print version) State Assessment Practices Table (print version)

In 2001, President Bush’s No Child Left Behind (NCLB) education plan proclaimed that the Nation’s education programs would be held accountable for student outcomes—including the educational outcomes of students who are neglected or delinquent (N or D). NCLB requires that programs funded by Title I, Part D, be evaluated at least once every 3 years. The goal of the evaluation, according to the legislation, is to “…determine Part D’s impact on the ability of participants to:

• Maintain and improve educational achievement;
• Accrue school credits that meet State requirements for grade promotion and secondary school graduation;
• Make the transition to a regular program or the education program operated by an LEA; and
• Complete secondary school (or secondary school equivalency requirements) and obtain employment after leaving the institution” [1].

While NCLB has increased the amount of attention focused on data and assessment, many providers of education to youth who are N or D are not yet prepared to address the related accountability measures.

Ideally, education programs collect student performance data not only to meet their reporting requirements, but also to ensure that the academic program is tailored to meet individual student needs. State administrators can help their education programs and facilities by understanding the capacity of each and the challenges they face in collecting and using assessment data, identifying and leveraging resources available at the State and local level, and providing staff training and technical assistance.

Why is pre-post assessment important?

When used properly, pre-post assessment data have the potential to positively impact all of those involved in the education of youth who are N or D. At the classroom level, pre-post testing allows teachers to track individual student performance, catch small problems before they become big problems, and effectively tailor instruction to individual student needs. Consistently using assessment data to update students’ academic records also helps to ensure that, even when students leave a facility with little advanced notice, the documentation they bring to their next placement is an accurate reflection of their progress and ability. For program managers and education directors, a review of pre-post assessment results at the school level can provide valuable information on the status of their education program, allowing them to identify effective practices and to direct resources and interventions to those teachers and students with the greatest needs.

The same benefits gained by collecting pre-post assessment data at the individual, classroom, and facility levels also extend to the State level. Just as reviewing assessment data across a single facility can provide those administrators with valuable information to guide program improvement, State administrators can conduct similar analysis across multiple programs or facilities. In doing so, administrators can identify particularly effective programs to serve as models for others; they can also identify struggling programs, facilitating the effective allocation of resources and technical assistance.

Additionally, having current and accurate pre-post assessment data readily available affords administrators the opportunity to share information regarding their State’s N or D education programs with a variety of stakeholders. The U.S. Department of Education, under Title I, Part D, now requires States to report information on student academic performance on an annual basis. The administration of pre-post assessments at the facility and local district level can facilitate the accurate and efficient collecting and reporting of N or D data at the State level.

Performance data can also help State administrators communicate the progress made by the students and education programs in their State when advocating for additional funding. Many stakeholders and members of the public are often more familiar with the “negative outcome data” related to youth who are N or D, such as crime rates, recidivism rates, and the costs associated with incarcerating juvenile offenders. Yet, research has repeatedly shown that academic achievement can be linked to reduced rates of recidivism and increased prosocial behavior [2]. Demonstrating increases in the academic performance of youth who are N or D can be a powerful tool for States to communicate the success of their education programs and the positive impact these programs can have on public safety to those who make decisions regarding the allocation of funding and resources.

How can I make the best use of my State’s data?

As outlined in the previous section, monitoring pre-post assessment data provides States with greater information from which to make decisions about institutional priorities and to efficiently allocate resources. However, in order for pre-post assessment data to be useful for demonstrating student progress, States must actively examine both pre-post assessment procedures and results across the various N or D education programs in the State.

Beginning at the End

As a State administrator, the best way for you to start thinking about your data collection processes is by envisioning what you would like to be able to do with the data once you have it. Consider creating templates of the different reports you might draft to communicate your data to different stakeholders and consumers. What types of comparisons will your audiences find compelling? What types of comparisons do you need to make to get the answers you seek? NDTAC has found that the best way to think through this process is by developing graphs with false (estimated) data. This process will help you identify exactly what data you need to collect as well as the procedures that should be in place to ensure that the data is useful and accurate.

Selecting Assessments

Following are key questions for you to consider when reviewing assessment practices in your State:

• Are the assessments in use in N or D facilities appropriate for pre-post testing?
• Can the results from the different assessments in use in the State be aggregated for reporting purposes (i.e., is it possible to equate and/or combine results from one assessment with results from another)?
• What are other States doing to improve this process that might work in my State?

In some States, assessment activities are focused on administering an annual State-required assessment. Unfortunately, State assessments are generally not appropriate tools for N or D programs to use to demonstrate the ongoing performance of their students for several reasons:

• Scores are not used to see individual student gain; scores are examined to see
  how all students at one grade level perform compared to that same grade level in another year.
• The State assessment is usually not designed to be given more than once per year; hence, it is not an appropriate pre-post test.
• Most of the students will have entered and exited the N or D system before the one-year mark (only about 10 percent of students are in facilities for more than a year) and may not be enrolled during the administration of the statewide test [3].

While the results of State assessments are useful in many other ways, they are not intended or designed to measure the academic performance of specific students within a given school year or reporting year. Please see NDTAC’s A Brief Guide to Selecting and Using Pre-Post Assessments for additional information regarding selecting appropriate pre-post assessments.

Collecting Assessment Data

The questions in the table below identify aspects of pre-post assessment that should be taken into consideration when evaluating your State’s practices for collecting and using assessment data. Completing the table provides an opportunity to reflect on which data related to pre-post assessment practices and results are currently available. It can also create a written record of such information for future reference, as well as a baseline for tracking your State’s progress in these areas.

Table: State Assessment Practices

Data Collection

How many educational programs administer Title I, Part D programs in the State? ________________ How many of these programs utilize pre-post assessment to collect academic performance data in reading and math? ________________ How many of these programs are able to report their pre-post assessment results?________________

Reading Assessment

Are there guidelines or legislation at the State level about what pre-post assessments programs can use for testing in reading? Yes. No, programs may choose the assessments. If yes, did the State conduct a review of possible reading assessments? When was the review conducted? Which assessments are currently in use to track student performance in reading in the State? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________

Math Assessment

Are there guidelines or legislation at the State level about what pre-post assessments programs can use for testing in math? Yes. No, programs may choose the assessments. If yes, did the State conduct a review of possible math assessments? When was the review conducted? Which assessments are currently in use to track student performance in math in the State? __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________

Performance Data

When does the State request that State agencies and local education agencies report student performance data (e.g., date or time of year)? What is the average increase in reading scores among students who are N or D in your State? __________________ What is the average increase in math scores among students who are N or D in your State? __________________

What’s next?

Review the information collected in the table above to gain a better understanding of the capacity of programs in your State to collect pre-post assessment data and identify current procedures and legislation in the State that may impact this capacity.

Begin by taking note of the questions in the preceding table that have been left unanswered. Ask yourself:

• Why is this information missing?
• Do you, as the State administrator, need to become more involved in the data collection and submission process, or are programs simply not able to provide the information?

The second step is to look at the questions that are answered along with any other data you regularly collect from the N or D education programs in your State (through monitoring visits or annual reports):

• Are there certain programs or facilities that are unable to provide student performance data? Are specific types of programs (neglected, at risk) having more difficulty than others?
• Are there certain programs or facilities that appear to be doing a particularly good job with their pre-post assessment procedures and data collection?
• Are there certain programs or facilities that appear to be doing a particularly effective job in raising student performance levels in math and reading?
• Are there programs or facilities that appear to be struggling in these areas?
• Are there areas in which your State as a whole appears to be particularly strong?
• Are there areas in which your State as a whole appears to need improvement?
• Does a large percentage of Title I, Part D funds go to one or two programs? If so, are there changes that can be made to these programs that would have a significant positive impact on the availability or quality of the data in the State overall?

Answering these questions should provide you with a clearer picture of the status of pre-post assessment practices in education programs serving youth who are N or D in your State. The remainder of this toolkit focuses on how State administrators can help their programs improve their pre-post assessment procedures and provides resources for further assistance.

How can State administrators help?

A key role of the State administrator in the pre-post assessment process is to provide training and technical assistance to sites and build programs’ capacity to effectively and efficiently collect and use pre-post assessment data. There are a number of ways that you as a State administrator can help:

1. Provide facilities with a copy of the program manager section of this toolkit and encourage them to use it. Included in the program manager’s toolkit are a number of tables (available both electronically and in hard copy) that can help facilities monitor their pre-post assessment practices and results. The electronic version of these data tables also generates graphs from the data inputted.
2. Consider moving to a uniform pre-post assessment across the State. Using only one test allows you to aggregate data across the State’s N or D programs and to compare your State’s overall performance to that of other States that have statewide information (e.g., Florida). A uniform assessment will also allow you to determine which programs have the highest scores and potentially identify innovative practices at those sites that you can then share with other programs. In addition, a single assessment allows for the coordination of staff training and technical assistance efforts.
   
3. Use the data that you collect. Send the data that you submit to the Federal Government back to your sites. If you collect additional data or conduct analyses with the data, share that information with the sites. They may not realize how they compare to their peers.
   
4. Work with facility administrators and/or program managers to identify local resources that can help them to improve their assessment and analysis practices. Especially important is connecting local personnel to the training specific to their assessments. Many publishing companies offer training on the administration of their assessments and the interpretation of the results. In addition, many school districts employ assessment experts who can provide training and technical assistance for those N or D programs and facilities in their districts.
   
5. Become familiar with the assessments in place in the N or D education programs in your State and assemble a list of professional development and technical assistance contacts specific to those assessments.
   
6. Help facilities serve as a resource for one another. If the data indicate that a particular facility is doing an effective job in collecting and utilizing assessment data, enlist that facility as a resource for those that are struggling.

Where can I go for additional information?

Reporting Examples From the States

A number of State juvenile justice agencies produce annual or biannual reports documenting student academic achievement. The following three States all include a discussion of student gains in math and reading.

Florida: The most recent Developing Effective Education in Department of Juvenile Justice and other Dropout Prevention Programs annual report is available on the Juvenile Justice Education page of the FL Department of Education Web site.

Texas: The Juvenile Justice Alternative Education Programs Performance Assessment Report is available at: http://www.tjjd.texas.gov/publications/reports/TJPCMISC0310.pdf.

Wisconsin: The most recent Wisconsin Department of Corrections, Division of Juvenile Corrections' (DJC) Programs and Student Outcomes Annual Report is available under the "Reports" section on the DJC homepage.

 

---

[1] Title I, Part D, Subpart 3, Section 1431 (A) of the Elementary and Secondary Education Act of 1965 (20 U.S.C. 6301 et seq.). Additional information on Title I, Part D legislation can be found on the NDTAC Web site at: http://www.neglected-delinquent.org/nd/resources/policy_portal.asp

[2] Katsiyannis, A., & Archwamety, T. (1999). Academic remediation/achievement and other factors related to recidivism rates among delinquent youths. Behavioral Disorders, 24, 93-101. White, C. (2002). Reclaiming incarcerated youth through education. Corrections Today, 174-188. Wolford, B., Purnell, B., & Brooks, C. C. (2000). Educating youth in the juvenile justice system. Richmond, KY: National Juvenile Detention Association.

[3] Snyder, H., & Sickmund, M. (1999). Juvenile offenders and victims: 1999 national report (ch. 7, p. 201). Washington, D.C.: Office of Juvenile Justice and Delinquency Prevention. Data Source: Office of Juvenile Justice and Delinquency Prevention. Census of Juveniles in Residential Placement 1997 [machine-readable data file]. Washington, D.C.: OJJDP, 1998. https://www.ncjrs.gov/html/ojjdp/nationalreport99/toc.html

 

Published April 2006 [modified May 2012].