Program Managers’ Assessment Toolkit:
Measuring Student Academic Performance

Background

Toolkit Program Managers' Assessment Toolkit (print version)

In 2001, President Bush’s No Child Left Behind (NCLB) education plan proclaimed that the Nation’s education programs would be held accountable for student outcomes across multiple domains including areas such as the education of youth who are neglected or delinquent (N or D) that have been largely overlooked in the past. At the most fundamental level, NCLB mandates that young people residing in juvenile institutions receive the same educational opportunities as students in the general population.

While schools within juvenile institutions share many of the same characteristics and challenges of underperforming schools throughout the country—a disproportionately low income and minority population, a high proportion of English language learners (ELLs), and low parent involvement—educators of youth who are N or D face a variety of challenges unique to their environment. These include multiple-ability levels in a single classroom, a constantly changing classroom population, and an institutional philosophy that often places education second to security [1].

The educational instruction in N or D programs must be tailored to best meet the specific needs of their students, including a high proportion of students with learning disabilities and students who have been out of the classroom for months or years. As a result, N or D programs must have tools and processes in place to accurately assess both student academic status upon entry and the progress students make while enrolled. Ideally, these processes are built around the administration of a pre-post assessment in reading and math upon entry and (minimally) upon exit.

This Assessment Toolkit will discuss the benefits of pre-post assessments for N or D program managers and educators of youth who are N or D and provide tools and resources to help facility administrators use pre-post assessment for student and program improvement.

Why is collecting pre-post assessment data important?

Academic assessment is an integral part of any student’s educational experience. Consistently measuring and updating the academic achievement levels of students in education programs for youth who are N or D is especially important. As students leave facilities with little or no advanced notice, it is vital that their educational records are updated frequently so that the documentation that they bring to their next placement accurately reflects the progress they have made. In addition, using assessment data to make decisions allows teachers to target academic instruction and better prepare students for their transition out of the facility.

Accurately Place Students

Many students enter juvenile justice placement with incomplete academic histories and records. In order to accurately place new students in an appropriate educational program, teachers and administrators must have an understanding of their academic strengths and needs. Administering a pretest is an efficient way to measure new students’ abilities and begin planning instructional strategies. As the authors of From Incarceration to Productive Lifestyle, Making the Transition: An Instructional Guide for Incarcerated Youth Education note: “Facilitating learning is basically a process that begins with identifying what students need…They come from different educational backgrounds and learn at different paces. The teacher’s ability to assess these differences will determine what, how, and when students will learn” [2]. The time that youth spend in residential programs is, for many, a rare opportunity to improve academically; efficient placement in the educational program will allow teachers and administrators to maximize that opportunity.

Monitor Student Progress

In many facilities, the classroom is full of learners of a variety of ages, experiences, and abilities. In some cases, the students come from and will return to a variety of schools and districts, each with their own standards and expectations. Many students involved in the juvenile justice or child welfare systems are functioning far below grade level and/or have special education needs that require an Individualized Education Plan (IEP) [3]. In order to meet these diverse student needs, correctional educators must be able to target instruction at the individual level. “Differentiated instruction can be the foundation of effective instruction… In order to plan differentiated instruction, teachers must have information based on learners’…strengths and needs” [4]. That information can be provided by pre-post assessment results.

Pre-post testing allows for real-time progress monitoring. Because multiple posttests can be administered throughout a student’s enrollment, educational gains can be monitored and instruction can be adjusted appropriately. Measuring academic progress through appropriately administered pre-post tests can be a powerful tool in providing teachers feedback about how to better meet students’ academic needs. In Assessing Formative Assessment, Matt Villano notes that “as teachers finish each lesson, instead of waiting for a high-stakes test at the end of the year, they do the assessment right then and there” [5]. Juvenile justice teachers in particular cannot wait for the results of annual State-required tests to assess student progress as many of their students will not be in the facility long enough for those results to be applicable. Frequent academic assessment helps teachers catch small problems before they escalate and ensures that students are actually learning what they are being taught.

Document and Share Student Progress

Pre-post testing is especially valuable to students who are N or D because it allows teachers to document their individual academic gains. Not only can teachers use the gains students make from pretest to posttest as a powerful motivator to keep students invested in their education, assessment scores also communicate that progress to those outside the classroom, including parents, administrators, and educators at a student’s next placement. Far too often, the hard work done by students and teachers to prepare for public school is negated when students arrive without the proper documentation of their achievement. At the very least, students waste valuable school days as they sit through a battery of placement tests, spend time in the wrong classes, or are required to repeat classes or grades. In the worst case scenario, students become so frustrated with the process of trying to reenroll in school that they drop out altogether. Using assessments to track student progress and recording those assessment results in student records can help ensure that students get credit for their work and are placed appropriately when they transition out of their current education program.

What are appropriate pre-post tests for the student population?

In some facilities, assessment activities are focused on administering an annual State assessment. However, State assessments are generally not appropriate tools for N or D programs to use to demonstrate the progress of their students for several reasons:

• Scores cannot be used to see individual student gain (e.g., a student takes a fourth-grade test and then a fifth-grade test instead of the fourth-grade test again).
• The State assessment is usually not designed to be given more than once per year; hence, it is not an appropriate pre-post test.
• It cannot be used to track ongoing academic progress of students during their enrollment period.
• Most of the students will have entered and exited the N or D system before the 1-year mark (only about 10 percent of students are in facilities for more than a year) and may not be enrolled during the administration of the statewide test [6].

Furthermore, the characteristics of students served in N or D programs may not be adequately addressed by State assessments (e.g., students who enter below grade level or have special education needs). Facilities need to assess their population and select a test that: (1) is designed as a pre-post test, and (2) best meets the needs of the population they serve. NDTAC’s A Brief Guide to Selecting and Using Pre-Post Assessments contains valuable information on what to consider when selecting a pre-post assessment.

Data collection and reporting: Where does my facility stand?

Data Tables

Excel Version Screenshot from the Excel version PDF Version (printer-friendly)

A series of tables accompany this toolkit for program managers to use when examining pre-post assessment procedures and student academic progress. Completing the tables can provide an opportunity to reflect on which data related to pre-post assessment practices and results are currently available. The tables may also be used to develop a history and record of procedures for new employees and to establish a baseline on which to track improvements and set new goals.

Two versions of these tables are available. The PDF version is available for those who prefer to complete the tables on paper. A downloadable Excel file version provides a more interactive format of the data tables. Each table provides fields for data entry, a place to set goals relevant to each data collection point, and space to document the steps needed to achieve those goals. The Excel tables allow for the entry of data at several points in time, can be used to track facility progress, and also feature graphs generated from the data entered.

Using the Data Collection Tables

Following is a description of the tables and issues to consider around pre-post testing. Consider expanding on and tailoring the questions in this toolkit to better address the needs of your facility.

Table 1: Documenting Process and Procedures
Before completing the data collection tables, take a moment to consider the questions in the Documenting Process and Procedures table. Use this table to identify the tests selected for monitoring student progress in reading and math, document the procedures for how/why these tests were selected, and note the procedures related to administration and tracking data. Are any of the assessments being used for pre-post assessment purposes? If so, were those assessments intended for pre-post use (as indicated by the publisher)? Additional information regarding selection and appropriate administration of pre-post assessments can be found in NDTAC’s A Brief Guide to Selecting and Using Pre-Post Assessments.

Table 2: Pretesting Procedures
Does the facility assess students’ academic ability upon entry? Is there a system in place to ensure that such assessment takes place in a timely manner? Many students arrive at a new placement with incomplete education records, making assessment essential to placing them in an appropriate education program. While there are a number of valid reasons why students might not be given an academic assessment on their first day in a facility (students may be struggling with substance use issues or too traumatized by their new placement to concentrate), every effort should be made to ensure that the majority of students are assessed soon after entry into the facility and that all students are assessed within 30 days of entry.

Table 3: Posttesting Procedures
Do students in the facility complete posttests in math and reading? Are the results of these posttests used to guide student instruction? While it is essential that posttests are administered at least once before the student leaves the facility in order to ensure that the student has updated academic achievement records, many pre-post assessments are designed for multiple posttests, allowing the results to be used to track student progress and guide instructional decisions.

Table 4: Records Procedures
How often are records updated in the facility? What percentage of students leave the facility with current records? Update academic records often, noting assessment scores (when available), grades, and credits earned in each subject. Updating student records on a regular basis ensures that they accurately reflect students’ needs and accomplishments should students be transferred with little or no warning.

Table 5: Student Progress
Take advantage of the information provided by the data collection as an opportunity to reflect on the educational services offered by the facility. Based on changes in students’ achievement in math and reading, what is the facility doing well across all students? Are there any areas in which the facility as a whole appears to be struggling? Are there particular students or particular teachers who might warrant more intensive intervention or assistance?

Something To Think About A number of factors impact student learning beyond what happens in the classroom. While this toolkit is focused primarily on assessment in math and reading, to best understand a student’s situation, we must also take into account all of the confounding factors that interact with his or her ability to perform academically, including mental, emotional, behavioral, and learning disorders. Does student assessment in your facility include the following components? Mental health Physical health Emotional/Behavioral health Substance abuse Special education status [8] Vocational/Life skills Student goals

Beyond data collection and review

As detailed in the tables, program managers should take the time to examine the procedures in place for effectively administering pre-post assessments, the program’s ability to transfer thorough and meaningful student academic information, and the actual results of the students’ assessments. As the program manager, you should use the information to reflect on where your program stands, to set realistic goals, and to identify concrete steps toward achieving them. Some goals may take longer than others and will require the involvement of and commitment from additional staff. You should plan to conduct another assessment in 6 or 12 months, understanding that some areas, such as student academic progress, might require review more frequently than others.

The bottom line is that “assessment data is only as good as the actions that arise from it” [7]. For educators of youth who are N or D, the actions that arise from data collection should be threefold:

1. Student assessment data should be analyzed so that instructional decisions can be made that reflect individual student needs.
2. Student achievement data should be analyzed at the facility level to guide program-level decisions regarding curricula, training, and resources.
3. Student assessment data should be collected and recorded on a regular basis to ensure that student records accurately reflect student progress and achievements.

Additional assistance

If the results of the data collection exercise indicate specific technical assistance or training needs, contact those administrators responsible for N or D education programs at the State level; these individuals may include your State’s Title I, Part D coordinator or the administrators of those agencies responsible for youth who are N or D in your State (e.g., department of juvenile justice or child welfare). Contact information for these individuals, as well as contact information for your State’s NDTAC liaison, can be found at www.neglected-delinquent.org. If you have any questions regarding this toolkit, please contact NDTAC by e-mail at NDTAC@air.org.

 

---

[1] Wolford, B., Purnell, B., & Brooks, C. C. (2000). Educating youth in the juvenile justice system. Richmond, KY: National Juvenile Detention Association.

[2] Hudson River Center for Program Development, Inc. (1995). From incarceration to productive lifestyle, making the transition: An instructional guide for incarcerated youth education. Glenmont, NY: Author.

[3] Rutherford, R. B., Bullis, M., Anderson, C. W., & Griller-Clark, H. M. (2002). Youth with disabilities in the corrections system: Prevalence rates and identification issues. Washington, DC: Center for Effective Collaboration and Practice, American Institutes for Research.

[4] Office of Vocational and Adult Education. (2005). STAR Project Reading Toolkit—Pilot Version. Washington, DC: Author.

[5] Villano, M. (2006, January). Assessing formative assessment. Downloaded from http://www.techlearning.com/index

[6] Snyder, H., & Sickmund, M. (1999). Juvenile offenders and victims: 1999 national report (p. 201). Washington, D.C.: Office of Juvenile Justice and Delinquency Prevention. Data Source: Office of Juvenile Justice and Delinquency Prevention. Census of Juveniles in Residential Placement 1997 [machine-readable data file]. Washington, D.C.: OJJDP, 1998. Available at: https://www.ncjrs.gov/html/ojjdp/nationalreport99/toc.html

[7] Villano, M. (2006, January). Assessing formative assessment. Downloaded from http://www.techlearning.com/index

[8] Facilities should be knowledgeable of the requirements of the Individuals with Disabilities Education Act (IDEA) as they apply to youth who are neglected or delinquent. Additional information on IDEA and students with disabilities can be found at http://www.neglected-delinquent.org/nd/resources/policy_portal.asp#idea

Published April 2006 [modified May 2012].