Jeanne Marie Olson was on WTTW’s Chicago Tonight on May 7 to talk about problems with the ways that CPS is calculating utilization and other factors used in making closing decisions. Watch the full interview.
Value-added scores alone would not have made a school higher-performing. If you look at total weighting, in our best three of four measures it’s much further weighted toward ISAT measures than value added.
—CPS’ Officer of Portfolio Planning and Strategy, Adam Anderson, quoted in a recent Chicago Tribune article.
CPS categorizes welcoming schools as “higher-performing” based on one of two sets of criteria:
1.The welcoming school must be at a higher Performance Policy Level than its respective closing school (with Level 1 being the highest, based on the total number of Performance Policy points). Value-added scores account for 15% of a school’s total Performance Policy points (a maximum of six out of a possible 42 points).
— or —
2. If two schools are at the same level, one is deemed “higher performing” if it is better on three of the following four criteria: a) Percentage of points on the CPS Performance Policy, b) Percentage of students meeting or exceeding the ISAT composite, c) Value added reading score, and d) Value added math score. By this criteria, a Level 3 school can be called “higher performing” than another Level 3 school, as if to say it’s a “better bad” school.
When Adam Anderson states that “value-added scores alone would not have made a school higher-performing,” he was probably talking about the impact of value-added on determining a school’s Level (1, 2, or 3). (More information on CPS’ Performance Policy.)
However, he adds “if you look at total weighting, in our best three of four measures it’s much further weighted toward ISAT measures than value-added.”
Here he is mistaken, because value-added accounts for two of the four measures he references and is itself based on ISAT scores. CPS explains value-added as the district measure of growth on the Illinois Standards Achievement Test (ISAT).
The second set of criteria has been used to justify closing 24 schools. In the case of five of the schools (Altgeld, Delano, Goodlow, Manierre, Mayo) the receiving school met the “three out of four criteria” by including both value-added measures, reading and math. In the 19 other cases the receiving school was higher on at least one of the value-added measures.
In moving from CPS’ original criteria of higher performing (e.g. Level 1, 2, or 3) to their redefined “better bad” criteria (e.g. higher on three of four metrics a, b, c, and d above) CPS essentially reweighted the impact of value-added in 19 cases from 15% to 33% (a, b, with c or d) and in five cases, to 66% (a or b, with c and d).
“The very day that CPS announced the school closing list, that evening a group of software coders put up the site Schoolcuts.org,” says Terry Mazany, President and CEO of the Chicago Community Trust, who had also served as interim CEO of the Chicago Public Schools in 2010.
“They had aggregated all the datasets about school performance. And geomapped the schools that are on the list for closing …. That’s the sort of service you would hope that government might provide but these groups did it out of a sense of community service,” he says.
“They had this site up and running — and it is masterful.”
Excerpted from the Trib’s story, School closings: A closer look at CPS strategy, Elnaz is quoted in the fourth paragraph shown below (italics), commenting on the use of Value Added scores in determining which schools to close.
– – – – – –
At the 25 schools whose students are being moved to a school at the same performance level, CPS considered additional criteria.
Adam Anderson, CPS’ officer of portfolio, planning and strategy, said the district looked for the receiving school to outscore the closing school on at least three of four measures: a higher percentage score within the performance rating; composite meets or exceeds score on the ISAT; an improvement metric for reading; and another one for math.
With two of the four measures dealing directly with improvement, schools with solid scores that dipped slightly in some cases fared worse than poorly performing schools that could show improvement.
Elnaz Moshfeghian of Open Data Institute, which helps produce the “Schoolcuts” blog to study the data surrounding CPS’ school closings, said a closer look at that controversial improvement metric shows it changes for schools year to year.
“It’s been called a noisy number,” Moshfeghian said. “It is not a reliable and stable school metric and should not be half the reason why one school stays open and the other closes. You wonder why it’s being used at all.”
Anderson said the improvement, or value-added, score did not carry significant weight as the district decided which schools to close.
“Value-added scores alone would not have made a school higher-performing,” he said. “If you look at total weighting, in our best three of four measure it’s much further weighted toward ISAT measures than value added.”
– – – – – –
Full story at http://trib.in/114lsUl
For every school designated for closure, CPS has identified one or more receiving schools that will welcome students currently enrolled at the closing school. However, what happens to future students and children not currently enrolled who would have otherwise attended the school that might close?
From CPS’s Transition Plan for Morgan:
As a result of this action, all returning Morgan students will be welcomed at Ryder, located at 8716 S. Wallace St.
It would be easy to assume that a receiving school is automatically the new neighborhood school. If my address is currently within Morgan’s attendance boundary, I see that Ryder is the receiving school for Morgan. Will my 2 year old or my future children attend Ryder?
Maybe, but maybe not.
When is a Receiving/Welcoming school not your neighborhood school?
On our SchoolCuts page for Morgan you can see the proposed boundary for Ryder. To the North the boundary ends at 83rd St. If you look at the current Morgan boundary you see that it extends farther north to 80th St and also goes several blocks west of Halsted.
These areas north of 83rd St will NOW fall into either Wescott or Gresham’s attendance boundaries. For people living in these areas their new neighborhood school will be Wescott or Gresham.
In these cases where the receiving schools’ new attendance boundary does not overlap entirely with the closing school’s current attendance boundary, what happens for future students and children not currently enrolled at the closing school?
Again from CPS’s Transition Plan for Morgan:
The geographic boundary currently associated with Morgan will be reassigned to Ryder, Gresham, located at 8524 S. Green St., and Oliver S Westcott (Westcott), located at 409 W. 80th St. This means that Ryder, Gresham or Westcott, depending on the student’s home address, will be the new neighborhood school for students living in the Morgan boundary who are not currently enrolled at Morgan.
For someone living at 8110 S Green St the current neighborhood school is Morgan BUT the future neighborhood school is Gresham. So a 2nd grade student living on this block will be welcomed at Ryder next year but his or her 1 year old sibling or neighbor would attend Gresham in the future based upon these neighborhood attendance boundaries.
What should you know?
It is important to remember that the designation of “receiving” appears to be only applicable for currently enrolled students. Look at section “II. Summary of Action” of the CPS Transition Plan for your school to see which if any additional schools will become new neighborhood schools in that attendance area.
Here at SchoolCuts, we are working to identify these situations and to make it easier to determine your future neighborhood schools.
The following is a list of schools we’ve identified so far. This is an incomplete list:
Shortly after CPS released proposed new attendance boundaries for receiving schools, we added them to our school profile pages. See the Cather school for an example.
All of the data that powers SchoolCuts.org comes from the websites of the Chicago Public Schools, the Illinois State Board of Education, or the University of Chicago Urban Education Institute. It’s often challenging to find education data. When you do, it can be in easy-to-use formats such as spreadsheets or must be input manually from PDFs or images.
It took the SchoolCuts team many hours pull the data together; to spare others the work, we are posting it in a Google spreadsheet at goo.gl/LSJLx.
CPS Guidelines For School Actions reads: “The CEO may only propose a [school] closure…if the students impacted…have the option to enroll in a higher performing school.” The Commission on School Utilization says, “The goal must be to enroll all displaced students into higher-performing schools.”
Looking at the list of closing and receiving schools—and the associated data—the question naturally arises, “Is CPS living up to its own standard?” The answer, unfortunately, is “Not always,” or more troubling, “No one really knows.”
For many years, CPS has classified schools as performing at levels 1 (best), 2, or 3 (worst). However, when you examine levels at closing and receiving schools, you find that in 25 cases, students are slated to be sent to a school that is not rated at a higher level.
CPS seems to have temporarily redefined “higher performing” to mean:
. . .higher on the majority [three out of four] of the following metrics for the 2011–2012 school year: percentage of points on the Performance Policy, ISAT composite meets or exceeds score, Value Added reading, and Value Added math.
CPS has created a new category of “better” bad that allows them to claim that all displaced students will be sent to better schools. Except, this “better” bad standard is not met in the case of closing Owens, with students sent to Gompers.
Using Value Added scores for this purpose, however, is problematic. The mechanics of calculating the scores are complex and education experts agree that these scores should not be used in this way.
Making matters worse, when you apply statistical analysis of validity to the underlying CPS metrics, they are shockingly unstable. From 2011 to 2012, 77 schools flipped from being among the top quarter of schools in Value Added Reading to the lowest quarter or vice versa.
How can CPS justify closing a school because it had a very low Value Added score in the most recent year even though it had a very high score in the previous year? Using their methods, might that school be just as likely to have a very high score next year? Statistically, this extremely “noisy” number is a difference without distinction and certainly no marker of “better.”
In recent weeks, debates have centered on utilization rates, savings, safety, neighborhood disruption, and racial discrimination. Statistical evidence indicates that—at the very least—many students will not be moving to a higher performing school.
She is interviewed about CPS school closings and the data about the closings on schoolcuts.org
Elnaz is interviewed about SchoolCuts.org data about the CPS school closings.