---

February 18, 2011  (NY Times)

City to Toughen Auditing of School Test Scores

By SHARON OTTERMAN

New York City school officials said Friday that they would introduce a new, rigorous system of auditing the test scores, grading practices and graduation rates of the public high schools, appearing to acknowledge rising concerns that some schools might be manipulating the statistics they are judged by.
The move comes as the city and the state have sought to raise standards to better prepare students for college and careers, and as mounting evidence has cast doubt on whether even the current standards are being met.
In at least the past two years, an unusually large number of students have obtained exactly the minimum score needed to pass state Regents exams, which are often graded by their regular teachers. City officials say the anomaly existed even before Mayor Michael R. Bloomberg took control of the city’s schools in 2003.
In an e-mail sent Friday to high school principals, Shael Polakow-Suransky, the department’s chief academic officer, said that auditors would look at how schools awarded course credits, graded Regents exams and tallied graduation figures in determining which schools to audit.
In a departure, the auditor general of the department, Brian Fleischer, is to oversee the new audits, to ensure greater independence. Previous audits were conducted case by case by the same office that develops the accountability practices.
“Ultimately, we want to have confidence, for ourselves, and for the public, in the data we use to measure schools,” Mr. Fleischer said. The new audit procedures, he said, “will be much more data-driven and systematic.”
About 60 high schools will be selected for the first round of audits, based on whether their data showed suspicious patterns, like sudden rises in scores, he said. Allegations of misconduct would be referred to the special commissioner of investigation for city schools. In the first year, however, the emphasis will be on providing guidance and training to schools so that employees understand what is expected.
Despite the increased scrutiny, Mr. Polakow-Suransky has said he does not believe there is widespread cheating. He said last week that the city aggressively investigated the “tiny handful of cases” where there were allegations. On the question of Regents scoring, a process determined by the state, he said, “We feel an obligation to work on this issue, despite the fact that they are not our tests.”
But observers of the school system, including those who have been skeptical of rising test scores and graduation rates, said officials seemed to acknowledge issues with some of the data.
“It seems to me that the D.O.E. is realizing that they have a credibility problem with their numbers and they’re trying to address that,” said Kim Sweet, executive director of Advocates for Children of New York, which has questioned whether some schools tally dropouts incorrectly to help graduation rates.
Officials said that they started working on toughening auditing procedures more than a year ago, but that in recent months they approached the issue with greater urgency.
The state comptroller recently completed a review of the city’s graduation rates, which has not yet been released to the public, though it has been submitted to city officials for their response.
The city had already begun investigating grading at the school that had the highest score on the department’s annual report cards, the Theatre Arts Production Company High School in the Bronx. Its teacher handbook indicated that failing grades were only for students who never went to class.
And an analysis by The New York Times found that on the English and history Regents exams in the past two years, students in the city’s public high schools were roughly five times as likely to score 65, the passing grade, or slightly above it than to score just below it.
Statisticians say that such a difference is out of line with the smooth scoring curve that should normally result. A recent report in The Wall Street Journal came to a similar conclusion.
Even on the algebra exam, where there are no essays, 8,451 students got grades of exactly 65, while only 7,145 students combined ended up with a score of 61, 62, 63 or 64.
Regents exams are graded by teachers within schools, and teachers are not barred from grading their own students. While the practice is controversial, some say it is appropriate to give students the benefit of the doubt.
There is ambiguity in grading essays, and even in mathematics tests, in which extra points can be given for students’ showing their work.
At one Queens high school, the number of students scoring 65 to 69 last year in the five most popular Regents exams — integrated algebra, global history, biology, English and United States history — was more than five times the number who scored 60 to 64.
“If you have a kid with a 64, you want to look at the paper again to give the kid an even chance,” said the school’s principal, who spoke only on condition that her name, and the name of her school, not be published. “We’re not talking about changing the grade where the kid got it wrong to make it right.”
David M. Steiner, the state education commissioner, acknowledged in an interview in January that the state had known for years of the spikes in scoring patterns.
The department has made some changes, like required training for teacher-scorers, and it is phasing in computer scoring for multiple-choice questions.
But of the practice of teachers’ grading their students’ exams, Dr. Steiner said, “obviously, it’s not ideal.”
The state, he said, was focused on building the next generation of exams, to come into use in three to five years, which may be completely graded by computer.

Amanda Cox, Robert Gebeloff and Fernanda Santos contributed reporting.