And this is from the NY Times -- a corporation which has shown great fondness for all things Bloomberg.
August 3, 2009, 6:41 pm
What Do School Tests Measure?
By The Editors
According to a New York Times analysis, New York City students have steadily improved their performance on statewide tests since Mayor Michael Bloomberg took control of the public schools seven years ago. While statewide passing rates on the tests have risen in every grade on English and math tests, New York City’s scores have gone up even more, and across all neighborhoods. The racial achievement gap has been cut in half on some tests.
This is good news for Mayor Bloomberg, who has made standardized testing a linchpin of his administration’s stewardship of the schools. Critics say the results are proof only that it is possible to “teach to the test.” What do the results mean? Are tests a good way to prepare students for future success?
Sandra Stotsky, professor of education reform
Richard D. Kahlenberg, Century Foundation
Bruce Fuller, professor education and public policy
Lance T. Izumi, Pacific Research Institute
Marcelo and Carola Suárez-Orozco, N.Y.U.’s immigration studies program
Marcus Winters, Manhattan Institute
--------------------------------------------------------------------------------
Are These Tests Any Good?
Sandra Stotsky is professor of education reform at the University of Arkansas.
I know of no country that doesn’t test its K-12 students at some point before they graduate from high school, with some countries giving more tests than others and at different grade levels depending on the number of years of compulsory education and levels of schooling available. Tests covering what students were expected to learn (guided by an agreed-upon curriculum) serve a useful purpose — to provide evidence of student effort, of student learning, of what teachers taught, and of what teachers may have failed to teach.
Good tests come in 47 varieties. There are reading-based prompts requiring extensive essays written on the spot, or sets of questions requiring a choice of the best answer from a group of possible answers (both formats can assess conceptual understanding). There are also “product” tests that reflect the application of the knowledge and skills acquired in a particular course of studies. The hysteria about testing per se is unwarranted.
While the number of students ‘passing’ has risen, nothing is happening at the level that should indicate academic success.
More serious questions arise about “teaching to the test.” If the test requires students to do something academically valuable — to demonstrate comprehension of high quality reading passages at an appropriate level of complexity and difficulty for the students’ grade, for example — then, of course, “teaching to the test” is appropriate. That is exactly what we want English or history teachers to do. What is not clear with respect to the New York State tests is the extent to which the English tests actually compel teachers to teach students how to read high quality literature written at an appropriate level of complexity or difficulty for the grade.
Based on a perusal of New York State’s grade 8 reading selections several years ago, I judged that the test was assessing the ability to understand passages more appropriate for grades 4 and 5. And this judgment was independent of where the cut score was set.
Why is this relevant today? The combined scores in New York City show a decline from grade 3 to grade 8 in the percentage of students who are “advanced proficient”: from 17 percent in grade 3, 20 percent in grade 4, 22 percent in grade 5, 16 percent in grade 6, 14 percent in grade 7 and 9 percent in grade 8. These combined scores are heavily influenced by the mathematics scores (which also decline regularly over the grades). If we look at the scores in English separately, we find that the percentage of students who are “advanced proficient: went from 8 percent in grade 3, 5 percent in grade 4, 10 percent in grade 5, 7 percent in grade 6, 4 percent in grade 7, to 3 percent in grade 8. Moreover, the percentages for this level of performance do not appear to have risen at all in the past decade; if anything, they appear to have declined. While the number of students “passing” has risen, nothing is happening at the level that should indicate academic success.
Reading is the crucial subject in the curriculum, affecting all the others, as we know. Teachers should be teaching to a demanding English test, but until we know that they are, one may ask: Should English teachers be teaching to these particular tests?
--------------------------------------------------------------------------------
Beware the Mayor’s Claims
Bruce Fuller is professor of education and public policy at the University of California, Berkeley.
New York City’s schools are getting better, according to multiple barometers, thanks to a variety of gutsy reform efforts. But the exaggerated trumpeting of modest progress — what’s become a yearly ritual in the mayor’s office — undercuts the credibility of Michael Bloomberg and his schools chief, Joel Klein.
Mayor Bloomberg claims that more than two-thirds of the city’s students are now proficient readers. But, according to federal education officials, only 25 percent cleared the proficient-achievement hurdle after taking the National Assessment of Education Progress, a more reliable and secure test in 2007.
When New York lowers standards and the mayor hypes the progress, it’s no surprise that parents and employers remain skeptical over the schools’ true efficacy.
This gap is precisely why few informed analysts still take seriously where state and city officials peg “proficient” student performance. Magically, even more students in Mississippi are “proficient” readers than allegedly high-flying New York City pupils.
The major lesson is that officials in all states — from New York to Mississippi — have succumbed to heavy political pressure to somehow show progress. They lower the proficiency bar, dumb down tests and distribute curricular guides to teachers filled with study questions that mirror state exams.
This is why the Obama administration has nudged 47 states to come around the table to define what a proficient student truly knows. Somehow the mayor’s back-slapping press release failed to mention this test-score inflation that has raged in New York and across the nation ever since No Child Left Behind let state officials define cut-points signaling when a student is proficient or not.
Mayor Bloomberg’s claim of dramatic achievement growth is dubious when placed in historical context. The mayor claims that the share of eighth graders proficient in math has climbed from 29 percent in 2002 to more than 82 percent this past spring — an eye-popping 52-point increase in just seven years, three times that detected by the more reliable federal assessment over the last two decades.
The city’s youngsters are certainly acquiring basic literacy skills more effectively now than a decade ago. But when Albany lowers standards and the mayor hypes the progress, it’s no surprise that parents and employers remain skeptical over the schools’ true efficacy.
What’s key in moving forward is to depoliticize student testing and hold public officials accountable when they grossly overstate progress. Given Mayor Bloomberg’s faith in education markets — forcefully backing charter schools and competition among reform groups that help lift the schools — he should know that parental choice works only when families have sound and reliable information about school quality.
The mayor’s self-congratulatory interpretation of student progress prompts a feeling of disbelief, not one of confidence.
--------------------------------------------------------------------------------
The Problem for Low-Income Students
Richard D. Kahlenberg is a senior fellow at the Century Foundation and author of “All Together Now: Creating Middle-Class Schools through Public School Choice,” and “Tough Liberal: Albert Shanker and the Battles Over Schools, Unions, Race and Democracy”.
Test score gains among New York City students are important because research finds that how well one performs on cognitive tests matters more to one’s life chances than ever before. Mastery of reading and math, in particular, are significant because they provide the gateway to higher learning and critical thinking. But test score results can also be easily overblown and obscure significant disadvantages still faced by children in New York City’s high poverty schools.
Whatever the score, children in high poverty are still cut off from networks of students, and students’ parents, who can ease access to employment.
Consider, for example, the over-the-top coverage provided to gains in New York state exams by students at an overwhelmingly low-income school that is part of the Harlem Children’s Zone. According to The Times columnist, David Brooks, by eighth grade, in math, the Zone’s middle-school, the Promise Academy, “eliminated the achievement gap between its black students and the city average for white students.” The “approach works,” Mr. Brooks wrote, implying that separate schools for rich and poor and black and white can, in fact, be equal after all. But this conclusion raises two problems, which illustrate the limitations of test score results.
First, just because students are trained to do well on a particular test doesn’t mean they’ve mastered certain skills. As the Columbia University professor Aaron Pallas pointed out, on a different assessment — the Iowa Test of Basic Skills — eighth-grade Promise Academy students scored at the 33rd percentile for a national sample in math. This is important, Mr. Pallas notes, because if the New York State test score gains are real, and not just the result of test prep, the success should transfer to other tests.
Second, whatever the test score results, children in high poverty schools like the Promise Academy are still cut off from networks of students, and students’ parents, who can ease access to employment. This is important given research finding that more than half of jobs are filled through connections.
By all means, let’s celebrate test score gains in New York City, and the narrowing of the achievement gap. But hold the champagne until we can show more.
--------------------------------------------------------------------------------
What Tests Can and Should Do
Lance T. Izumi is the senior director of education studies at the Pacific Research Institute.
In the wake of widespread state testing following the enactment of the No Child Left Behind Act, critics have claimed that teachers are simply teaching to the tests. Yet this argument is overly simplistic and ignores the benefits that come from good tests.
Opponents of testing try to have it both ways. When test scores are low they argue for a holistic view of student achievement that focuses on non-test indicators of performance rather than teaching to the test. When results are high, as in the recent rise in New York City scores, they counter that the scores are suspect because teachers are just teaching to the test. If a state test is well conceived, both these arguments fail to hold water.
If tests are reliably aligned with rigorous state academic content standards, then teachers are right to teach to the test.
Susan Philips, a professor of education at Michigan State University and one of the nation’s leading testing experts, has testified that well-developed standardized multiple-choice tests give more individual examples of student knowledge and skills, are more consistent in scoring, are capable of measuring higher-order thinking and are fairer than other non-standardized assessments. Since standardized testing can accurately assess the “whole” student, low test scores can be a real indicator of student knowledge and deficiencies.
If tests are reliably aligned with rigorous state academic content standards, then teachers who teach to the standards are teaching to the test, and there is nothing wrong with that. E.D. Hirsch, author and University of Virginia education professor, notes that “grade-by-grade standards and some form of fair grade-by-grade tests are logically necessary for monitoring and attaining grade-by-grade readiness.” Many teachers at high-performing, high-poverty schools have said they use student test scores as diagnostic tools to address student weaknesses and raise achievement.
While inappropriate use of test materials should not be countenanced, a valid standardized test linked to tough standards is a critical tool for measuring and improving student performance. Assuming New York has such a test, when Joel Klein says that if test prep means “teaching people to read and understand paragraphs, that’s what I think education is about,” then he’s right.
--------------------------------------------------------------------------------
Ignoring the Needs of English-Language Learners
Marcelo Suárez-Orozco and Carola Suárez-Orozco are the co-directors of the immigration studies program at New York University and the co-authors of “Learning A New Land: Immigrant Students in American Society.”
This article overlooks the elephant in the American classroom: the educational progress of English-language learners, the fastest-growing group of students in American schools. This neglect may not be surprising given that the ever more Byzantine academic testing architecture has never been designed with the educational needs of this population in mind.
This test regime has huge implications for dropout rates and access for immigrant children.
The current high-stakes testing and accountability systems create unintended consequences for immigrant English-language learners, which outweigh whatever benefits standardized tests may have. Because too many immigrant students attend highly segregated and impoverished schools, are not exposed to quality curricula and undergo multiple school and programmatic transitions, their performance on such tests is often compromised. Is it any surprise then that in the “gold standard” (not what The Times calls “the dumbed down” New York State Test) National Assessment of Educational Progress reading assessment for 2007, 71 percent of English-language learners in the eighth grade scored “below basic” in reading and zero percent scored at the “advanced” level?
The high-stakes testing context is proving to be extremely challenging to newcomers. Not only are many immigrant children tested before their academic language skills have adequately developed, but all too often their day-to-day educational experiences are shaped by instruction that teaches to the test, which is far from an adequate measure of what it takes to succeed in the complex and challenging economies and societies of the 21st century.
This eye on the omnipresent “adequate yearly progress” is more often than not at the expense of more engaging, broader academic knowledge. What’s more, this test regime has huge implications for dropout rates as well as college access for children of immigrant families.
--------------------------------------------------------------------------------
Making the Best of a Flawed System
Marcus Winters is a senior fellow at the Manhattan Institute, where he has done several studies on high-stakes testing, school report cards and the effects of vouchers on the public school system.
One of the pitfalls of standardized tests, perhaps the most important accountability-focused reform, is their elevation of scores over genuine learning. In high-stakes testing jurisdictions, anxious teachers, in order to avoid earning bad grades themselves, “teach to the test,” as the saying goes. In doing so, fortunately, some teachers adopt classroom techniques that produce real increases in student proficiency, particularly among the lowest-performing students. Tough proctoring rules can deter less well-motivated teachers from raising test scores in more underhanded ways.
The bigger problem with standardized tests is their emphasis on the achievement of only minimal proficiency.
The bigger problem with standardized tests is their emphasis on the achievement of only minimal proficiency. In most programs, the proficiency benchmarks that students must pass are levels of literacy and numeracy so low that only the most academically troubled students will find themselves better prepared for the outside world. High-achieving students, by contrast, will have already far exceeded them. While it is imperative that even the least accomplished students have sufficient reading and calculating skills to become self-supporting, these are nonetheless the students with, overall, the fewest opportunities in the working world. Meanwhile, limited resources are relocated away from the most promising students. If the premise of our educational system is that all students must be able to crawl before we help others to run, then such a policy is a worthy one.
Regardless of how high or low we choose to set the proficiency bar, standardized test scores are the most objective and best way of measuring it. Still, they are flawed. On a multiple choice exam, a child can demonstrate whether he can read and grasp the gist of a piece of writing, but he cannot usually demonstrate the depth or thoroughness with which he comprehends it. The gap between proficiency and true comprehension would be especially wide in the case of the brightest students. These would be the ones least well-served by high-stakes testing.
