Can we learn any more from tests?
Wednesday, August 13, 2008Written by: Todd Engdahl
How much testing do we need?
Now before you accountability absolutists start hyperventilating, rest assured that I’m not one of those bleeding hearts who wants to abolish CSAPs because they traumatize sensitive 4th graders with iffy math skills, cost millions or tempt jumpy principals to flirt with bribery.
What I wonder about, though, is whether we’re learning anything new from standardized tests.
August is a time to look forward, with school starting, but it’s also the month to look back, because it’s test results season.
The 2008 CSAP results were released a couple of weeks ago, the national ACT results came out Wednesday and the annual SAT report is coming soon.
Because we Americans are addicted to stats and scores (just listen to those announcers at the Olympics), every release of test results is parsed exhaustively. Gains of a few tenths of a percent at the local elementary school are hailed as encouraging progress, and comparisons are eagerly made between districts, and between a district’s scores and state averages.
None of that may mean much at all. (Sorry DPS.)
Over time, if you look at CSAP percentages of advanced or proficient or at ACT average composites, it’s striking how little change there is. The numbers fluctuate within single digits over several years.
When you dig deeper into the scores, you see the same patterns you saw the year before and the year before that. Among them:
-
Minority kids don’t do as well as white kids.
Poverty is a factor.
There are differences between boys and girls.
Overall performance declines as kids get older.
There’s an astonishingly low level of math and science proficiency.
High schoolers who take a more rigorous class schedule do better on the ACT.
Most kids aren’t ready for college work.
No, we shouldn’t do away with standardized tests. The CDE’s growth model offers some potentially useful new ways to look at the annual avalanche of CSAP stats. (Although, growth stats also may just tell us what we already know. As CDE assessment whiz Rich Wenning said at the CSAP news conference on July 29, “Our lowest achieving students are not making enough growth to catch up.” He repeated that comment to the State Board of Education on Aug. 13.)
The real questions are what can be done to help them catch up, and how many of them can realistically be expected to catch up?
August 14th, 2008 at 8:31 am
Todd makes an excellent point, and hopefully helps us move behind the simplistic “CSAPs good vs. CSAPs bad” discussion.
The aggregated CSAPs are not going to tell us a great deal, because they move so slowly, if at all. And, yes longitudinal, value-added tests are giong to be more helpful.
I think we need to start viewing CSAP as a “medium stakes” sort of test, and perhaps fade it out slowly in favor of better tests. We need more “lower stakes” tests within schools, with shorter feedback loops, so that teachers and instructional leaders can see hwo students are doing, and very quickly adderss their problems. Some districts already do this well, but not a large number of them. With today’s technology, this is much more feasible than ever before.
On the other end, a couple of real high stakes tests, like ACT/SAT, probably provide better general assessment information, especially for individuals, than CSAPs, in part because the students have higher incentives to do well.