Test scores and expectations
Tuesday, May 6, 2008Written by: Pol Econ Ed
The reality of yet another year of basically flat third grade CSAPs scores, after more attempts at reforms, generated a lot of heat on this blog, with everyone trying to figure out what it means for future reforms.
But, I continue to be frustrated by the ad hoc nature of the ex poste “spin” around test score results. How about a discussion of what we would realistically hope to see from CSAPs, or other standardized tests.
Before the full results come out this summer, what would a positive result look like for Colorado? We know that a decline in test scores would be considered quite bad, though for districts or schools undergoing major reform it is usually optimistically called an “implementation dip.” Flat is not very impressive.
But, how positive should positive be? NCLB wants no child left behind and all students up to proficiency by 2014. For third grade reading statewide, with 71% proficient, that would require increases in the proficiency numbers of about 5% points each year (e.g., 29 points over 6 years). That is our current national goal. Anyone expect that to happen in Colorado?
Statewide improvement of more than 5 points would be looked upon with skepticism, about cheating, extreme teaching to test, or actual errors. Statistically, with such large numbers, such huge improvement seems highly unlikely.
Obviously, any positive movement is better than flat. But a point or two, would that be worth celebrating, with all of the efforts underway, plus the very real likelihood that schools are spending a lot more time teaching at least towards the test, if lots of anecdotes and stories are to be believed?
I think that an appropriate but ambitious goal is to aim for annual increases in the 3-5 percent range. Over five years, that would lead to cumulative 15-25% growth, which would be huge if the gains could be sustained.
Of course, “value-added” testing makes much more sense, actually measuring what the same set of students have learned from one point in the year to a later time – because the actual kids are held “constant.” Without value added tests, we don’t know much. It is at least possible that Colorado schools really did do a better job with third grade reading in 2007-8 than last year, but this year might have featured a 3rd grade school population with more at-risk students than last year, leading to flat aggregate scores.
What is your over/under for the full CSAP results to be released this summer?
May 6th, 2008 at 8:20 am
An excellent post. I did a similar analysis a few weeks ago which calculated that DPS needs an improvement in proficiency of about 2.5% points each year, for 10 straight years, just to get to the State average - and this is assuming that the State (outside Denver) does not improve at all. Since DPS has never had that increase in a single year, it’s a safe bet they won’t do it 10 years in a row. So without extraordinary change its safe to say DPS will remain well behind Colorado for the near future.
However, within any average there is a large variance, and I’m really hoping that the School Performance Framework will give DPS both the data and the political coverage to grow and replicate the schools that are beating this mark, and close ones who are not. This won’t solve the problem, but over the longer term it would help bridge the gap.
Of course, it’s easier to get the increase when you are near the bottom than the top - to go from 30% proficiency to 32.5% than from 70% to 72.5%
Over under for this year? Great question. Last year the increase was 0.6% (all grades tested, in reading, writing and math). At this end of the scale I think anything less than twice that - say 1.25% - should be considered failure if we are serious about reform. But if we were using real money, I would only take the over on 1%.