From the editor

Recently released Colorado Student Assessment Program (CSAP) results offered us another dose of grim news regarding efforts to improve public education in Colorado. You can slice and dice the results any way you wish. The bottom line is that scores remain essentially flat, as they have done since CSAP was launched over a decade ago.

We can argue all day about whether using one test as an all-powerful measure of school quality is fair, or even accurate. But that's almost entirely beside the point. In any case, as Colorado develops longitudinal measurement -- tracking cohorts of kids from year to year instead of comparing one year's fourth-grade class to another's -- the quality of data promises to improve.

It's wise not to get distracted by complaints about test-obsessed accountability. Like it or not, the CSAP does a decent job of telling us what kind of job our schools are doing preparing kids in a given grade for success in higher grades.

At some point, and soon, we all need to ask ourselves whether the way we are  doing school can ever work to prepare kids for the 21^st century. This question is especially pressing for urban school districts, where large percentages of kids continue to test far below proficiency, with poor prospects for ever catching up.

As Blogfather wrote on the HeadFirst Schools for Tomorrow blog:

How much longer do we have to study state test results before we all acknowledge that what we're doing simply isn't working? Our annual breathless anticipation reminds me of the joke about the eternally optimistic child, whose parents decide that he needs a dose of reality. So for Christmas, they give him a steaming pile of horse manure. The kid unwraps the pungent package and exclaims: Oh, goody! I got a horse! I just haven't found him yet! Folks, I'm beginning to think that when it comes to ed reform, there is no horse. Thinking radically increasingly seems the sensible course.

We can take some solace from the sterling results produced by the Denver School of Science and Technology, KIPP Sunshine Peak Academy and West Denver Preparatory. But all three are charters, which underlines the pressing question to which we keep returning: can school districts as currently constructed close achievement gaps and prepare all kids for the post-secondary world -- be it college or career?

I'm increasingly convinced that the answer to that question is a resounding NO. What do you think?

-- Alan Gottlieb

Blog highlights

CSAP results and the governor's P-20 Council dominated the Schools for Tomorrow blog over the past couple of weeks. Here are some highlights:

CSAP Scores Generate…

Written by: Uncle Charley

Reports from the Rocky Mountain News and Denver Post on the newly released CSAP scores are not very encouraging. Yes, testing in science and most middle school scores demonstrated improvement. But frustrations over lack of success in reading likely will overshadow the positive developments in upcoming debates and discussions.

Give credit to incoming commissioner Dwight Jones for at least saying the right things:

The fewest gains occurred in reading, considered by educators to be the key to success in all other subjects. Scores were flat and even down in some cases since 2003.

"That has been and continues to be one of our most disappointing areas," said education commissioner Dwight Jones.

"We've invested a lot of money, a lot of training, a lot of support … around reading, yet the results continue to elude us," Jones said. "(We're) still trying to figure out why that is happening, because I know it's not because of lack of effort."

Jones acknowledges that large investments of resources have taken place to bolster reading results. It seems like the call should be for revamping some reading curricula or teacher training programs according to the research-based evidence of what works. Nope. Instead, predictably tired responses come from a predictable source:

University of Colorado education professor Kevin Welner, who follows testing issues, said scores are flat in many other states.

Schools have learned to use test data to identify students who need extra help; they have a good idea of what instruction to provide. But they can't provide help without additional resources, especially when the most needy kids require expensive small-group or even one-on-one attention, Welner said.

"Kids don't learn more and schools don't necessarily do better without providing resources to do that," he said. "Simply telling schools to do better " which is what No Child Left Behind does " doesn't accomplish a heck of a lot."

There sure are problems with No Child Left Behind, but making it a punching bag for stagnant CSAP scores doesn't get us very far. Especially when you ignore the fact that federal K-12 education funding has grown 41 percent since 2001. And a 2006 education study shows that the state's reading teachers could be a lot better prepared to teach the vitals of phonics and other key elements to create successful readers.

Let's get on the right track first, then figure out how much it will cost.

6 Responses to CSAP Scores Generate…”

1. edmove Says:
   July 31st, 2007 at 4:21 pm e

Funny…you'd think that test scores would go up given that that the curriculum has been narrowed and all teachers are spending most of their school days teaching CSAP if you listen to some of classic liberal testing critics. Testing needs to be improved but it is hardly the problem here. I guess I see this as further evidence that the public education system needs some fairly fundamental change for something like 50% of the kids in the system. Yes, it will cost something to fix the system though I'm not sure the operations costs need to be higher. Only about 4% of Latino kids that score unsatisfactory at the third grade ever get to proficiency (only 6% get into proficiency for your average CO kid). Clearly the current system does not add value for many kids.

2. Kevin Welner Says:
   August 1st, 2007 at 1:01 pm e

Uncle Charley has tossed out some interesting accusations and assertions here. Allow me to offer some responses.

1. Although the reading efforts accompanying NCLB have probably had a positive overall effect (this is very difficult to isolate and measure), I do understand the frustrations expressed by the new Ed Commissioner. Where Uncle seemingly goes astray is with the following: It seems like the call should be for revamping some reading curricula or teacher training programs according to the research-based evidence of what works.”

In fact, the Reading First initiative funded under NCLB reflects President Bush's demand that reading instruction be firmly grounded in phonemic awareness (what Bush calls scientifically based reading research”). Although many experts in the field do dispute this approach, I'm going to go out on a limb and guess that Uncle is not among them. Judging from his other comments here, it seems to me that he'd support the increased teaching of phonics. Is Uncle calling for revamping of the phonics curriculum because he's dissatisfied with the current results? Or is he blaming schools for inadequately implementing the president's plan? Any evidence of such implementation failures? Is he willing to consider the possibility that the current approach might itself be flawed and that students might learn more if teachers are freed up to use less rigid approaches to teach reading?

More importantly from my perspective, are we willing to acknowledge the evidence that many of our most disadvantaged students are currently given inadequate resources to excel in reading, no matter which method of teaching is used?

2. Uncle then goes on to address what he calls predictably tired responses come from a predictable source (me). Setting aside the ad hominem silliness, his point is that federal education spending has increased a total of 41% over five years. A full rejoinder here would be very long indeed, addressing issues like historical spending increases, shifting burdens from federal gov't to state gov't, additional burdens placed on states and local gov'ts (not the least of which is the cost of testing itself), and population increases. States as different as Connecticut and Utah have balked at the NCLB mandates, inexplicably ungrateful for the increased federal largess. Here's a piece I wrote recently called, Money Mutterers (http://www.aasa.org/publications/saissuedetail.cfm?ItemNumber=9070).

I agree with the point made in the other comment (edmove) about some testing benefits being likely to show up if only because of narrowed curriculum. I made a similar point to Berny Morson, the Rocky reporter, but I also explained that these sorts of gains are the low-hanging fruit that was picked early on " those are the gains we saw under CSAP even before NCLB came along. Maintaining the narrowed curriculum might maintain those earlier test score increases, but they can do little to add to those increases.

Yet test scores can still improve. Kids can learn much more, but teaching and learning require resources. That's the inconvenient truth that Uncle and others here don't seem to want to accept. Yes, we should be willing to experiment with ways to get more bang from the education buck, but blindly grasping at straws (vouchers likely being the one that Uncle likes best) is no substitute for investment in resources that we know to work.

3. Uncle Charley Says:
   August 1st, 2007 at 3:14 pm e

Kevin,

Thanks for your long response. I'm glad to stimulate discussion and response. Here are my answers in brief to your two points:

1. First, I must clarify that I am no fan of federally-imposed standards, and am not a whole-cloth apologist for NCLB.

That being said, can you point to any research on the relative success of implementing Reading First in schools? It appears that too many of the school's practitioners aren't doing it right. I have seen a report, mentioned above, that shows Colorado reading teachers woefully underprepared, and another report highlighting the fact that schools of education are not training their teachers how to teach phonemic awareness, etc.

Yes, the research does show phonics works. Who are the experts who disagree with this assessment, and what is their reasoning? When was the last time the market showed parental demand for Hooked on Whole Language? I don't believe in a one-size-fits-all approach to teaching reading, but those teaching reading all need a basic grasp of certain fundamentals.

2. Of course, teaching and learning require resources. Catchy phrases aside, you shouldn't waste your time beating down a strawman. But simply placing faith in throwing good resources after bad doesn't help those who are supposed to be served by our education system.

Equating vouchers - which are not the panacea to education, but a badly needed reform that opens up the education market to consumer demand - with blindly grasping at straws merely reveals your own bias, and ignores the promising research behind them.

My question for you: Can you give me one recent concrete example of a way to get more bang from the education buck, and one concrete example of a way you would propose to get more bang from the education buck”? Thanks.

4. Kevin Welner Says:
   August 1st, 2007 at 4:38 pm e

U.C. " I want to start by pointing out that I'm not an expert on teaching reading. I have colleagues who are, but our areas of expertise here tend not to overlap much.

So I'll just point to what I understand to be the definitive report on teaching reading. It was issued in 1998 by the National Research Council. Called, Preventing Reading Difficulties in Young Children, the report reflects a consensus among the NRC committee of nationally recognized professors and educators, chaired by Harvard's Catherine Snow, that no single reading method used in isolation works best for all children. Teachers should instead use a mix of phonics and language experiences (often called whole language”) to support the development of diverse learning approaches. You can find the report at http://books.nap.edu/openbook.php?isbn=030906418X

Regarding a concrete example of getting more bang for the buck, I can point to a very successful reform I've been studying in Long Island, which you can read a bit about here:
http://www.pdkintl.org/kappan/k_v86/k0504bur.htm

But more to the point, consider this What Works Clearinghouse report on Reading Recovery, which is an intense (and resource intensive) intervention to help struggling readers. http://www.whatworks.ed.gov/InterventionReportLinks.asp?iid=209&tid=01
fyi, the Clearinghouse has been sharply criticized for being too negative/skeptical about the evidence supporting interventions, so the positive report about Reading Recovery seems particularly noteworthy.

My question for you: where is this promising research behind [vouchers]”? I try to keep up with the research, but I find little or nothing there to support vouchers' effectiveness. A few years back, there was a brief, optimistic blip on the radar " a study trumpeted by a voucher advocate name Paul Peterson. But Peterson's own research colleagues quickly distanced themselves from his characterization of the research (concerning a privately funded NYC voucher program) and a Princeton research team then reanalyzed the data and found the positive findings to be non-robust (that is, they disappeared when the data was analyzed from other angles).

5. Uncle Charley Says:
   August 2nd, 2007 at 2:34 pm e

Funny that we have two secondhand, non-experts debating the science of reading instruction. My expert contacts tell me the thorough and highly credible 2003 National Reading Panel report is the most definitive: http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/16/61/b8.pdf - since this synthesized previous work and came out after 1998, I don't know why your contacts disregard it. (And you never answered my point about so many licensed teachers in Colorado & elsewhere not being prepared to teach the elements from the National Reading Panel.)

I have nothing at all against Reading Recovery or other effective interventions, though many students in need of intervention would have benefited from learning correctly the first time.

Thank you for answering some of my questions, though. I read the Long Island article, and would probably largely agree with your recommendations. But where is the extra bang for the buck in this proposal?

Competition through broad-based choice and competition (i.e., vouchers & tax credits) lead to better bang for the buck. Smaller-scale experiments with such programs in the U.S. tell us just that (and more): here's a sample of some studies:

- Voucher students improve math scores
http://ideas.repec.org/a/bes/jnlasa/v98y2003p299-323.html
http://www.nber.org/digest/nov97/w5964.html

- Voucher & tax credit programs are more efficient
http://www.friedmanfoundation.org/friedman/research/ShowResearchItem.do?id=10079

- Voucher schools promote racial integration
http://www.friedmanfoundation.org/friedman/research/ShowResearchItem.do?id=10064

- Voucher schools increase parental satisfaction
http://hpi.georgetown.edu/scdp/

Etc, etc…. Again, vouchers or tax credits are not a magical panacea, but rather highly promising means to open up the educational system to the same forces that have brought improvements to virtually every other area of life we know. Research shows the promise. Research has undermined many of the arguments used against it.

And so the debate continues ….

6. Kevin Welner Says:
   August 2nd, 2007 at 4:09 pm e

Regarding the National Reading Panel versus the National Academy panel, I honestly can't speak with much knowledge. It's really not my field. I do know that the NRP report has been subject to some harsh critiques by researchers I know and respect. See
Camilli, G. & Wolfe, P. (2004). Research on reading: A cautionary tale. Educational Leadership, 61(6), 26-29.
And I know that NRC work " committees put together by the highly respected National Academies " is usually beyond reproach. But I can't offer much beyond that, except to say that common sense suggests to me that reading teachers should use a balanced combination of phonemic awareness and so-called ‘whole language', tailored to each child's needs. (I also can't speak in any detail to the question of whether teachers are prepared to teach as the NRP says they should. It probably varies by the skill, experience and courses taken by each teacher " most such things do.)

Regarding the voucher research, I can't say I'm completely familiar with all these, but two of them were reviewed as part of the think tank review project that I co-direct. You can find the reviews at the following urls. (The racial integration one held up to the scrutiny a bit better than did the efficiency one " but in both cases the reviewers raised important questions about the broad, sweeping policy conclusions offered by the reports' authors.)
http://epsl.asu.edu/epru/ttreviews/EPSL-0705-235-EPRU.pdf
http://epsl.asu.edu/epru/ttreviews/EPSL-0610-213-EPRU.pdf

I'll speak only generally to the issues of parental satisfaction and student achievement.

Satisfaction: I have always assumed that parents who actively choose a school will, at least initially, express greater satisfaction with the school. But there are three other questions that I think should be asked: (1) does the satisfaction continue over time, or does it wane? (2) is overall parental satisfaction increased by choice (is this a zero-sum game where kids and parents who are not actively choosing feel less well off)? (3) is any increase in parental satisfaction the result of educational improvements or is it something else? (The answer to this last point may not undermine the conclusion that the parental satisfaction is a good thing, but one would certainly want to know and understand.)

Achievement: Cecilia Rouse's reanalysis of the Milwaukee voucher data is one of the few studies that finds a statistically significant benefit (for math " no positive effect for children's reading). Unfortunately, most other data sets (including the better data coming the pro-voucher Peterson shop at Harvard) have generally found no such benefits.

The Barnard reanalysis of the NYC data is one of a half-dozen or so that have looked at this same data. Some researchers find an isolated but statistically significant benefit. Others do not. Most notably, Princeton researcher Alan Krueger reanalyzed this data and found any benefits to be highly dependent upon such things as how one defines African American (if only children with Black mothers are included, one sees a benefit, but the benefit disappears " I think entirely, if I recall correctly " if one also includes children with Black fathers).

So we should have lots of questions here about, e.g., why vouchers might help kids a bit in math but not in reading? Why it might help Black kids but not Latino or White kids? Why it might help in some grades but not in others?

But the truth is that these small blips of possible benefits are hardly sufficient to base a finding that vouchers provide any meaningful academic benefit. Most studies find no achievement benefit. And those few that do find a benefit have found only small, isolated, fleeting and generally non-robust benefits (that is, they don't stand up to reanalyses).

The fact that you support market-based, competitive policies is something I respect. I agree that competition works well in many areas of our society. Where we differ, it seems, is that I do not place non-empirical faith in the belief that market-based policies will drive improvements (and not drive further inequalities) in k-12 schooling. I'm certainly interested in the experiments, and I would be very happy if they did, in fact, turn out to be highly promising, as you say. But I just don't see it.

P20: Packed in Pikes Peak

Written by: Ed Rooney

The Packed solid Pikes Peak P-20 Meeting opened with the usual hyper-inflated introductions. So and so was honored to introduce their good friend so and so.  So and so was one of the most courageous leaders in education, yada yada.  The introductory remarks were intelligent, took on urgent tones and cited statistics that had been ignored or denied in the past. Most of the audience listened politely and then, in all likelihood, immediately forgot nearly everything that had been said, and walked away wondering if there would be Cheese-its in the vending machine.

A brief recap follows:

Barbara O'Brien: There are schools that prove they can close the gap; this should be done for whole state.

Joe Garcia: We must break down barriers that exist within systems.”

Dwight Jones: Colorado should make an effort to make sure that all kids graduate ready to choose whether to attend college/other.  He said that this was our moral obligation.

Gov. Ritter: How can children born today enter workforce so that they can compete, regardless of socio-economic status?”

Bruce Benson:  Urged participants to think outside box. Everything on table. Let's have great debate.”

Subcommittee meetings ensued. I didn't go to all of the rooms. I sat in on a couple, including much of the Dropout Prevention & Recovery meeting.

The subcommittees have just a few weeks to make legislative recommendations. They were tasked with answering a series of questions asked by the Governor's office. But the first outside of the box, thoughtful, everything-on-the-table-debate that I listened to centered around: what should the true name of the subcommittee should be: Retaining dropouts or preventing dropouts? Should they include both retaining and preventing in the name, or just one?  I grumbled.

Then, instead of thinking of ways that a system could be created which would retain students, the conversation jumped right back into the smallest box I could think of.

When should PPOR be counted?

While there are people with good ideas about this micro-topic, some of whom sat around the table, the subcommittee wasn't having the right discussion. 

Shouldn't we be asking can we engage students, not how schools should get paid for each student that doesn't drop out?

My point isn't really about this brief conversation by one of the subcommittees, which probably isn't a big deal one way or the other. Rather, my point is that the P-20 Council has been asked to think in very big ways about ways we can fundamentally change the system so that we don't continue to drop the ball.  Changing the way we educate our kids by talking about the number of school counselors in a school, the PPOR count dates, and the best name for the subcommittee is like trying to fix global warming by switching our lightbulbs or bringing canvas grocery bags to the quickie-mart. It ain't gonna happen.

One Response to P20: Packed in Pikes Peak”

1. Jeff Miller Says:
   August 8th, 2007 at 1:53 pm e

I think you're wrong. It's by talking about those little things that we get to the bigger things. When people talk about the name of the committee, they're really talking about what they want the committee to accomplish. When they talk about the PPOR count dates or the number of counselors, that's a way of getting to the question of whether they're effectively serving the students they have. The number of counselors also relates to your point about whether they're engaging kids enough to stay in school.

Like you, I'd like to see the Council take on the big issues. And I also don't think they're really going to put everything on the table because the Council doesn't include a lot of real iconoclasts. But I do think people have to talk about the little things that they understand as a way of getting to the bigger, more complex questions you and I really want them to address.

But then, I've switched my light bulbs and bought my own grocery bags.

Comments (0)

Subscribe to this comment's feed

Show/Hide comment form