<< Previous Post | Main | Next Post >>

TELLING TRUTH WITH STATISTICS

Interesting report on the difeerence - or rather, the lack thereof - between public and private school educations. It seems that once you control for socioeconomic status, public school kids perform as well or better than private school kids on standardized tests.

There's plenty to read in the full report, a document which apparently at least one private school representative hasn't read. Take a look:

Joseph McTighe, executive director of the Council for American Private Education, an umbrella organization that represents 80 percent of private elementary and secondary schools, said the statistical analysis had little to do with parents’ choices on educating their children.

"In the real world, private school kids outperform public school kids," Mr. McTighe said. "That’s the real world, and the way things actually are."

Maybe I'm being unfair. Maybe Mr. McTighe has actually read it. Maybe instead the man has no clue how statistics work. What the analysis shows is that it isn't private vs. public that makes the difference - its the socioeconomic status of the child's family that's key. Social scientists have long understood that the higher one's SES, the more likely one is to do well in school. What this study shows is that once you control for SES, private vs. public has little to no positive impact on achievement. Or worse, that it at times has a negative effect. I understand that he's paid not to understand it, but still...

And while I'm dissecting this report, I have one other nit to pick. First, the quote from the Bush administration, as interpreted by the so often clueless NYT:

An Education Department official who insisted on anonymity because of the climate surrounding the report, said researchers were "extra cautious" in reviewing it and were aware of its “political sensitivity.”

The official said the warning against drawing unsupported conclusions was expanded somewhat as the report went through in the review.

The report cautions, for example, against concluding that children do better because of the type of school as opposed to unknown factors. It also warns of great variations of performance among private schools, making a blanket comparison of public and private schools “of modest utility.” And the scores on which its findings are based reflect only a snapshot of student performance at a point in time and say nothing about individual student progress in different settings.

It's that "of modest utility" part that's got me bothered. I wondered what they meant by that, so I headed into the full report to take a look. Here's that line in context:

When interpreting the results from any of these analyses, it should be borne in mind that private schools constitute a heterogeneous category and may differ from one another as much as they differ from public schools. Public schools also constitute a heterogeneous category. Consequently, an overall comparison of the two types of schools is of modest utility. The more focused comparisons conducted as part of this study may be of greater value. However, interpretations of the results should take into account the variability due to the relatively small sizes of the samples drawn from each category of private school, as well as the possible bias introduced by the differential participation rates across private school categories.

The point of this line wasn't to suggest that the comparisons made in the report are of modest value. If anything, that interpretation has it backwards. What its really saying is that you need to dig deeper into the analysis, to look past the public vs private dichotomy to compare subsets of school within each of the groups. And if you do that, that's where the report gets really interesting. And to see that you need look no further than the highlights provided by the NYT:

Students in private schools typically score higher than those in public schools, a finding confirmed in the study. The report then dug deeper to compare students of like racial, economic and social backgrounds. When it did that, the private school advantage disappeared in all areas except eighth-grade reading.

And in math, 4th graders attending public school were nearly half a year ahead of comparable students in private school, according to the report.

The report separated private schools by type and found that among private school students, those in Lutheran schools performed best, while those in conservative Christian schools did worst.

In eighth-grade reading, children in conservative Christian schools scored no better than comparable children in public schools.

In eighth-grade math, children in Lutheran schools scored significantly better than children in public schools, but those in conservative Christian schools fared worse.

Now gee... why would the Bush administration want to bury a report showing that kids in conservative Christian schools were at a disadvantage to those in public schools? I cannot possibly imagine....


0 TrackBacks

Listed below are links to blogs that reference this entry: TELLING TRUTH WITH STATISTICS.

TrackBack URL for this entry: http://blog.alexwhalen.com/cgi-bin/mt/mt-tb.cgi/39

Leave a comment