So, what was the frustration that finally broke the camel's resistance to jumping on the blog bandwagon? A 'report' released by the Council of Great City Schools and its corresponding New York Times article.
Here's a quote from the article:
"Only 12 percent of black fourth-grade boys are proficient in reading, compared with 38 percent of white boys, and only 12 percent of black eighth-grade boys are proficient in math, compared with 44 percent of white boys.
Poverty alone does not seem to explain the differences: poor white boys do just as well as African-American boys who do not live in poverty, measured by whether they qualify for subsidized school lunches.
The data was distilled from highly respected national math and reading tests, known as the National Assessment for Educational Progress, which are given to students in fourth and eighth grades, most recently in 2009. The report, “A Call for Change,” is to be released Tuesday by the Council of the Great City Schools, an advocacy group for urban public schools."
So what's my problem, you ask? There are several issues at play.
The first is that in general, people do not make the distinction between 'report' and 'research article in a peer reviewed journal'. The work in question is a report. That means a team of people got together, analyzed data, made pretty graphs, stated some conclusions and made recommendations. Then, these elements were compiled into a pdf and uploaded to a web site.
On the other hand, a research article would need to be reviewed by experts in the authors' field, preferably who did not have any direct ties to the work in said article. Those experts would provide critical feedback to the authors, and the authors would be required to address such criticism before the work could be published.
That is problem 1. There is no distinction made between some very different levels of work. People reading the New York Times article will generally trust that the work is accurate. They will cite the report's findings as fact.
On to problem 2. The statement in the NYT article that 'poor white boys do just as well as African-American boys who do not live in poverty, measured by whether they qualify for subsidized school lunches.' is misleading. At least the author states that the measure for poverty is whether or not one qualifies for subsidized school lunches. But is this an accurate measure of poverty? Let's get back to that question in a moment, after a word about the comparison of populations.
When comparing 'poor whites' to non-poverty stricken African Americans, the report compares white males in national public schools (NP) vs. black males in large city (LC) schools. NP means all public schools in the nation, LC means schools in cities with populations over 200,000. There is no justification for the comparison of two different populations. In fact, the authors state that when comparing LC to NP, tests of significance of difference could not be performed. Right. That's because they are comparing two different populations. Why did the authors choose to compare NP whites to LC blacks? A guess is that the population of white male in LC schools is too low to make meaningful comparisons. Whatever the reason, the topic of whether it is meaningful to compare these two groups could be the basis for a full (peer-reviewed) research article in itself.
Now, getting back to the question of poverty, consider the indicator: free or reduced price lunch (FRPL). These are students for whom a responsible adult has filled out the application form and whose household income meets requirements. In NC, where I live and have kids in the public school system, the eligibility requirements are the same statewide. That means that the same income levels are eligible, whether the child lives in a city (read higher prices for rent, food, etc) or in rural areas (read lower real estate values, lower prices overall). Additionally, FRPL is at best a rough indicator for poverty. Again in NC, the cutoff for reduced price lunch is 165% of the national poverty level. I would not suggest that 165% of the poverty level is well-off, but certainly there is a huge difference in quality of life for a child whose caretaker has an income of say, $0. In using a rough indicator and then stretching it across two different populations, the authors are left with a pretty much meaningless comparison.
That about sums up problem 2.
Problem 3 concerns overstating sketchy results. The authors point directly at this deeply flawed comparison and state:
"In fact, large city (LC) Black males not eligible for free or reduced-price lunch had reading and mathematics scores similar to or lower than those of White males in national public schools (NP) who were eligible for free or reduced-price lunch."
[Emphasis in original].
So let's put this all together. A group writes a report that is not peer-reviewed and which contains data analysis that is questionable. In that report, results are overstated. A NY Times journalist reads (skims?) the report and regurgitates some of the most inflammatory/controversial 'results' for public consumption. This is the source of my frustration.
I make no claim as to whether the authors' conclusions are correct or incorrect, I simply do not believe the arguments that lead up to them. And neither should you.
