We are living in the age of big data. Everything we do, everywhere we go, every
website we visit, every book we read, every comment we make on social media is
being recorded by people who wish to make a profit out of information about us. Everything known about you is fed into
analysis routines that try to characterize you and predict how you are likely
to behave as a consumer, as a voter, as an employee, as member of an insurance
plan….. The list of uses of your
information is already long and continues to grow. Cathy O’Neil has written a fascinating, and
somewhat scary, evaluation of where this big data economy is taking us in her
book Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy.
She describes a number of troubling
developments that have received little discussion. One of the areas in which supposedly unbiased
data analysis routines inject bias into evaluations of individuals is the ranking
of teachers. In a brief preface to that
discussion O’Neil provided a startling revelation about a famous (now infamous)
document that altered the history of education in this country and fueled a movement
to replace public education by private education. That revelation will be the topic here.
Back in 1983 the economy was not particularly healthy, and
countries such as Japan and Germany seemed to have passed us by economically. Someone had to be at fault. A group was convened during Reagan’s administration
to determine how our education system could be the problem. This group issued a document titled A Nation at Risk. One of the conclusions of the report was that
average SAT (Scholastic Aptitude Test—used for college admission) scores had
been falling for a considerable period; therefore our schools are failing us
and putting our national security at risk. It turns out those who wrote the report got
the math all wrong.
“In fact, misinterpreted
statistics run through the history of teacher evaluation. The problem started with a momentous
statistical boo-boo in the analysis of the original Nation at Risk report. It
turned out that the very researchers who were decrying a national catastrophe
were basing their judgment on a fundamental error, something an undergrad
should have caught. In fact, if they
wanted to serve up an example of America’s educational shortcomings, their own
misreading of statistics could serve as exhibit A.”
There is something called Simpson’s Paradox, a situation
where the average of a population can trend in one direction while subsets of
the population can have averages that trend in the other direction. This can occur when the numbers in the
population subgroups are changing. It is
well known that performance on the SAT correlates strongly with family
income. Over the period studied, a large
number of lower-income applicants began taking the SAT test causing the average
to go down. However, if the test takers
were broken into income groups, then the data indicated that SAT scores were
actually increasing for all income groups.
The education problem did not exist.
“Seven years after A Nation at Risk was published with such
fanfare, researchers at Sandia National Laboratories took a second look at the
data gathered for the report. These people
were no amateurs when it came to statistics—they build and maintain nuclear
weapons—and they quickly found the error.
Yes, it was true that SAT scores had gone down on average. However, the number of students taking the
test had ballooned over the course of those seventeen years. Universities were opening their doors to more
poor students and minorities. Opportunities
were expanding. This signaled social
success. But naturally, this influx of
newcomers dragged down the average scores.
However, when statisticians broke down the population into income groups,
scores for every single group were rising, from poor to the rich.”
“The damning conclusion in the Nation at Risk report, the one that
spurred the entire teacher evaluation movement, was drawn from a grievous
misinterpretation of data.”
O’Neil references an article by Tamim Ansary written in
2007, Education at Risk: Fallout from a Flawed Report, as the source for this information. Ansary provides us with valuable background
information.
“What we now call school reform isn't the product of a gradual consensus
emerging among educators about how kids learn; it's a political movement that
grew out of one seed planted in 1983. I became aware of this fact some years
ago, when I started writing about education issues and found that every reform
initiative I read about -- standards, testing, whatever -- referred me back to
a seminal text entitled ‘A Nation at Risk’."
“Naturally, I assumed this bible of school reform was a scientific research
study full of charts and data that proved something. Yet when I finally
looked it up, I found a thirty-page political document issued by the National
Commission on Excellence in Education, a group convened by Ronald Reagan's
secretary of education, Terrell Bell.”
Ansary
provides a direct quote from the document to illustrate the political hyperbole.
"Our Nation is at risk . .
. . The educational foundations of our society are presently being eroded by a
rising tide of mediocrity that threatens our very future as a Nation and a
people . . . . If an unfriendly foreign power had attempted to impose on
America the mediocre educational performance that exists today, we might well
have viewed it as an act of war . . . . We have, in effect, been committing an
act of unthinking, unilateral educational disarmament . . . ."
It seems Reagan was, as usual, clueless about the meaning
of the report.
“As commission member Gerald Holton recalls, Reagan thanked the
commissioners at a White House ceremony for endorsing school prayer, vouchers,
and the elimination of the Department of Education. In fact, the newly printed
blue-cover report never mentioned these pet passions of the president. ‘The one
important reader of the report had apparently not read it after all,’ Holton
said.”
However, Reagan’s handlers saw the advantage that could
be had.
“Once launched, the report, which warned of ‘a rising level of mediocrity,’
took off like wildfire. During the next month, the Washington Post alone
ran some two dozen stories about it, and the buzz kept spreading. Although
Reagan counselor (and, later, attorney general) Edwin Meese III urged him to
reject the report because it undermined the president's basic education agenda
-- to get government out of education -- White House advisers Jim Baker and
Michael Deaver argued that ‘A Nation at Risk’ provided good campaign fodder.”
“Reagan agreed, and, in his second run for the presidency, he gave
fifty-one speeches calling for tough school reform. The ‘high political payoff,’
Bell wrote in his memoir, ‘stole the education issue from Walter Mondale -- and
it cost us nothing’."
“What made ‘A Nation at Risk’ so useful to Reagan? For one thing, its
language echoed the get-tough rhetoric of the growing conservative movement.
For another, its diagnosis lent color to the charge that, under liberals,
American education had dissolved into a mush of self-esteem classes.”
But what about
the Sandia analysis that indicated that so many of the conclusions of the
report were wrong? Ansary tells us that,
for reasons that are not clear, the secretary of energy, Admiral James Watkins,
asked one of the Department of Energy National Laboratories, Sandia, to
reevaluate the issue in 1990 using the relevant data.
“Systems scientists there
produced a study consisting almost entirely of charts, tables, and graphs, plus
brief analyses of what the numbers signified, which amounted to a major ‘Oops!’
As their puzzled preface put it, ‘To our surprise, on nearly every measure, we
found steady or slightly improving trends’."
So, the original conclusions were proven wrong. There must have been some reaction. Right?
“The government never released
the Sandia report. It went into peer review and there died a quiet death.
Hardly anyone else knew it even existed until, in 1993, the Journal of
Educational Research, read by only a small group of specialists, printed the
report.”
Ansary does not say when he became aware of the Sandia
report, but one suspects it took him until near the time of his article, 2007,
to realize it existed. Meanwhile, in
spite of the existence of the Sandia conclusions, each successive president has
tried to demonstrate the ability to be tougher on our “failing” school systems.
The Republican Party is still trying to eliminate the
Department of Education, and it is still trying to take education out of the
hands of educators and turn control over to profit-making or religious entities. All the initiatives, such as vouchers to
provide choice, increases in charter schools, and destruction of teachers’
unions, are aimed at funneling public education funds into private pockets.
The truth is that students with a good economic backing,
either from family or from the state, do well in school compared to students
from other countries. Students from poor
backgrounds, either due to family or to state circumstances, do poorly. As long as the number of poorly-supported
children is maintained or grows, we, as a country, will average out to
something that could be called mediocrity.
The problem is not our education system; the problem is our political
system.
The interested reader might find the following articles
informative:
No comments:
Post a Comment