Statistics: Tool of an Elitist Bastard

I was prepared to enjoy Susan Jacoby‘s Op-Ed in the New York Times, “Best Is the New Worst” (30 May 2008). Yes, the title “Best Is The New Worst” is a snowclone, the sort of trite phrasal template with plug-and-play slots which is the refuge of lazy writing. In fact, “X is the new Y” is a classic of the genre. (Is this headline choice an example of such, or a clever meta-reference intended to mock declining standards? Discuss.) However, this bit was more troubling, and made me look askance:

Another peculiar new use of “elitist” (often coupled with “Luddite”) is its application to any caveats about the Internet as a source of knowledge. After listening to one of my lectures, a college student told me that it was elitist to express alarm that one in four Americans, according to the National Constitution Center, cannot name any First Amendment rights or that 62 percent cannot name the three branches of government. “You don’t need to have that in your head,” the student said, “because you can just look it up on the Web.”

Ahem. Laziness in a student may well be a cause for alarm, but uncritically accepting the spin which innumerate and agenda-driven reportage has placed upon a survey is, I dare to suggest, even worse. I’m going to go out on a limb here and say that these numbers, intimidating though they may be, are in fact completely worthless. They don’t measure what they’re claimed to measure. The reason goes back to the First Amendment itself:

Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the government for a redress of grievances.

This breaks down to six different rights. The first half of the religion bit — the Establishment Clause — covers a different territory than the second half, the Free Exercise Clause. It’s possible to have the right to follow your personal religion even in a country which has an established church (hint, hint: the United Kingdom has Muslims). Speech and the press get one “freedom” each, but the people have a “right” to assemble and another “right” to petition.

The National Constitution Center website features a 1997 survey which speaks of “four rights” in the First Amendment, and lists them as “speech,” “religion,” “press” and “assembly.” Now, I’d argue that this obscures the constitutional status of religion, reflecting or enforcing an oversimplified view of the matter, but more importantly, the right to petition vanished down a memory hole.

If it were universally acknowledged that, say, petitioning and assembly were two faces of the same freedom, then I wouldn’t mind so much. However, a 2006 survey by the McCormick Tribune Freedom Museum asked respondents to name their First Amendment rights, and their “correct” answers were the freedoms of religion, speech, press, petition and assembly. When two different love-the-Constitution groups can’t even be consistent on how many freedoms exist, can the results be considered reliable? What, exactly, are we citizens supposed to know, and will we lose credit for knowing the wrong “right” answer? Suppose that the NCC called you one September morning in 1997, and you said that the First Amendment covered speech, religion, press and peaceable assembly. You would be a sterling citizen in 1997, but nine years later, the Freedom Museum would stamp you defective.

Both surveys share another flaw. As was pointed out back in 2006,
several “freedoms” which are likely to be uppermost in an American’s mind are found in other, later amendments: bearing arms, avoiding self-incrimination, avoiding unreasonable search and seizure, not being enslaved (took a while for that one) and so forth. What happens if you remember a whole list of these rights, but can’t recall which amendment they go under?

The performance of the vaunted old media regarding that 2006 survey was, the Gentle Reader might recall, spin-a-licious: the survey said that 28 percent could name “more than one” of the five freedoms, but this quickly got turned around into the assertion that only 28 percent could name “one or more” of them. Oops. As Mark Liberman said of the 2006 survey, the Freedom Museum’s presentation “was a classic example of the rhetoric of public relations — not the most dishonest one ever seen, but not overly scrupulous either — and the press swallowed it hook line and sinker.”

What about the “62 percent cannot name the three branches of government” figure? Well, let’s set aside Alex Knott’s claim that lobbyists are a fourth branch of government (he works for the Center for Public Integrity, and hey, those pinkos were funded by George Soros). Likewise, let’s ignore the reading I had to do in civics class which said that the federal administrative agencies, including those nominally under the Executive Branch, in fact constitute a fourth branch unto themselves. They combine the powers which Montesquieu said we should separate, and they’re not really covered by the Constitution, except for an offhand mention of unspecified “executive Departments.” No doubt a survey of the general American populace would find them even more ignorant of these matters than of the canonical Constitution itself.

Suppose we take that 62% figure as measuring how well people know what the Constitution says, never mind what political wonks have asserted the reality to be. Is it actually informative? I don’t think it is. It might mean that 62% (plus or minus the sampling error of a 1,000-individual sample) can’t name any branch of government, while the other 38% can name them all. Or, perhaps, “cannot name the three branches of government” means that a respondent gets two out of three correct and flubs the third. Or, that 62% might include respondents who say, “The President, the Supreme Court and Congress.” It might even include the total smart-aleck bastards — i.e., the people who took AP U.S. Government — who rattle off, “Executive, Legislative, Judicial, Administrative and the Press.”

The NCC gives a few more details:

62% cannot name all three branches of the Federal government. (One-third cannot name even one. 14% can only name one. 15% can name two.)

So, that 62% figure includes people who got one or two branches correct — not so dismal as we had led you to believe! Back in 1997, the NCC survey found that a bit more than 50% of respondents could name the legislative branch, and likewise for the executive and judicial. (Roughly 60% knew that the Constitution specified three branches.) I would be surprised if these were independent: it seems likely that knowing two of the three makes you more likely to know the third, since people who know them all were drilled in them. If they were independent, then the probability of naming all three would be the product of the probabilities for correctly naming each branch, which works out to about 15%. This is significantly lower than the 38% who nailed the question. Furthermore, the probability of getting none correct if each answer is independent — being able to name no branches — is about 11%, which is only a third of the probability reported by the survey.

If the branch-naming probabilities are not independent, then those who do know their Constitution are plausibly likely to know it fairly well. This is an important thing to recognize: For example, when faced with daunting statistics which indicate an uneducated populace, do we overhaul the school curricula, or do we try to keep kids in school longer? These are not mutually exclusive options, but we have to set priorities. Given incomplete or misleading information, we will not be able to plan effectively, and the figures waved about by Jacoby and the NCC are, I believe, just that.

Now, I don’t carry this knowledge around in my head. I remembered reading criticisms of the Freedom Museum’s methodology two years ago, and I looked up those critiques again. If you do just carry such figures around in your head, and you don’t check the digital record before speaking up, you’re likely to present numbers out of context, leaving your argument resting upon statistics which are perhaps suggestive but definitely sketchy. That’s the paradox of the Internet: it gives lazy people the tools to get by without working, but to those who care, those same tools are brain enhancement devices. The true Elitist Bastard, I hypothesize, condemns the cause, apathy, rather than the symptom of Google-reliance.

NOTE TO THE READER: This post initially contained a stupid error on my part. Working without sufficient caffeine, I had said that the two surveys were conducted by the same organization. In fixing that error, I took the opportunity to add a few remarks on the “name the branches of government” figure. Mea maxima culpa.

9 thoughts on “Statistics: Tool of an Elitist Bastard”

  1. Actually if you want to nit-pick more. It’s 6, not 5 or 4 in the 1st Amendment.

    1:The right to have a government that does NOT establish a religion.
    2:The right to practice your religon.
    3:Freedom of speech
    4:Freedom of the press
    5:Freedom of assembly
    6:Freedom to tell the politicos that they’re a buncha poopieheads.

    The difference between a Freedom and a Right is more or less linguistic hooey in this context.
    I pretty much agree with you as well. Homer Simpson said it best: “Oh, people can come up with statistics to prove anything. 14% of people know that.”

  2. Good point.

    I remembered the difference between the “establishment clause” and the “free exercise clause” from civics class long ago, but this post was getting nitpicky enough that I decided to go ahead and publish it before it imploded into a crotchety black hole.

  3. This business about the first amendment is bad surveying and reporting, I agree, but I’d still like Americans to know what the three branches of government are. It’s hard to screw that one up.

  4. Oh, this is weird. Like I said, the 1997 survey was done by the NCC, but the 2006 one was done by the McCormick Tribune Freedom Museum, which didn’t even exist until 2006. So, I was wrong in saying they were conducted by the same group . . . but two different love-the-Constitution organizations can’t agree on what the First Amendment says!

    I think I’m going to have to overhaul this post.

    Back in 1997, the NCC survey found that a bit more than 50% of respondents could name the legislative branch, and likewise for the executive and judicial. Roughly 60% knew that the Constitution specified three branches, and about 53% of respondents could name two or more of them.

  5. Blake… I think you just performed a miracle. This was an article on statistics. I understood it. It didn’t make my eyes glaze. It was interesting. If you understood the bad blood between me and statistics, you’d know why I’m stunned by this. Nicely done!

  6. You’re missing the point.

    The point: an alarming number of Americans take their freedoms and rights for granted, and don’t understand their own system of government. All statistics you present in this article, right or wrong, lead to these same conclusions. Fine, so it’s a 15% chance of getting all three branches right, and that’s lower than 38%. Either way, more people should know.

    Are you saying, you think enough people know the right answers? Do you abstain from making these kind of jugdements? Is it in fact *you* who is placing undue weight on numbers? What exactly is your point?

    Here’s mine. People should properly quote and cite sources, and not grant undue weight to statistics, but your nit-picking has added nothing of value to this particular discourse.

  7. frank said,

    Fine, so it’s a 15% chance of getting all three branches right, and that’s lower than 38%. Either way, more people should know.

    If I understand correctly what you wrote here, what you wrote is not what I wrote. I argued that the limited data the NCC made available does not support the hypothesis of independent events. From their numbers, if you assume that the event “person knows the name of the legislative branch” is statistically independent of the event “person knows the name of the executive branch,” then the total fraction of people who can name all three branches would be about 15%. The figure they measured was 38%. Therefore, it is reasonable to say that the data falsifies the hypothesis of statistical independence.

    Consequently, we might propose new hypotheses: for example, we could suggest that people who are well-enough educated that they know any branch are pretty likely to know them all. If this were true, it would have policy implications. In order to make well-informed policy decisions, we need additional data which the NCC has not made available. Citing their figures as if they had immediate policy implications is deceptive and does little more than monger fear.

    That’s my point.

    Yes, more people should know. That much is obvious. Ignore your rights and they’ll go away, etc. In order to solve the problem, however, we need to know more about what is actually happening than we have been told.

Comments are closed.