Kanazawa's Black Day

Satoshi "The Scientific Fundamentalist" Kanazawa has suddenly become the most talked-about neuro/psychology blogger in the world, getting a BBC News article all to himself.


There's no need for the rest of us to feel jealous though. There may be no such thing as bad publicity, but I think that being accused of being a racist and a sexist who should be sacked, for something you wrote on your blog, something that swiftly got pulled, must come pretty close.

You can read the controversial article here, and in other places, because it was helpfully archived. It used to be here.

Kanazawa based his argument on the Add Health project which was a massive observational study of American adolescents and young adults.

Add Health is huge. It's produced over 3,000 scientific papers, presentations and other documents. That's because it collected a wealth of data on everything from genetics to blood chemistry to social relationships and emotional issues.

Kanazawa looked at the data on physical attractiveness. Attractiveness was rated by an interviewer. Each subject got interviewed for a couple of hours by one interviewer and at the end the interviewer rated how hot they found them.

The fateful post claimed that, according to the Add Health data, black women were rated less attractive, on average, than white, Asian and Native American ones. Let's assume that he's done his sums correctly and that this is true of the data.

The obvious problem is that maybe the interviewers were biased against black women, and rated them lower for that reason. Kanazawa didn't consider this in his post, which is unquestionably an oversight, but he did go on to speculate as to the biological reasons why they might be less attractive.

However, looking at the original Add Health data, can we check whether this bias was at play or not?

Short answer: I found no evidence either way.

Long answer: I first looked over the Add Health website but it doesn't seem to mention anything about who the interviewers were. It doesn't mention their own ethnicity, which would be helpful, although even if they were all black themselves, they might have internalized racism, so that wouldn't be conclusive. They were trained, but then, you can't train someone to not be a racist.

Then I decided to look at the publications. I searched Google Scholar for "Add Health" + attractiveness. This reveals a number of articles, including a 2007 one by Kanazawa ironically, but only one seemed really relevant: Weight Preoccupation as a Function of Observed Physical Attractiveness. (There are other hits, but I skimmed the most likely looking ones and they didn't address bias.)

The details are unimportant, but it involved race and attractiveness, so the authors had to deal with the question of potential rater bias. Unlike Kanazawa they didn't just brush this under the carpet:

Although the interviewers were different races and ethnicities, there is no information about the race or ethnicity of the interviewer for any one respondent to examine systematic bias.

However, post hoc cluster analyses that controlled for an interviewer effect yielded similar results; thus, it is unlikely that interviewers had any substantial biases against any one ethnic group or that they rated attractiveness significantly differently from each other.

The point about "post-hoc cluster analysis" is the key here. To try to control for rater effects (not just racial ones) they analyzed the data covarying for which interviewer rated each girl. They didn't know what races the interviewers were, but they did know which girls got rated by the same interviewer. They found that controlling for the rater did not affect their results.

So does that mean there was no bias? No. Because - this only applies to their results, which were not about attractiveness per se, but about the interaction of attractiveness with other factors to predict an outcome variable (dieting and concern about weight) within a given race.

Even supposing that half of the raters were KKK members who cruelly subtracted, say, a million points from the rated attractiveness of any given black subject, so long as they still rated some black subjects as more attractive than others, all of the comparisons within the black subjects would still work fine: the millions would all cancel out.

So in my judgement, we just can't tell. Unless I've missed something, in which case, please tell us about it in the comments.