Vaccines Cause Autism, Until You Look At The Data

According to a much-discussed new paper, vaccines may cause autism after all: A Positive Association found between Autism Prevalence and Childhood Vaccination uptake across the U.S. Population.

The author is Gayle DeLong, who "teaches international finance at Baruch College, City University of New York", according to her profile as a board member of anti-vaccine group SafeMinds. She correlated rates of coverage of the government recommended full set of vaccines in the 51 US states including Washington D.C., with registered rates of autism in those states six years later.

Uh-oh - there was a correlation between vaccination in two year kids, and the rate of autism in the state six years later, when those kids were eight. As the abstract says:

The higher the proportion of children receiving recommended vaccinations, the higher was the prevalence of AUT... The results suggest that although mercury has been removed from many vaccines, other culprits may link vaccines to autism. Further study into the relationship between vaccines and autism is warranted.

Sounds rather scary. Until you look at the data, helpfully provided in the paper. First up, here's the scatterplot of all of the vaccination rates and all of the autism-six-years-later rates:

There's more than 51 data points as you can see: there's actually 355 because each state had seven different datapoints (1995 vaccines vs 2001 autism though to 2001 vs 2007). This scatterplot shows no correlation. You can tell just from looking at it, but the correlation coefficient confirms this, as it's a tiny r 0.012 (from a possible range of 0 to 1).

To be fair, that's a very noisy measure, because each state has unique characteristics, so the effect of vaccines will be diluted. However, it's still a useful sanity check, and shows that there can't be a major effect, otherwise it would be too big to get diluted.

To get around this I next looked at the change in the rates of vaccination from one year to the next, and correlated that with the corresponding change in future rates of autism, within each state. A "change" of 1 means no change, 0.5 means it halved and 2 means it doubled, etc.

Zilch. Correlation coeffiencent r is 0.034.

Maybe the changes year-to-year were too small? So I checked the changes between the last year, and the first year.

This made the changes bigger, because more tends to change over six years than in just one. And, to be fair, this does produces a slightly stronger vaccine-autism effect... but it's still tiny. The correlation coefficient here, r, is 0.18 which means that vaccination changes accounts for 3% of the variability in autism changes (r^2 = 0.034.) The p value is 0.20, statistically insignificant.

My conclusion is that this dataset shows no evidence of any association. The author nonetheless found one. How? By doing some statistical wizardry.

The statistical model used took into consideration the unique characteristics of each state. For example, each state had a unique mixture of pollution, which may have affected the prevalence of autism, yet such an effect was not included in this study. A fixed-effects, within-group panel regression (Hall and Cummins 2005) controlled for these unique yet undefined characteristics by deriving a different starting point (intercept) for each state.

The 51 different intercepts - one for each state - reflected the base level of autism or speech disorders occurring in that state that were not explained by the other independent variables (vaccination rates, income, or ethnicity). The model then produced a single relationship between the independent variables and the prevalence of autism or speech disorders.

OK, that's all very fancy, but when the raw data shows zilch and you can only find a signal by "controlling for" stuff, alarm bells start ringing. Given sufficient statistical analysis you can make any data say anything you want.

If the author had given details of the methods, and explained why she chose to control for the variables she did, and not others, that might be different. But she didn't. Nor did she justify only looking at the effects six years later, when five or seven or ten would be just as sensible... and so on.

(Note: whenever I've said "autism", that's my shorthand for autism + SLI, which is what the paper looked at; autism alone data are not presented. Note also that by "vaccination %" I mean "% who got the full vaccine schedule"; the other kids may have got vaccines, just not all of them.)

Delong G (2011). A Positive Association found between Autism Prevalence and Childhood Vaccination uptake across the U.S. Population. Journal of toxicology and environmental health. Part A, 74 (14), 903-16 PMID: 21623535