Mice, Math and Drugs: On Science without Understanding

The latest issue of Neuropsychopharmacology is chock full of goodies - not only one of the first ever controlled trials of medical marijuana, but also a surprise gem from an American-Israeli collaboration, called A Data Mining Approach to In Vivo Classification of Psychopharmacological Drugs. Yet despite being an excellent paper, it raises some worrying questions about what is and isn't science.

In a nutshell, the authors sought to discover a way of efficiently determining what a drug does. There are several broad classes of psychoactive drugs, such as stimulants, e.g. cocaine, and opioids, e.g. morphine. If you want to find out whether an unknown drug has opioid-like painkilling effects, for example, you have to test for them specifically - e.g. by measuring how the drug alters a mouse's pain threshold in a test called the Hot Plate test (guess what that involves.) If you want to test whether the same compound has antidepressant effects, you would have to do a different test entirely, like the Porsolt test. And so on.

The authors tried - and claim to have succeeded - to find a way of detecting the effects of drugs in a single, simple test. The test involved putting a mouse onto an empty circular platform (an "open field") and just allowing it to run around for an hour. A camera records the movements of the mouse, and a computer analyzes the video to give the mouse's position every 1/30th of a second. The result is a series of numbers showing the path which the mouse took around the area.

The clever bit follows: from this path data, one can derive various other numbers - for example, the mouse's velocity, acceleration, and direction of movement relative to the wall of the platform, at any given point in time. An hour of a mouse's life can be broken down into a veritable mountain of data (especially since there are 30 x 60 seconds x 60 minutes = 108,000 time points.)

The authors then used a technique called data mining to discover patterns in this data which could be useful in discovering drugs. Data mining is nothing complicated - it essentially means taking a lot of data and searching it all for anything interesting. In this case, they injected mice with various doses of various different drugs from three different classes - stimulants, opioids, and "psychotomimetics" such as phencyclidine (angel dust) and ketamine. They recorded their movement over the course of an hour and analyzed it to get 10 numbers ("attributes") at each of the 108,000 time points. They then considered the combination of up to 4 different attributes simultaneously in a procedure they call (and have no doubt patented as) "Pattern Array Analysis".

The single-attribute pattern coded P{*,*,3,*,*,*,*,*,*,*} is defined only by the third bin (40-60 cm/s) of the third attribute (speed), ie the animal is moving moderately fast... as more attributes are added to the definition of a pattern it becomes more and more specific, eg the four-attribute pattern P{*,*,1,2,*,1,5,*,*,*} means moving very slowly while slightly decelerating in the direction of the arena wall but turning sharply away from it.

They then took every one of this huge number of possible "behaviour patterns" (there were 73,042), measured how many times each mouse did each one over the course of the hour, and worked out which patterns became more or less common after giving each of the different drugs. They ended up with this:
This is a plot with 73,042 dots on it. Each dot represents a pattern of mouse movement behaviour. Dots further to the right represent behaviours which are more common, while dots higher up represent behaviours the frequency of which is most significantly different between mice given opioids and mice given other drugs (or no drugs). Most of the dots are low down the plot, showing that the opioids had little effect on them. But the dot with an arrow pointing at it represents a behaviour which is both common, and much, much less common in mice injected with opioids; in fact the significance p value of the difference is below 0.00000000000001 (that's 15 zeroes).

What is this behaviour? It's P{*,*,*,*,4,*,*,*,*,*} (‘moderately positive jerk’), meaning that the mouse's acceleration was increasing at a certain point in time (for those who know calculus: the second derivative of speed was positive & quite high). So, give a mouse morphine, and you can be pretty sure that its acceleration won't be increasing very often. Hmm. A similar procedure was performed for the other two classes of drugs.

Now, what on earth does that mean? Why do opioids suppress the ‘moderately positive jerk’? No-one knows - and the odd thing is that we don't need to know. Once we've identified the pattern of behaviour to look for, we could use it to determine whether drugs have opioid-like activity, even if you haven't got any idea why it works. And it does work - the authors report that by looking for the right behaviours, they could successfully classify a range of other drugs, including a couple of mystery drugs for which the person running the experiment didn't know what they were. This plot shows the success rate; the three classes of drugs are in different colours, and they clearly occupy three distinct regions of the "space", the two dimensions of which are frequency of two different patterns of behaviour. Overall, this is a very impressive paper, and the practical implications are potentially very great - soon, it might be possible to tell what effects a newly designed drug has, all in a single mouse test. This could greatly speed up, and reduce the cost, of drug discovery. For drug companies, it could be very useful indeed.

But is it "science"? This paper doesn't really add to our understanding of the world - all it does is tell us that a seriously obscure aspect of mouse movement, 'moderately positive jerk’, is altered by opioids. This is a potentially useful fact, especially if you're a drug company, but it's a completely uninterpretable one - it doesn't help us to explain, or understand, anything about mice, or opioids, or anything. It's not a theory or a hypothesis, and it will probably never give rise to one. It's just an isolated, brute fact. This is the kind of "science" that the most hard-core logical positivist would be happy with.

And this kind of thing is becoming popular in neuroscience. Essentially similar techniques are becoming widely used in fMRI data analysis. Here's a diagram from another paper from 2007 reporting on a method of using genetic algorithms to data-mine MEG data (a way of recording changes in the magnetic field surrounding the brain) to discover patterns which could be used to diagnose various neurological and psychiatric illnesses. It works:
It's an elegant technique and it's a nice result. But again, no-one has any idea what this diagram really "means" and almost certainly no-one never will. The fact that the schizophrenia patients and the Alzheimer's disease patients occupy different areas of this imaginary 2D "space" defined by two complex variables somehow derived from a huge mountain of numbers is potentially useful, if you want to diagnose a disease, but it tells you absolutely nothing about that disease. It's like going to a witch-doctor and asking if someone is ill; she's always right, but if you ask her how she knows, she just says "By magic".

Data mining's cool, but when it's done like this, it's not science...

Neri Kafkafi, Daniel Yekutieli, Greg I Elmer (2008). A Data Mining Approach to In Vivo Classification of Psychopharmacological Drugs Neuropsychopharmacology, 34 (3), 607-623 DOI: 10.1038/npp.2008.103


Apostolos P Georgopoulos, Elissaios Karageorgiou, Arthur C Leuthold, Scott M Lewis, Joshua K Lynch, Aurelio A Alonso, Zaheer Aslam, Adam F Carpenter, Angeliki Georgopoulos, Laura S Hemmy, Ioannis G Koutlas, Frederick J P Langheim, J Riley McCarten, Susan E McPherson, José V Pardo, Patricia J Pardo, Gareth J Parry, Susan J Rottunda, Barbara M Segal, Scott R Sponheim, John J Stanwyck, Massoud Stephane, Joseph J Westermeyer (2007). Synchronous neural interactions assessed by magnetoencephalography: a functional biomarker for brain disorders Journal of Neural Engineering, 4 (4), 349-355 DOI: 10.1088/1741-2560/4/4/001