Dr. John Halpern Responds to UK National Health Service Critiques of New MDMA Study

An article from the U.K. National Health Service (NHS) adeptly summarizes the purpose, methods, and results of Dr. John H. Halpern’s latest study of the risks of Ecstasy. which showed no evidence of cognitive damage associated with heavy recreational Ecstasy use (MAPS press release; Addiction journal press release). However, while it rightly acknowledges that “this research cannot confirm that Ecstasy is a safe drug,” it does make some glaring errors in its conclusions about the study’s significance. After reading the NHS article, see below for Dr. Halpern’s careful and well-reasoned response.

  1. NHS: “However, the number of participants was low and the researchers highlight that the small sample size may have prevented an effect from being observed.”
    JHH: No study can exclude completely that there are methodological limitations. That being said, the sample size was not low or small. In fact, this is (I believe) the largest neurocognitive study of Ecstasy users ever attempted or reported. Where were these critics when all prior studies with even smaller sample sizes were relied upon almost without question just because those studies had findings of harm? There is no credibility to the claim that our study was “relatively small.”
  2. NHS: “Additionally, the study did not follow the participants over time to assess whether their brains had changed with ecstasy use.”
    JHH: This is irrelevant if not meaningless, because we took people who had used ecstasy 20 or more times (range: 22-450 lifetime experiences). We don’t need to study individuals repeatedly to evaluate chronic, long term effects. If someone is an alcoholic for 20 years, do we need to do serial testing to confirm that damage worsens over time? That is a different study. We can compare heavy alcoholics to non-users, and such a study will indeed reflect the cumulative effect of use–no different from the study we completed.
  3. NHS: “While the study was well conducted, illicit drug use can be difficult to research, and this research cannot confirm that ecstasy is a safe drug.”
    JHH: Who said our study was about confirming “that ecstasy is a safe drug?” In fact, we specifically stated that our report should not be twisted into asserting that MDMA is “risk-free.”
  4. NHS: “Without further research, it is not possible to say conclusively that this statement is correct [statement is that there is no evidence that ecstasy causes brain damage].”
    JHH: As if these people ever said that about any of the prior studies that had negative findings with much smaller samples and tons of confounds. What is critical to understand here is that we don’t have these sorts of debates over the damage from alcohol. The fact is that there are 20-plus years of research in this field and it still isn’t conclusive for harm: This alone is quite telling. We don’t have this sort of debate over alcohol, so why with MDMA? I suspect that it’s quite likely that our findings have zoomed in on what has achieved the wide range of results in studies of the cognitive performance of Ecstasy users: Confounds were poorly controlled for in the past.
  5. NHS: “The participants had been asked to refrain from taking ecstasy for 10 days before these tests.”
    JHH: In fact, we required at least 21 days from last use. This is important because almost all other research evaluated users after little to no delay from last use. Only a few other studies waited at least 14 days from last use. Why delay? So that we don’t test people in some sort of prolonged recovery phase. We were striving to evaluate for long-term effects, not capture decreased performance from most recent use. In fact, only 4 subjects were tested within 40 days from last use, with the shortest time being 25 days from last use.
  6. NHS: “Some exclusion criteria, such as having fewer than 50 sustained drinking sessions, were relatively restrictive given that the study looked at illegal drug use. Therefore, the participants may not have been representative of typical ecstasy users. It also suggested that participants may not have mixed their ecstasy use with drinking or other drugs, a behavior that might potentially have some effect on the brain.”
    JHH: I don’t believe the NHS ever complained about polydrug use as the culprit in prior evaluations of Ecstasy users. Our study showed that any harms found in polydrug Ecstasy users may be due to other drugs or combinations of drugs. This must now be dealt with more directly than just using statistics to control for other drug use. We purposely sought out relatively exclusive users of ecstasy to zero in on the neurocognitive performance of those users, free from the confound of polydrug use. The NHS commentary points out a confound in opposition: If we just stick with polydrug users in this type of research, then the picture remains murky on harm, enabling them to stay “on message” that all drugs cause damage. Our ambition is only to engage the scientific method to elucidate useful data. We published our findings, and doubters are free to follow the published methodology and attempt to replicate our work. By the way, we did publish mostly similar results in 2004: that was our pilot data and constitutes a separate study. Our present study does not include any data from the 2004 paper, meaning that the majority of our findings are already replicated in our present report.
  7. NHS: “This study looked at cognitive function using various tests, but did not look at brain structures (such as by using brain scans). As this study was not designed to detect brain damage and did not follow people over time, any differences it might have found in brain function could not have been confirmed as permanent or temporary.”
    JHH: This is quite bizarre from my perspective. It appears that the NHS believes that only by looking at “brain structures (such as by using brain scans)” is it possible to prove brain damage. This is false (except, of course, for visualizing images of tumors, strokes/infarctions, or other neuroanatomical damage). Images of the brain will not show that a person has damaged cognitive performance. Poor cognitive performance indeed can, however, indicate brain damage. Indeed, the Revised Strategic Application Test (RSAT) was developed specifically to evaluate those with traumatic brain injury and other physical injuries to the brain. As for the complaint about not following people over time, this is not a limitation of our research (see my comments above). The issue of “permanent” versus “temporary” just makes no sense: We evaluated longstanding users of Ecstasy versus non-users and we waited a long time from last use in order to focus on the longer-term functional consequences of use. Or, rather, is the NHS claiming that healthy brain function is “temporary?” It is true that no study is perfect, and that there are limitations and benefits to each approach. That being said, let’s see what other research teams might find if they follow our lead in removing the types of confounds that have plagued this area of research for far too long. I suspect that most will soldier on explaining away their study confounds rather than addressing them (which, in all fairness, is exactly what I am doing in offering up my perspective on the limitation
    s of our present work raised by the NHS). I salute the NHS for offering a mostly fair, reasoned, and reasonable commentary on which then to debate this science.