A politically incorrect assessment of the new e-cigarette study

We have observed that most everyone who is interested in THR and has a blog has spun the results of recent research by Michael Siegel as wonderful good news for THR.  While it is tempting to join the cheerleading, it is our assessment of the results that they are a useful addition to our knowledge but, from the perspective of advocacy, a little disappointing.

I (CVP) used a news story that reported  this study as an example of bad health news reporting in my ep-ology blog’s Unhealthful News series, where I also pointed out how this needs to be interpreted as a report about a convenience sample, not a population-based study.  I will probably follow up on that post in another Unhealthful News within a week, picking up on some other technical points about the reporting.  (It is a good example for me to use because I am guessing that many of my readers are interested enough to read the news and journal articles in this case.)

So why are the results disappointing, you ask.

(I certainly hope no one is asking, “How dare you say that the results are disappointing?!!”  I would like to think that those of us who are interested in THR are honest enough to not behave like those on the other side.  If the results were preordained to be good news then it would not be scientific research, it would just be a demonstration.  If we were to always spin whatever result we got as being A Brand New Reason To Believe In Our Cause, then we would not be honest researcher-advocates, we would be the anti-tobacco extremists.)

Most of the results of the survey tend to support what we already knew with a high level of confidence:  E-cigarettes are an effective substitute for smoking, such that people who use e-cigarettes a lot are unlikely to be smoking also, and smokers who use e-cigarettes at all are likely to smoke less.  A decent portion of people who are dedicated to quitting smoking on their own, soon, by whatever means seems promising, with product substitution as an option (as demonstrated, e.g., by buying an e-cigarette kit), do succeed.  This contrasts with people who start using anti-smoking medicines, who are largely demonstrating a desire to change their preferences, which usually fails.  Also, people who cross the threshold of performing a costly act of volition (e.g., buying an e-cigarette kit) are not only already people who are more likely to follow up on the act, but also become more likely to follow up due to a useful “foot in the door” effect of cognitive dissonance.

In short, we know that if smokers have e-cigarettes, many of them will substitute them for smoking, at least to some extent.  Moreover, we predict that if smokers buy e-cigarettes, even more of them will substitute.  It is good to have more data that confirms these points.  If the results of the study had been radically less supportive of the value of e-cigarettes it would have tended to call into question what we think we know.  While the limitations of the study mean that it cannot really expand upon our expectations for e-cigarettes, it adds empirical support because it is consistent with what we believed.  (The key scientific point is that it could have had results that caused us to doubt our beliefs.  That possibility is the classic difference between science and religion.)  That is why empirical tests like this are useful, but should not be misinterpreted as telling us something new (as some empirical studies can, of course).

With that in mind, we have to observe that, given our existing knowledge and beliefs, the results of the study are a bit weaker than we might have hoped for.  The main result was that just under 1/3 of the e-cigarette buyers who chose to respond to the survey were abstinent from smoking at six months.  It is difficult to know exactly what this means because we do not know what questions the subjects answered (hint hint to MS! – please post the survey questions somewhere), but it does not seem to be a large number given that (a) it is one point in time (i.e., someone could have smoked a week before and a week after), (b) we would expect a response bias (and any such bias has large effects given that only 5% of people receiving an invitation responding), such that people who became fans of e-cigarettes were more likely to take the time to answer, and (c) we cannot rule out from what was reported (another hint!) that some respondents were not regular smokers in the first place (they might have even been buying the product for someone else).  Also, it is not clear from what was written whether some of the respondents might have been e-cigarette users already, at the time they made the purchase that defined them as part of the target population (the merchant that participated in the study was new at the time, but someone could have been a regular user trying a different brand).

I would have really hoped that the average (no response bias) regular smoker who was destined to still be a smoker in six months if unaided, who was not already a vaper, and who became knowledgeable and interested enough to order an e-cigarette kit, would have had at least a 10% and perhaps as much as a 20% chance of not smoking a few months later.  We can only guess how the results, which measure something quite different from that, compare to those figures, but they do not seem to rise to that level.

Also, a study of this kind does not let us estimate how many of the nonsmokers (at 6 months) would have been smokers without their new e-cigarettes.  No, this does not mean that I think that a clinical trial would be the best study, since its negatives are too important compared to its advantages, as I noted previously.  But it does mean that for this study design (or any other that has been done on the topic) we cannot attribute all of the abstinence to e-cigarettes, especially among those who had quit smoking but were not using e-cigarettes at follow-up.  At a minimum, it would be useful to compare the abstinence rate to known figures about the six month abstinence rate for unaided quitting among dedicated quitters (a bit under 10% seems like a good estimate).

Siegel attributes the subjects reporting complete nicotine abstinence at six months to the effect of e-cigarettes as a weaning tool, which might well be true but we can hardly be sure of that.  An entirely plausible alternative is that those e-cigarette buyers were destined to quit, one way or another.  This, of course, is the argument that is commonly made by those who deny the value of e-cigarettes as a cessation tool.  We have ample evidence that this does not explain most of the cessation-by-switching that is observed (thanks to detailed reports of individual switchers), but this dataset does not seem to improve our knowledge on that point.

In sum, the new study is a welcome contribution and we are glad that Siegel made the effort to do it, especially given the lack of funding available for research on the topic.  (Though — per the hints — it would be more welcome if we had a bit more information about the methods and results. Authors are forced into such inadequate reporting due to anti-scientific restrictions by journals, which prohibit taking enough words to really explain the methods.  The journal the report appeared in even prints the methods in a smaller font as if they do not matter! But since MS has a blog, it would not be hard for him to post the survey instrument and provide some more details about inclusion criteria, etc.)

But we should not overstate its importance — most importantly because overstating it tends to diminish the credit due to the sources of knowledge that we already have.  Also, because of the study limitations (note the comments I made about the response rate in my ep-ology post, in particular) we really do not want this to be treated as the landmark study which is our definitive source of knowledge on the subject, as some proponents seem to want to spin it.  It is useful, but it is not why we believe that e-cigarettes are a proven effective way to quit smoking.

As an aside, we have no idea how the merchant involved in the research advertised back in 2009 when they recruited the customers who are the study subjects, but if you look at their website now the images of users seem to have the age range of the cast of Glee (minus the teachers) and the message, “look how cool vaping is”.  We have tended to focus our field research on populations that are at lest 30 or even 35, where spontaneous quitting is less likely and thus THR is more important.  (The way that age ranges were reported in the new paper, we cannot tell how many of the respondents were under 30 or 35, but it could have been more than half.)  Vaping is always a healthier choice than smoking, young or old, but optics matter a lot.  The images at this now highlighted e-cigarette merchant are probably not good for the goal of convincing the world that e-cigarettes are targeted at dedicated smokers.