Half of four posts about bad recent bad analysis

by Carl V Phillips

Today I am posting four critiques of reports about tobacco.  The two that do not relate directly to THR are on my ep-ology blog, in the Unhealthful News series.  If you read this but not that, you might want to check out that post and also my review of the “Cigarette Wars” television show that I posted the previous two days.

The first study, from a journal called Human Psychopharmacology: Clinical and Experimental, is a good simple example of how bad tobacco-related and pysch research can be.  The study was an experiment on 15 smokers to determine whether consuming “Swedish-style snus” affected their desire to smoke, also looking at the effects of nicotine lozenges.  The suggestion that Swedish-style snus is fundamentally different other smokeless tobacco products is a error I can only stand to bother with  so often (it is a marketing gimmick that may have some value in promoting tobacco harm reduction, but scientists who presume to write about the subject should know better).  So, ignoring that, the most notable absurdity is trying to measure something that will vary drastically across people with only 15 subjects.  Typical psych research.  But what is really funny is that they lost 4 of them to followup.  Yes, they started with only 15 and could still could not even manage to keep 75% of them participating, and yet it is considered worthy of publication in that field.

The second absurdity was doing this as a blinded study with the participants sometimes getting a nicotine-free lozenge or tobacco-free “snus” (yes, they still called it snus even though it was not).  Why?  It might be interesting to ask a question like “how much of the effect of snus in satisfying smokers who are switching to it is due to the nicotine and how much actually still exists in the absence of nicotine?  In that case, this methodology (with a lot more subjects) might be interesting.  But they were not asking that.  Rather, their major claims were:

Findings suggest that [snus] is effective in acutely suppressing craving and smoking in at least some smokers, but that its acceptability may be limited.

In other words, they claimed to be trying to figure out whether snus is a good enough substitute for some smokers.  Since part of the actual experience of snus, as opposed to their experiment, comes from knowing what it is and that it is providing nicotine, they are not actually measuring what they claim to measure.  They are measuring the effects of nicotine in an experimental situation and with participants who are being made anxious about not knowing whether they are getting nicotine.  Here is a news flash for all you psych researchers:  It is possible to trick people in to feeling and doing things that they would not do under natural circumstances.

As for going to all this trouble to figure out that some smokers find snus to be a satisfying substitute, but that many do not like it when they first try it, there are easier research methods.  Like, say, doing a little background reading.

In another study, a group of researchers primarily employed by the American Legacy Foundation, recently published “Prevalence of Trial of Snus Products Among Adult Smokers” in the American Journal of Public Health.  The actual results of the study are little more than a matter of technical curiosity, with no practical implications.  But the authors made a valiant effort to try to misconstrue them as hinting that smokeless tobacco use causes people to not quit smoking, though they showed no such thing.  This is what we would expect from a Legacy study, given their anti-harm-reduction positions, attacking smokeless tobacco products (e.g., this from just the last week or two).  Funny, though, they do not disclose this rather glaring conflict of interest.  Nor do they disclose that all of their money comes from tobacco companies; as in one of the examples in today’s Unhealthful News post, we can observe the remarkable coincidence that complaints about tobacco industry funding never seem to occur when the anti-tobacco extremists like an article’s claims.

The study reported some results from a phone survey conducted in 2008 in several cities where American tobacco companies test marketed their new “snus” products and several cities where they did not.  The focus was on which subjects reported trying one of Taboka, Camel Snus, Skoal Dry, Marlboro Snus, Triumph Snus, Grand Prix Snus, or Tourney Snus.  They polled 4000 smokers, but did not bother to report how many actually reported trying snus (or “trialed” as marketers sometimes call it).  It is possible to figure out that it was about 200.

As anyone could tell you without even reading the article, men and people living in cities where new snus products were test marketed were a lot more likely to have tried than women or those living where the products were not available.  However, it was actually surprising how many had tried the products in other cities (they may have tried some other pouched Skoal product – it is not clear if the survey was designed to minimize that risk).  Also predictable was that younger people were more likely to try.

The reported results were rather a muddle because the women and people in non-test-market cities were included in all of the reported counts.  Since the few such smokers who trialed are so different from the more typical young men in test market cities, including them just creates confusion.  That was just bad reporting, though it is not clear it changed the general lack of interesting results.  Also, the phone survey was primarily an evaluation of one of Legacy’s stop smoking campaigns, so it is possible that participants’ answers that relate to cessation were influenced by them realizing this was what the interviewer was interested in.  On a more technical note, it should be pointed out that the authors’ “throw in every variable we have and call that controlling for confounding” approach is simply bad epidemiology.  Common, but bad.

The result that the authors found most politically promising and tried to play up was that those who plan to quit smoking in the next 30 days were somewhat less likely to trial.  Can I get a “duh!”.  Anyone who is legitimately that far down the road to quitting is not going to be very interested in new options.  And as anyone who has actually talked to smokers about THR knows, one of the common responses is “oh, I don’t need to reduce my risks; I am going to be quitting soon”.  Of course it is not usually true that the smoker does quit soon, but a lot of the time he clearly believes it.  Indeed, in this survey, almost 20% of the respondents said they were going to quit in the next 30 days.  But the authors tried to dredge out something to support their political position from this result:

Does having low motivation to quit smoking lead one to try snus, or does trying snus lower one’s motivation to quit?  ….  …if trial of snus leads to the lowering of motivation to quit smoking, then health risks would increase if the cessation rate were actually reduced by the product.

And what could possibly be the reason that trying snus would cause someone to want to keep smoking?  The authors do not offer any possible reason why, probably because the only one that has ever been proposed by the activists is that smokeless alternatives reduce the suffering that place-specific smoking bans are intended to inflict on smokers, torturing them into quitting.  The reason they probably did not want to admit that is the only proposed hypothesis that could support their little “gee, maybe it could be…I wonder if…” exercise is that their survey found remarkably little greater trialing among smokers who were not allowed to smoke at work or in their home.

This is actually quite surprising given that manufacturers were not allowed to tell people the truth about the low risks from these products, their pitch for why a smoker might want to try these was restricted to pointing out that they can be used anywhere.  Not only does this argue against the political claim that authors attempted to make, but it is further evidence against the underlying hypothesis in general.  Funny they did not think it was worth discussing.  I guess we should just be happy that they did not simply choose to suppress the result.

Also interesting was that those who actually quit smoking between the two rounds of the survey (there was follow-up after about six months) were more likely to have trialed than those who did not quit, though since only 5% of the population actually did quit (over this much-longer-than-30-day period) and only about 11 of them tried snus, we cannot make much of the result.  Of course, 11 subjects was enough for the authors of the previously reported study, but some of us prefer to know a bit more before announcing sweeping conclusions.

Nevertheless, if you want to get some political mileage out of this study, I think it makes a rather better case for the following:

New study by researchers at the anti-tobacco activist organization American Legacy Foundation and Boston University, published in the American Journal of Public Health, found that smokers seldom try smokeless tobacco in order to deal with workplace or household smoking bans, and smokers who are interested in quitting are unlikely to try smokeless tobacco.  This suggests that the promotion and availability of low-risk alternatives to smoking is not interfering with quit attempts or even with policies designed to force smokers to quit.  Moreover, those who did quit smoking over the course of the six months of the study were more likely to have tried new snus products than those who did not manage to quit, and much more likely than those who said they would quit but failed to do so, suggesting that these products are useful in aiding quitting.

Go ahead and use that if you want.  Consider it a press release.

Both comments and trackbacks are currently closed.
%d bloggers like this: