Washington Post doesn’t buy Campaign for Safe Cosmetics scare study

Not every news organization transcribed the “toxic tubs” study put out by the coalition of activist groups called the Campaign for Safe Cosmetics, David Brown of the Washington Post actually decided to see what the scientific research said about the two chemicals, formaldehyde and 1,4-dioxane:

Extensive studies of two toxic chemicals found in children’s bath and personal care products suggest that if they pose a health hazard, it is likely to be extremely small and probably incalculable, a review of scientific research shows.

Despite this opening graph, the copy desk chose to headline the story “Review finds modest risk from children’s toiletries” even though there’s nothing to support this ratcheting up of the threat (although what does  a “modest risk” actually mean?). Brown also added some useful information to our deconstruction of the Campaign for Safe Cosmetics report on the main STATS website.

At the Society of Toxicology’s annual meeting, which will be held in Baltimore next week, a CDC scientist is scheduled to present a study in which blood levels of 1,4-dioxane were measured in about 2,000 Americans 12 and older. No detectable amounts were found. That suggests actual — as opposed to theoretical — exposure to the compound is virtually nil.

The Post piece is a reminder (to, for example, USA Today ) that these activist reports are neither peer-reviewed nor scientific, and cherry-pick data to make the case for their overarching agenda.

One percent of health reporters say health coverage in U.S. is excellent

The Association of Health Care Journalists and the Kaiser Family Foundation conducted a survey and found that only 1 percent of reporters covering health thought that coverage of health in the media was “excellent.” A majority – 51 percent – said that coverage was fair, 34 percent said it was good, while 14 percent rated it as poor. The situation doesn’t look promising as respondents argued that they are getting fewer resources than ever before, and are expected to produce more material in less time. Still, reporters appeared divided as to whether health care journalism in the U.S. is going in the wrong direction – 52 percent said it was going in the right direction, and 67 percent said it was going in the right direction at their organization. In other words, the problem is worse if you look at someone else’s back yard.

The inconsistencies in the results suggest that what is needed to truly answer these questions is a study which tallied the accuracy of health coverage with reporters’ self-evaluations of their accuracy. Indeed, the critical question raised by this study is how these changes in journalism are affecting the quality of reporting. The survey suggested that reporters acknowledged one of STATS major gripes, churning articles out of press releases without due regard to what the study actually says or what experts think of its conclusions:

Nearly nine out of ten (88%) say the media’s coverage of health care leans too much toward short stories and quick hits; and two-thirds (64%) say the situation in that regard has gotten worse in the past few years. Perhaps as a consequence of the time crunch, just under half (44%) of respondents say their organization sometimes (34%) or frequently (10%) reports stories based on news releases without substantial additional reporting (56% say that rarely or never happens).

89 percent said that “time to research” is one of the top two important elements to high quality reporting, but 53 percent said that this had suffered in the past few years. Remarkably, the one area where health reporters think they’re doing a reasonable job – covering medical research  (only 12 percent say that media coverage is poor) – is the area where STATS finds the most concern in terms of statistical and scientific analysis.

Indeed, the study suggests that reporters are more concerned about other health issues, such as health disparities and health care politics, which means that attempts to get more and better quantitative reasoning into the news coverage are unlikely to happen anytime soon. The statistics about health care reporting, in other words, are becoming even scarier than they were when they only applied to the medical studies.

The 5 percent or less chance… how likely is unlikely?

What happens in that five percent margin of 95 percent probability? Two recent articles, one from the Wall Street Journal, the other from the Financial Times explore the statistical problem in near certainty – that the normal measure for statistical confidence, 95 percent, still leaves us with the possibility that the unlikely will occur.

Carl Bialik,  the Wall Street Journal’s Numbers Guy examines three highly improbable events – the collision between two nuclear submarines, two satellites in orbit, and the disaster, averted by dazzling piloting, that occurred when geese flew into both engines of a commercial aircraft over New York. As Bialik notes:

Each of these collisions is the first of its kind, or nearly so. Never have two subs carrying ballistic missiles collided in 50 years of patrols by these war vessels, according to maritime experts. Nor have two satellites collided in the space era. And a search by the National Transportation Safety Board turned up just a handful of cases in the last quarter-century in which more than one engine was disabled by birds in the wrong place at the wrong time. None involved large passenger jets.

Knowing just how improbable these events are can help designers decide whether it’s worth taking steps to prevent recurrence. Yet calculating the probability is a lot harder than beating the house in blackjack.

The problem is that if you wait long enough and keep rolling the dice, so to speak, you’ll encounter the unlikely.

Using the scenario of a confined patrol area, science writers David Curran and Luke McKinney modeled the subs’ routes after the random motion of particles. That rough calculation yielded one collision every 500 years. Allow the subs to roam freely in the Atlantic, and the expected frequency plummets to once every 50,000 years.

Bialik has another great example on his blog:

Jessica Utts, the author of “Seeing Through Statistics,” cautions that events that seem unlikely at first glance may not be that unlikely given enough opportunity. For example, Utts says, if everyone dreamed about a plane crash once in their lives, a few thousand people would have the crash dream on any given night — including the night of a crash. “These specific incidences may be unlikely, but the combined probability of something similar at some point in time is probably fairly high,” Utts, a statistician at the University of California, Irvine, said about the recent collisions.

Stefan Stern looks at the problem from a management perspective in his column in the Financial Times. As he puts it “Bad risk-management helped get us into the current mess. It is vital that we learn the right lessons about risk from the crisis. What are they?” Citing “Six Ways Companies Mismanage Risk,”  an article by René M. Stulz in the current issue of the Harvard Business Review, Stern says that extrapolating from what happened in the past fails to take account not only of the odds of chance happening, but that the odds of chance increase when the present conditions have significantly changed.

…narrow daily measures – in banking these are known as “value at risk” measures – have underestimated the risks that are being run. The assumption behind a daily measure of risk is that action can be taken quickly (through an asset sale) to remove that risk. But, as the current crisis has shown, such rapid moves become impossible when markets seize up.

Stern adds that maturity and experience provide a vital context for risk assessment. The longer you are around,  the greater the probability that you will think more about the downside risks.

Breast cancer alcohol link dismissed

A junk study says Spiked Online: The results were observational, based on self-reported analysis of drinking habits, and were undermined by the absence of a dose-response relationship. To wit, those who didn’t drink at all had a higher proportion of cancers than those that did…