Why Critics of New York Times “Law and Order – Vets in America” Series Are Also Statistically Challenged

Trevor Butterworth

A lot of people have gotten carried away by the suggestive statistics in the New York Times story on veterans and homicide rates, to the point of adding suggestive statistics of their own (see here, here and here).

First of all, comparing the homicide rate of veterans with the homicide rate of the general population aged 18-34 and then noting that the latter is higher than the former (ergo, combat is not predictive of increased PTSD, violence or whatever) is a poor way of conceptualizing an answer to the question of whether exposure to combat is predictive of increased rates of violence in veterans. Here’s why:

Soldiers who experience combat are a unique sample of the population: they are, in the main, highly trained and highly disciplined. And in combat they are exposed to highly unusual stresses.

They are not directly comparable to dentists who commit homicide or don’t commit homicide, or to violent or pacific nuns, or – more to the point – drug dealers, gang members and any other social group where lack of empathy, social circumstances and/or other factors drive high rates of homicide and violence. The general homicide rate in the population tells us very little about who is committing homicides or why. The best it does is show rate by age and race – and no, surprise, young men tend to be at the top of the heap.

One would expect, given the training and discipline of army life (and the fact that the services do a good job of discharging those with “problems”) that soldiers, generally, would have lower homicide rates than the general population, which they do. But simply pointing this out forecloses the key question – does combat, or a certain amount of combat, or even a certain kind of combat lead to the kinds of stress that foreshadow violence? To ask this question is not to disparage the veterans or current service men or women; it is, rather, a way of paying them the respect of recognizing the sacrifices they have made – sacrifices that are unique to them as a group. In other words, what we want to do statistically is compare apples with apples, not apples with oranges.

In order to know whether there is a statistical association between combat and increased post-combat violence or other behaviors, one would have to create a study which compared the homicide rate of veterans who saw combat with that of veterans who didn’t see combat. Ideally, one would also compare the homicide rate of soldiers per hour of experienced combat or sustained fire, or per hour of type of combat, or per numbers of tours of duty to see if there was a strong association between a certain amount of stress and certain kinds of violence, a tipping point so to speak. House-to-house combat is, for instance, different to armored assault; repeated house-to-house combat is different to occasional armored assault; repeated house-to-house combat with high casualties is different to repeated house-to-house combat with low casualties.

This kind of study also has to be longitudinal, as the reaction to combat stress can take a long time to manifest – for instance, in Britain, Combat Stress, which was founded in 1919 to try and deal more humanely with the after effects of “shell shock” than dumping veterans in lunatic asylums, has found spikes in calls during victory anniversaries and that it is beginning to see increased numbers of soldiers who served in Northern Ireland. (This delayed onset is supported by a couple of studies: Bonwick RJ, Morris PLP. Post-traumatic stress disorder in elderly war veterans. Int J Geriatric Psychiatry. 1996;11:1071–1076; Herman N, Enyavec G. Delayed onset PTSD in world war II veterans. Can J Psychiatry. 1994;39:439–441.)

For those who think the idea of PTSD is overblown, one need only look to World War Two, where, contrary to popular imagination, “battle fatigue” was rampant. As Brian H. Chermol (at the time of publication a Lt. Colonel in the U.S. Army) pointed out in his brilliant essay “Wounds without Scars: Treatment of Battle Fatigue in the U.S. Armed Forces in the Second World War,” (Military Affairs, Vol. 49, No. 1., Jan., 1985, pp. 9-12.)

The magnitude of the problem that challenged the military’s medical departments is best exemplified by the campaign in North Africa. In November 1942. the 1st and 3rd Infantry Divisions and the 2nd Armored Division, under Major Generals Patton and Fredendall. landed in North Africa to assist the British in driving Rommel’s forces from the continent. Within a few months the number of soldiers being evacuated due to battle fatigue (i.e. ‘shell shock,’ ‘combat fatigue.’ ‘war neurosis.’ etc.) exceeded the number of replacements who could be shipped to the theater of operations. Cooks. mechanics. and other support personnel were quickly pressed into service as infantrymen. Battle fatigue (BF) to WIA ratios often exceeded 1:2. As the campaign continued. BF casualties continued to mount. Within infantry units, with few exceptions. every soldier involved in the initial landings who was not killed. injured. or diseased eventually became ineffective due to BF.

Chermol also noted a plethora of other factors affected rates of battle fatigue:

Surveys conducted during World War 11 revealed that BF rates covaried with battle losses (WIA and KIA) and with length of exposure to combat; these factors may be viewed as crude measures of fear and fatigue. respectively. Other factors correlated with increased BF rates were the death or injury of friends; the loss of leaders due to BF, death. or injury; lack of confidence in the unit or unit leaders; pessimism regarding the outcome of the conflict or personal survival; the noise and vibration within armored vehicles or aircraft: lack of information; the ‘anticipation of heavy combat action (particularly among aviators); inadequate sleep or nutrition; numerous ‘near’ misses; slow advance against a determined enemy (as occurred in the South Pacific); primitive or isolated living conditions (as occurred in the South Pacific and the Aleutians): and the inability to take personal. aggressive action against an enemy threat, e.g., soldiers were medically discharged. In the Army (to include the Army Air Force), of about one million admissions. some 300.000 received a medical discharge. Four decades later. many of these psychiatric casualties continue to receive treatment through the Veterans Administration system for their post-traumatic stress disorders (though a large percentage had pre-existing psychiatric disorders prior to their induction).

Chermol added that even these numbers under-represent the reality on the battlefield of less extreme forms of battle fatigue. The crucial point is that in its most debilitating form, battle fatigue was correlated with exaggerated aggressiveness, social withdrawal, open fearfulness, and fantastic beliefs – all of which appear to manifest in the behavior of the soldiers portrayed in the New York Times series.

As far as I can ascertain for the purposes of a blog post, we do not have the relevant data to determine whether the recent conflicts in Iraq or Afghanistan have led to high rates of PTSD that, in turn, have led to higher rates of violence and homicide than we would have expected to see in soldiers not exposed to the peculiar circumstances of these conflicts. But if the New York Times over-reached statistically, its critics are, I’m afraid over-reaching too. The homicide rate for the general population may be the only comparative data we have, but that fact alone doesn’t make the comparison meaningful – or more importantly, useful.

The danger with this issue lies not in over-estimating the problem but with dismissing it by virtue of “only having” inadequate measurement. The repeated tours and sustained combat against an elusive enemy, the widely-shared view on the home front that Iraq was a misadventure, are all heavily predictive of PTSD in veterans. The most disturbing thing about the Times piece was not that they fudged the numbers, but that the military – and soldiers themselves – may be fudging this problem. As Chermol concluded in his essay, the military did an outstanding job of dealing with these issues in World War Two; the evidence is less convincing that those lessons have been adequately absorbed and remembered.

About these ads

6 Responses to Why Critics of New York Times “Law and Order – Vets in America” Series Are Also Statistically Challenged

  1. John says:

    While the Times didn’t go the length you rightly suggest they should have in order to make their comparisons apt, they did make efforts, primarily by comparing pre-war crime rates in the military to rates after the war began. In that sense they were much closer to performing apples to apples comparisons than their critics. At least the Times was comparing soldiers to soldiers, rather than soldiers to the general population.

  2. tr3vor56 says:

    Good point, John; and if I didn’t make it clear in my original post, I wish to do so now: I thought the NYT (which normally gets criticized on this site) published an excellent piece of journalism about a very serious issue, whose minor faults are being systematically elevated over its virtues.

    I recall having a conversation several years ago with a psychiatrist at a VA hospital, who was worried by what he was seeing in returning vets. The extraordinary irony of the Chermol essay (available if you have access to JSTOR) is that PTSD, as we now call it, may well have been better appreciated and more comprehensively treated in WW2 than in succeeding wars.

    Trevor

  3. Zubin says:

    Your points are very well taken.

    As one of the critics you link to, my biggest problem with the NYT piece was how poorly the idea that increased instances of PTSD might be leading to more violence was presented.

    For example, the top of the web page the series appears in says:

    “A series of articles and multimedia about veterans of the wars in Iraq and Afghanistan who have committed killings, or been charged with them, after coming home.”

    The key component of the series – PTSD – is missing. A more clear case of this is in the discussion of the increase in killings pre- and post-Iraq and Afghanistan:

    “This showed an 89 percent increase during the present wartime period, to 349 cases from 184, about three-quarters of which involved Iraq and Afghanistan war veterans.”

    If you want to talk about PTSD as a precursor to violence, then why include in the % increase the one-quarter that weren’t Iraq and Afghanistan veterans?

    But still your criticism of statistical methods is very welcome.

  4. Thanks for taking the time to comment. Interesting point about PTSD in non-combat zone vets – I’m curious as to how those apparently non-combat vet numbers break down.

    Of course, the real story is how battle fatigue in all its forms is being measured by the military and/or reported by soldiers. Unfortunately, this is a story that will keep unfolding for the next 50 years.

  5. TD says:

    Mr. Butterworth’s analysis is very reasonable. However, all demographic comparisons – generally of necessity – assume a normative baseline data point of some kind, usually the population as a whole or a stated subcategory thereof. Yes, this assumption is problematic to the extent that most such data points can be shown to be imperfect, i.e. overly broad, etc. Still, demographic comparisons are commonly presented and – most importantly – understood in this manner. If I assert that Irish-American men ages 18-24 are more likely to commit homicide, people will understand that I’m highlighting a significant trend, though strictly speaking one can’t compare Irish-American men with any other U.S. demographic subgroup or the overall population for a host of socioeconomic and other reasons. It’s not truly apples to apples, as it were. It seems to me the Times wants it both ways. That is, the journalists who worked on these stories at first clearly sought to imply that returning veterans were more likely to commit homicide than other members of society. Once that claim was shown to be statistically invalid, they invoked the idea that homicides committed by said returning veterans – both collectively and individually – are tragic. The former implication is manifestly erroneous, which for me is the salient point. It’s a given that murder is a tragedy. I appreciate your analysis, though.

  6. Robert Bateman says:

    Trevor,

    You miss something though, in your analysis as well. The NYT did not just examine active duty, and it didn’t just examine homicides. That 121 number included DUIs (22 which ended fatally), traffic accidents which were non-DUI but which were caused by speeding, 5 cases of self-defense or cases in which the accused were found completely innocent, one case of a cop (reservist) involved (with two other cops) in shooting a victim during the course of what they thought was a crack-house bust, and in only about 15%-20% of all of the cases was it asserted that the person accused/convicted had actually seen any combat or experienced any trauma whatsoever.

    So they did not screen for PTSD. They only made the assertion that service in Iraq/Afghanistan was the qualifier. Frankly, that’s not enough. You don’t get PTSD on a FOB, generally speaking. (Mortuary Affairs, and Docs might be excepted.) You get it out there running the roads, or working in direct ground combat roles. And the NYT didn’t establish that at all, except in their four profiles.

    If, by using the NYT own words, and knocking down to the cases where only those people described as having “seen combat” were included, you’re talking about 45 or maybe 55 people out of 700,000 over the course of six years.

    See?

    They didn’t make the connection to PTSD. They didn’t make the connection to combat (vs. mere “service in a combat zone”), and they didn’t even really significantly raise the issue of VA availablity. Heck, in their second story they made it clear, over and over, that the perp had access, and refused it, and then went back (after the murder).

    It was just lousy journalism, and by missing that Trevor, I am afraid you’re committing the same logical non-sequitor error.

    Bob Bateman

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 35 other followers