Most of my half-dozen readers will by now have heard about the research conducted by Les Roberts, Riyadh Lafta, Richard Garfield, Jamal Khudhairi and Gilbert Burnham, published in the current issue of The Lancet, estimating the number of civilians killed during the war against Iraq (so far). If you have not already done so, please read the paper; it is interesting, and will tell you much more about it than any number of `about the research' pieces in the newspapers or (god forbid) on people's web logs.
... Now you have done so, you will be familiar with the central conclusion of the piece, expressed by its authors as a quiet understatement:
In this case, the lack of precision does not hinder the clear identication of the major public-health problem in Iraq-- violence.
Very briefly, the way the study was conducted was to conduct surveys of clusters of households randomly assigned to bits of Iraq weighted by population. Each survey asked householders to record deaths of members of the household in a period prior to the invasion of Iraq and in a similar period after the invasion. From these data they estimated the death rate before and after the invasion, and from this formed an estimate of the total number of excess deaths from the invasion.
The authors acknowledge that, under current conditions, Iraq is not an easy country in which to conduct such a survey:
During September, 2004, many roads were not under the control of the Government of Iraq or coalition forces. Local police checkpoints were perceived by team members as target identification screens for rebel groups. To lessen risks to investigators,we sought to minimise travel distances and the number of Governorates to visit,while still sampling from all regions of the country.
-- and the survey certainly isn't as accurate as the ideal which could be expected in a country at peace. The headline figure -- of a conservative estimate of 98,000 deaths due to the war -- is the center of a 95% confidence interval which stretches from 8,000 to 194,000 deaths. (Note that this figure does not include information from Fallujah, an area which has suffered more violence than most and in which it was very difficult to do the survey. Including the figured from Fallujah increases the estimate by about 200,000 but broadens the confidence interval further.)
A number of commentators have argued that this wide uncertainty makes the study worthless. This is nonsense, but it is important to understand what it actually means.
Specifically, under the assumptions of the survey, the authors estimate that there is a 95% chance that the two limiting values they obtain enclose the true number of deaths. Loosely, you can turn this around and regard this as a 95% chance that the true value lies in the interval (and hence a 5% chance that the true number lies outside that interval). Further (if we assume a symmetric distribution, which is plausible), there is a 50% chance that the total number of deaths exceeded 98,000, and a 2.5% chance that it was less than 8,000.
To put this into English, it means `we're not sure exactly how many, but a hell of a lot of people almost certainly died'. Make up your own mind, but in my view the low estimates of the number of dead in Iraq should now be regarded with deep suspicion.
There have been several criticisms of this study, most of them rather silly. Tim Lambert and Daniel Davies on Crooked Timber discuss and rebut a variety of complaints. A few more criticisms should, perhaps, be added:
-
``I supported the war, and I don't believe that 100,000 people have died. Therefore the study must be wrong or biased or both.''
-
``100,000 is a lot more than the number of casualties estimated by the Iraq Body Count. Therefore it must be wrong.''
(Hilariously, this criticism seems to come from the same set of people who previously objected to the methodology of the Iraq Body Count people, apparently because they found too many casualties.)
Actually this discrepancy doesn't tell us anything about the relative accuracy of the two studies. The Iraq Body Count is counting civilian deaths reported by the news media, while the Lancet study is estimating the total number of excess deaths caused by the war. This is also the answer to a criticism made by a commenter on Crooked Timber, who wrote,
One criticism that you dont take on here is the observation that 100,000 people in a year and a half is 183 people a day. This would imply that there'd be some days when thousands of extra people died, which would surely have shown up in the news somewhere.
Well, no. That's not how newspapers work. Journalists report events -- they don't go out into the streets to count every dead body, especially when many of those deaths are the indirect consequence of violence. And in the places where violence is going on, there probably aren't a lot of journalists on the streets anyway.
-
``The pre-war infant mortality rate reported by this study is far below that from other statistics; therefore we can't trust this study.''
This one's much more interesting. The Lancet study reported an infant mortality rate of 29 per 1,000 live births (95% confidence interval 0 -- 64), as compared to a UNICEF estimate of 107 per 1,000 live births. (For comparison, the value for the UK in 2003 is 6 per 1,000 births.)
This suggests that the samples they took were not representative of this segment of the population.
Two immediate comments: firstly, the UNICEF figure has no error estimate, and so is difficult to compare to the new estimate; secondly, the old Iraqi régime may have inflated infant mortality figures to support the argument that sanctions were responsible for a large rise in infant mortality. (I can't justify the latter argument -- it's just a hypothesis; in any case, the Coalition Provisional Authority quoted a similar figure, 108 deaths / 1,000 births, in June this year.)
One other possibility: the researchers counted the numbers of births and deaths among the interviewed samples over a fixed interval. This will only give an accurate estimate of the infant mortality rate if the birth rate doesn't vary substantially (a child who dies during the survey period may have been born before it). However, to explain the discrepancy, a large increase in the birth rate in the immediate pre-war period would be required, and there's no reason to suppose that had happened.
This issue, unlike most of the others, is a bit troubling. But the bulk of the deaths estimated by this study occurred among adults, not children, so -- unless this is evidence for a wider problem -- this doesn't affect the basic conclusion; in any case, the survey compared the households' pre- and post-war circumstances using the same methodology, and it is the ratio of pre- and post-war mortality rates from which the estimate for the total numbers of deaths was derived.
Another source of criticism of this research has been 10 Downing Street. Last Friday, the Official Spokesman complained that,
Firstly, the survey appeared to be based on an extrapolation technique rather than a detailed body count. Our worries centred on the fact that the technique in question appeared to treat Iraq as if every area was one and the same. In terms of the level of conflict, that was definitely not the case. Secondly, the survey appeared to assume that bombing had taken place throughout Iraq. Again, that was not true. It had been focussed primarily on areas such as Fallujah. Consequently, we did not believe that extrapolation was an appropriate technique to use.
This is -- I've searched in vain for a politer way to put it -- crap.
Almost all national-level statistics are based on extrapolation (or, more accurately, sampling). Are we supposed to believe that (say) the Census or (for another example) Government research showing public support for ID cards are worthless because they did not count each individual living person in the country?
The implication that no casualty figures could be accurate unless they are derived from a `detailed body count' is also absurd, especially given that Coalition forces have refused to conduct any such research; in any case, a `body count' would severely underestimate the total number killed -- partly because many bodies will not be recovered (for instance, those killed when bombed buildings collapse), and partly because it's now impossible accurately to count the bodies of those who have already died and been buried.
Further, the survey did not treat `Iraq as if every area was one and the same', as even a cursory inspection of the paper will tell the reader. Similarly, the survey did not `assume that bombing had taken place throughout Iraq'; instead, samples were taken at numerous locations in order to account for the geographical distribution of damage (many of the sampled areas were unbombed, as you would expect). Specifically, as I have remarked, the headline number excludes Fallujah, because of the high concentration of bombing and difficulties of conducting the survey there.
And yesterday, the Official Spokesman was at it again, repeating the same false statements and adding some new stuff to confuse the lobby journalists:
Asked to explain further the Government's previous concerns and doubts about the methodology applied in the ``Lancet'' article about the number of Iraqi deaths, the PMOS replied because it relied on the extrapolation technique assumed, that Iraq was uniform in terms of intensity of conflict. It wasn't. The article also assumed that bombing was general throughout Iraq, which was not the case. The Iraqi Department of Health had issued figures that showed over a 6 month period there were about 3,000 deaths, which was a long way short of the figures quoted in the ``Lancet''. The Iraqi DOH measured those figures by the number of people who came into hospitals throughout Iraq, and it was very difficult to rely on any such figures quoted in the ``Lancet'' with any certainty.
So: the Iraqi Department of Health counted the number of people who came into hospitals and then died. And they got a different figure from a survey which attempted to estimate the total number of people who died, whether in or out of hospital. Quelle fucking surprise. It's -- at the risk of pointing out the obvious -- a war zone! People who get hit by a LASER-guided bomb don't have time to go to hospital before they die! Of course the fucking numbers differ.
What's going on here, then? I am not sure, and indeed am having trouble untangling my cynicism about others' competence from my cynicism about others' honesty. But my best guess: part of this is misdirection -- the idea, I think, being to try to convince journalists who may be too lazy to check the story properly that It's Not Quite As Bad As They Think; as for the rest, well, the charitable explanation is that the Downing Street press office doesn't know any statistics and doesn't know anyone who does. The more cynical explanation is that this is simple and shameful dishonesty, a feeble attempt to discredit a study which suggests that the war against Iraq has killed the very people it was supposed to save by the tens of thousands.
This is one of the most disappointing pieces of government information I've seen since -- not coincidentally -- that crap they put out about `weapons of mass destruction'. A sad business, all in all.
Moving on, many supporters of the war have been embarrassed by this study, apparently because they do not understand what a war is. For the avoidance of doubt, the fact that one hundred thousand people may have been killed as a result of the late unpleasantness does not necessarily mean that the war was a mistake or `wrong', any more than the fact that a third to one half that number were killed in a single day in Dresden in February 1945 made the war against the Nazis a mistake or `wrong'. (Of course, the civilians killed in Dresden were killed deliberately, whereas -- charitably -- most of those killed in Iraq have been killed recklessly.)
It may tell us more about the conduct of the war (is it really a good idea to supress an insurgency by dropping five hundred pound bombs on urban areas, even if they are LASER-guided?); but in my opinion it is more telling that many apologists for the present war -- usually the first people to compare Saddam to Hitler and Iraq in 2003-4 to Germany in 1939-45 -- have confused these two issues. In wars, people die, often by mistake and often in large numbers. The case the war apologists have now to make is not about the exact numbers of deaths -- later, we will better know how many died, but for the moment 98,000 is the best estimate available -- but whether the deaths were necessary.
And, frankly, it doesn't look very good so far, does it?
Lastly, as a brief comment on the conduct of the war, I will draw your attention to this film taken from an F16 flying over Fallujah in April. (Don't pay too much attention to the commentary on that page, but the video is linked from there; sadly it's in the proprietary -- and rubbish -- Microsoft `Media Player' format, but `xine' will play it.)
Channel 4 News covered this footage, which shows a pilot guiding a bomb towards a building, presumably under instructions from a forward air controller on the ground. While the bomb is falling, a crowd of people appears in the street adjacent to the building and the pilot asks whether he should direct the bomb onto them. The person on the ground tells him to, and he does.
There is some controversy about whether the people bombed -- from a rough count there are thirty to forty people on the video -- were combatants or not. You can read about that in the sites I've linked to. That's not what I want to draw your attention to, though. Here are some frame grabs from the video (lousy quality, sorry). First, a general view from before the crowd emerges, to give a sense of scale:
Note cross-hair, showing the original target of the bomb, designated by a LASER shining from the 'plane onto the ground. The pointing of the LASER can be changed as the bomb is falling, allowing the target of the bomb to be altered. The street in the center of the picture is quite wide -- about 40 meters, judging from the size of the people in the second picture, and it is surrounded by two- and three-storey buildings.
Here is the crowd of people. By this stage the pilot of the 'plane is steering the bomb towards them. For those who can't play the video, the dialogue between the pilot and ground controller ran,
Pilot: I got numerous individuals on the road. Do you want me to take those out?
(As an aside, remember that the controller on the ground may have a much better view of proceedings than the pilot -- these people may have been running towards his position shooting at him, for instance, and we would not be able to tell that from the video.)
Note the counter at the lower right of the picture, reading `000:11' -- that is the time before the bomb hits the ground.
And here is the cloud of debris about a second after the explosion.
Bearing in mind the width of the street and the nature of the surrounding area, consider this:
Most everything will be severely damaged, injured, destroyed, or killed within 20 meters of a 500-pound bomb blast....
Safe distances for unprotected troops are approximately 1,000 meters for 2000-pound bombs and 500 meters for 500-pound ones. Even protected troops are not entirely safe within 240 meters of a 2,000-pound bomb or 220 meters of a 500-pound bomb.
During the war proper -- before the `Mission Accomplished' stunt -- the Americans dropped about 18,000 guided bombs. (This ignores a further 10,000 or so `dumb' bombs.) As a brief estimation exercise, if all of those were dropped in urban areas with a population density of 5,000/km² (probably an underestimate), then for a uniform distribution of targets we'd expect around 100,000 casualties from bombs with a 20-meter lethal radius. That's certainly an overestimate (even the US Air Force no longer bombs at random) but the order of magnitude is... suggestive.
Comments
Posted by Roy Badami, Tuesday, 2 November 2004 19:51 (link):
the authors estimate that there is a 95% chance that the two limiting values they obtain enclose the true number of deaths. Loosely, you can turn this around and regard this as a 95% chance that the true value lies in the interval
I think both your statements are equivalent. Denote the limiting values as Dmin and Dmax, and the actual (unknown) value as Dtrue.
Then both your statements above boil down to there is a 95% probability that Dmin<=Dtrue<=Dmax
-roy
Posted by Jim Snow, Thursday, 4 November 2004 01:59 (link):
Both the original statement and your symbolic expression are correct. The reason the second sentence begins with "loosely" is that it has the form of a probability statement about a fixed (though unknown) constant. A strict definition of probabilty means that the probability of a constant being equal to any particular value is either zero or one.
Posted by Chris Lightfoot, Thursday, 4 November 2004 10:07 (link):
Exactly. Of course, rather than `strict' and `loose' we could use `frequentist' and `Bayesian', but this isn't the context for that debate, I think.
Posted by Alastair, Tuesday, 2 November 2004 21:29 (link):
Further (if we assume a symmetric distribution, which is plausible), there is a 50% chance that the total number of deaths exceeded 98,000, and a 2.5% chance that it was less than 8,000.
Err, one of those figures is wrong. Either it's a 50% chance GIVEN the figure occurs outside the confidence level, or it's a 2.5% chance overall.
Posted by Jim Snow, Thursday, 4 November 2004 02:15 (link):
No. If we use the approach of the sentence beginning with "loosely" , there is a 2.5% chance that the true number of deaths is less than 8000;there is a 47.5% chance that the true number is between 8000 and 98000 ;there is a 47.5% chance that the true value is between 98000 and 194000; and a 2.5% chance that the true number is greater than 194000. As I wrote in my reply to the previous post, this is a loose use of the word probability. A more pedantic statement is ,for example, that the interval from 8000 to 98000 has a 47.5% chance of including the true value.
Posted by Dan, Wednesday, 3 November 2004 20:33 (link):
If you want more background on pre-war mortality figures, have a look through the unicef reports at http://www.casi.org.uk/info/themes.html#hum. You'll also find a couple of reports by Richard Garfield (one of the authors of the lancet report) on mortality in the late 90s
On the iraqi department of health figures the PMOS mentions, I don't think their data was released to the public, but some news agency got hold of it a month or so ago - see http://www.realcities.com/mld/krwashington/9753603.htm
There's another take on the report, and in particular on the first comment from the PMOS, at http://www.casi.org.uk/analysis/2004/msg00480.html
Posted by David Handelman, Thursday, 4 November 2004 17:47 (link):
Several reasons why the "pre-war" infant mortality rate reported in the UNICEF study may not conflict much or at all with the Johns Hopkins study. First, the figure of 108/1000 refers only to south and central Iraq (i.e. outside autonomous Kurdistan); the corresponding figure in northern Iraq was 59/1000, resulting in an overall figure of around 90/1000; meanwhile, the Johns Hopkins figure of 29/1000 refers to all of Iraq.
But more importantly, the UNICEF figures refer to the period from Feb/March 1994 to Feb/March 1999, when the study was conducted. The oil-for-food program did not start until 1996, and was not fully ramped up until 1999. As far as I can tell, there was no study of the impact of the oil-for-food program on infant mortality (unless you count the new Johns Hopkins study), although a 2002 report found that child malnourishment had been cut roughly in half.
In other words, the comparison is not between a pre-war mortality rate of 108/1000 and a comparable rate of 29/1000, but of a mostly-pre-oil-for-food rate of 90/1000 and a post-oil-for-food rate of 29/1000.
Posted by Chris Lightfoot, Thursday, 4 November 2004 11:51 (link):
A quick update: Stephen Soldz has written a piece about this study which makes another important point. Read the whole thing, but here is a relevant excerpt:
The point being that the northern part of Iraq, under control of the Kurdistan Regional Government, hasn't been the scene of significant fighting in the present war, nor has it suffered much from the insurgency. The death rate in Kurdistan has decreased, rather than increased, since the war; this depresses the estimate of casualties in the whole of Iraq. (See, for instance, this rather optimistic article on Kurdistan from the International Herald Tribune.)
Posted by Dan Hardie, Thursday, 4 November 2004 19:27 (link):
Anybody know how to run a photo enhancement of that footage? It would be very useful to know if those people were carrying weapons or not. Certainly they are moving slowly, not using cover, and are clustered together, none of which being what one would expect from fighters in a street battle.
Posted by Chris Lightfoot, Thursday, 4 November 2004 19:44 (link):
`Useful' is understating it! Unfortunately the video that I can find on the web is encoded at too low a resolution and bitrate to answer that question satisfactorily. Whatever enhancement is done to it, I don't think the information is there.
The original pictures must have been much better quality (otherwise all the text and graphics around the edge of the screen would be unreadable to the pilot). But I don't whether a better-quality copy is available anywhere (certainly, I couldn't find one).
For what it's worth, I can't see any evidence for the people in the picture carrying rifles or any other kind of long-barreled weapons (e.g. RPG launchers or whatever), but they would only be visible if held horizontally, and perhaps not even then.
I don't know enough about military tactics to comment intelligently on the way the people in the picture are moving, but certainly the Channel 4 News story raised the same concerns that you do. A second report states,
Three questions from me as a nonexpert:
I'd be interested to hear any answers from those better informed than I.
Posted by Dan Hardie, Thursday, 4 November 2004 22:48 (link):
Yes, if the USMC spokesman said they were firing, that's balls. They cluster together in a group of thirty, then walk slowly down the road and get zapped. Western armies teach 'fire and movement'- basically one fraction of guys in every group hits the dirt and fires at the enemy, another fraction runs towards the enemy, they swap over and repeat the process. And if you're just firing and not moving, you're going to take cover- particularly if the people you're shooting at are US Marines. Just maybe this was a poorly-trained but armed rabble mooching down the road. But no way were they firing at anything. You either fire from a stationary position, preferably in cover, or you fire whilst moving quickly, and these people were just bimbling.
Posted by Don, Friday, 26 May 2006 17:04 (link):
Pffft to Dan Hardie's comment near the top of the page. The US Marines aren't regarded as particularly good troops outside the US and the BBC journalist John Simpson made an interesting comparison with British troops. While the Brits were steady under pressure and willing to investigate things up close and put themselves at risk, the US Marines would panic and open fire wildly if they heard a mouse rustling away on a crisp bag! A bunch of gunmen would much rather face a jittery USMC patrol than say a bunch of Israeli Paras or Royal marines.
Posted by Dan Hardie, Thursday, 4 November 2004 23:06 (link):
I won't call this definite until I've seen enhanced photos, but I've blown the video up to full screen and I can't see anything looking like a weapon. An AK47 protrudes about a foot from the body when held with two hands, and would almost certainly be visible if slung across someone's shoulder. And the posture of the arms would be different too- if there was fighting on then most of them would have the weapon held in both hands, so as to bring it up to fire easily. The weapon is dark-coloured, particularly the magazine, and you'd expect it to be visible given that some of them have light-coloured clothes on. Not only that but they move slowly in a big group down the middle of the road- not even sticking to the sidewalks. Okay, as the Pentagon spokesman says, untrained fighters will make 'tactical errors', but if they were fighters why the hell would they do that?
Posted by Heiko, Saturday, 6 November 2004 21:08 (link):
Could you have a look at my comments below? http://cgi.cse.unsw.edu.au/~lambert/cgi-bin/blog/science/LancetIraq/brignell.html
As I say there (and you say yourself), most of the criticisms of the Lancet study you read in blogs are pretty rubbishy.
However, that doesn't validate the claimed results of the Lancet study, and I think my own criticisms have, so far, not been shown to be erroneous. But, I'd certainly appreciate a thorough counter-critique.
Posted by Chris Lightfoot, Sunday, 7 November 2004 00:26 (link):
OK, the substantive points you make on Tim Lambert's page are:
- Differences in the infant mortality rate found by this study, and found by UNICEF.
- Various concerns about the reliability of self-reporting, and about whether confirmation of deaths was sought.
- Differences in aggregate death rate between this study and data from WHO.
- The nature of violent deaths in the areas outside Fallujah is not broken down in sufficient detail to support the conclusion that coalition bombing was the predominant cause of violent death, and this conclusion is therefore reliant on the Fallujah data. Further, the majority of violent deaths [13 of 21 --CL] were among men of fighting age. The conclusion that most casualties are women and children and most of those casualties were caused by coalition bombing cannot be supported without using the Fallujah data.
- after the invasion violence was the primary cause of death;
- violent deaths were... mainly attributed to coalition forces;
- most individuals reportedly killed by coalition forces were women and children;
- violence accounted for most of the excess deaths; and
- air strikes from coalition forces accounted for most violent deaths.
- most excess mortality was due to coalition bombing; and
- most victims were women and children.
You are also skeptical about conclusions drawn from small numbers of deaths among certain categories. The sample sizes are small, but I see no reason to disbelieve the reported confidence intervals (which are, after all, pretty wide). Reporting of cause of death may well be inaccurate, but it's a bit hard to tell because there's no other research against which to compare it. In any case I agree that the cause-of-death data is likely to suffer from accuracy problems which the mortality rate data doesn't. (In particular I would guess -- without any evidence, mind -- that Iraqis may prefer blame coalition forces than others for violent deaths.)On this, see David Handelman's comment (above). The UNICEF study excluded Kurdistan, whereas Roberts et al. include that area. This will depress the mortality rate in the present work, relative to previously reported data.
I'm not really sure what your point is here. You say, for instance,
Well, yes, that's fine. That's why they sample randomly, and that variability is accounted for in the error estimates. The authors write, (page 6, in their discussion of the difficulty of sampling in Falluja)
and this is an issue I allude to at the end of my piece.
Same issue as the UNICEF data, since the 8/1,000/year death rate figure comes from the same UNICEF study.
The authors claim,
whereas you contend that they claim,
This is not the same. Specifically, most excess mortality was caused by violence in general, not coalition bombing; and women and children make up the majority of victims of all coalition violence, not bombing specifically.
(Excluding the Fallujah data, the crude mortality rates for women and children rise from 3.4/1,000/year to 3.8/1,000/year and 3.6/1,000/year to 8.0/1,000/year. The former is unlikely to be significant given the accuracy of the study; the latter probably is. For men, the rise is from 1.5/1,000/year to 10/1,000/year. Crudely, this indicates that the largest number of deaths caused by the war was among men of fighting age. I agree that it is disappointing that the paper lacks a detailed table of the reported causes of deaths.)
Posted by Heiko, Sunday, 7 November 2004 19:15 (link):
The numbers for infant mortality and overall mortality are for all of Iraq and for the year 2002. WHO and Unicef say that very clearly. What they do not say terribly clearly is how exactly they arrived at those estimates (what I think they did from reading their reports was to take all the available information, put it into models they've developed and verified for other countries and time periods and have come up with an estimate they believe is accurate to roughly plus minus 20%).
Now look at your first sentence on the subject: "Most of my half-dozen readers will by now have heard about the research conducted by Les Roberts, Riyadh Lafta, Richard Garfield, Jamal Khudhairi and Gilbert Burnham, published in the current issue of The Lancet, estimating the number of civilians killed during the war against Iraq (so far)."
Is it all that unreasonable to believe that the authors wanted to give the impression to the media that 100,000 innocent civilians got slaughtered by US bombing?
And if that's what an independently minded person hears (+ an author got away with a demand that peer review be sped up to make sure the study would come out right before the election), isn't there some reason to feel skeptical?
Posted by Chris Lightfoot, Sunday, 7 November 2004 19:51 (link):
Oh? I can't find that in the reference given, which is UNICEF's The State of the World's Children, 2003. It makes no statement about how the data were acquired, beyond that infant mortality data are from UNICEF. I guess that the data come from UNICEF's The Situation of Children in Iraq, which states,
The figures for `1995-99' are the latest quoted in that report, and the methodology is not explained. However, the numbers match and the reference to `South/Centre Iraq' does indeed suggest that Kurdistan was excluded.
As for,
you are imputing to the authors of the paper and the editors of the Lancet motives which I cannot judge. As I have pointed out above, the authors do not state that 100,000 `innocent' `civilians' were `slaughtered' by `US bombing' (even if you remove the value judgments from your statement).
I wonder whether you have any evidence that the author, editors of the Lancet, and reviewers conspired to have the paper published immediately before the US election?
Posted by Heiko, Sunday, 7 November 2004 22:12 (link):
http://www.who.int/disasters/stats/detail.cfm?indicatortypeID=139&countryID=28
http://www.unicef.org/sowc03/tables/table1.html (that's where you go, when you click link to source and choose basic indicators, ie the first table)
That leads you to numbers for 2001 and all of Iraq (infant mortality of 107).
http://www.unicef.org/infobycountry/iraq.html
And here under statistics you'll find the number for 2002 and all of Iraq (infant mortality of 102).
http://www.who.int/whr/2003/annex/en/
http://www.who.int/entity/whr/2003/en/AnnexNotes.pdf
Sorry to quote at length, but I think this section is rather important: "To assess overall levels of health achievement, it is crucial to develop the best possible assessment of the life table for each country. New life tables have been developed for all 192 Member States starting with a systematic review of all available evidence from surveys, censuses, sample registration systems, population laboratories and vital registration on levels and trends in child mortality and adult mortality (1). This review benefited greatly from a collaborative assessment of child mortality levels for 2001 by WHO and UNICEF and from analyses of general mortality by the United States Census Bureau (2) and the United Nations Population Division (3). All estimates of population size and structure for 2002 are based on the 2002 demographic assessments prepared by the United Nations Population Division (3). These estimates refer to the de facto population and not the de jure population in each Member State. To aid in demographic, cause-of-death and burden-of-disease analyses, the 192 Member States have been divided into five mortality strata on the basis of their level of child and adult male mortality. The matrix defined by the six WHO regions and the five mortality strata leads to 14 subregions, since not every mortality stratum is represented in every region. These subregions are defined on pages 184 185 and used in Tables 2 and 3 for presentation of results. Because of increasing heterogeneity of patterns of adult and child mortality, WHO has developed a model life table system of two-parameter logit life tables using a global standard, and with additional age-specific parameters to correct for systematic biases in the application of a two-parameter system (4). This system of model life tables has been used extensively in the development of life tables for those Member States without adequate vital registration and in projecting life tables to 2002 when the most recent data available are from earlier years. Demographic techniques (Preston Coale method, Brass Growth Balance method, Generalized Growth Balance method and Bennett Horiuchi method) have been applied, as appropriate, to assess the level of completeness of recorded mortality data for Member States with vital registration systems. For Member States without national vital registration systems, all available survey, census and vital registration data were assessed, adjusted and averaged to estimate the probable trend in child mortality over the past few decades. This trend was projected to estimate child mortality levels in 2002. In addition, adult sibling survival data from available population surveys were analysed to obtain additional information on adult mortality. WHO uses a standard method to estimate and project life tables for all Member States with comparable data. This may lead to minor differences compared with official life tables prepared by Member States. Life expectancies for the year 2002 for many Member States have been revised from those published for 2000 and 2001 in The World Health Report 2002 to take into account more recently available mortality data. To capture the uncertainty resulting from sampling, indirect estimation technique or projection to 2002, a total of 1000 life tables have been developed for each Member State. Uncertainty bounds are reported in Annex Table 1 by giving key life table values at the 2.5th percentile and the 97.5th percentile. This uncertainty analysis was facilitated by the development of new methods and software tools (5). In countries with a substantial HIV/AIDS epidemic, recent estimates of the level and uncertainty range of the magnitude of the epidemic have been incorporated into the life table uncertainty analysis."
Look at the table, and note that under 5 mortality in 2002 for boys is given as 119 and for girls as 110. The confidence intervals given are 101-139 and 94-127 respectively.
http://news.bbc.co.uk/1/hi/world/middle_east/3964311.stm
<<>>
http://www.techcentralstation.com/102904J.html "About 100,000 Iraqi civilians -- half of them women and children -- have died in Iraq since the invasion, mostly as a result of airstrikes by coalition forces, according to the first reliable study of the death toll from Iraqi and US public health experts." (that's a quote from the Guardian)
You say: "As I have pointed out above, the authors do not state that 100,000 `innocent' `civilians' were `slaughtered' by `US bombing'".
So what do they state?
"We estimate that 98 000 more deaths than expected (8000 - 194 000) happened after the invasion outside of Falluja and far more if the outlier Falluja cluster is included. ... Most individuals reportedly killed by coalition forces were women and children.... Making conservative assumptions, we think that about 100 000 excess deaths, or more have happened since the 2003 invasion of Iraq. Violence accounted for most of the excess deaths and air strikes from coalition forces accounted for most violent deaths."
http://www.denverpost.com/Stories/0,1413,36%257E6439%257E2498563,00.html
<<< The report was released just days before the U.S. presidential election, and the lead researcher said he wanted it that way.
The Lancet routinely publishes papers on the Web before they appear in print, particularly if it considers the findings of urgent public health interest.
Les Roberts, lead researcher from Johns Hopkins, said the article's timing was up to him.
"I e-mailed it in on Sept. 30 under the condition that it came out before the election," Roberts told The Associated Press.
"My motive in doing that was not to skew the election," he said. "My motive was that if this came out during the campaign, both candidates would be forced to pledge to protect civilian lives in Iraq.
"I was opposed to the war, and I still think that the war was a bad idea, but I think that our science has transcended our perspectives. As an American, I am really, really sorry to be reporting this." >>>
Hope this clears things up somewhat.
I think there is good evidence that certain kinds of violence have gone up: violent crime, insurgent/terrorist combatants getting killed by the coalition, civilians being killed by insurgent/terrorists bombings and coalition bombings.
Others, like torture and murder of Shiite prisoners in Saddam's prison, have likely gone down.
As for infant mortality and overall mortality, I think the evidence so far is murky, but the numbers under Saddam were so bad compared to neighbouring countries that it won't be hard to improve on them.
Where the violence is concerned, it does matter who does the killing and who is getting killed. Terrorists being killed by coalition forces is hardly something anybody would regard as something negative. US forces killing innocent civilians is rather different.
So, if the true numbers were (the uncertainty is massive, for the moment take these numbers as illustrative) 2500 civilians killed by the coalition, 25,000 insurgent/terrorists killed by the coalition, 10,000 killed through ordinary crime and 10,000 innocents/Iraqi forces by the insurgency/terrorists,
we have a rather different picture than 100,000 civilians killed by coalition bombing.
And the Lancet study leaves ample room for my sample numbers to be true.
Posted by Chris Lightfoot, Monday, 8 November 2004 00:41 (link):
Posted by Heiko Gerhauser, Monday, 8 November 2004 10:28 (link):
I am a research fellow at the University of Aston in Birmingham (UK), 30 years old, of German nationality but have been living in the UK for about 10 years.
http://www.aston-berg.co.uk/
Sorry about my html skills, maybe I should do a course sometime.
On your point 2:
What I wanted to show you is the tables on Unicef's website for infant mortality in Iraq in the years 2001 and 2002. Nowhere does Unicef here in any way indicate that those two figures (107 for 2001 and 102 for 2002) are for Iraq ex Kurdistan. These tables have been prepared for all the world's countries, it would make no sense whatsoever to quote figures for countries and to actually only be referring to parts of the country.
Unicef did a very extensive study in 99, and they did that in both the parts of Iraq controlled by Saddam and the Kurdish North. They've got lots of other data available, eg surveys on malnutrition, or sanitation, and they can use these data to estimate infant mortality.
There are several important points made in the long section I quoted on methodology:
1. If Unicef/WHO don't have census data: "all available survey, census and vital registration data were assessed, adjusted and averaged to estimate the probable trend in child mortality over the past few decades. This trend was projected to estimate child mortality levels in 2002."
2. "Uncertainty bounds are reported in Annex Table 1 by giving key life table values at the 2.5th percentile and the 97.5th percentile"
I didn't quote the section to prove that the numbers on Unicef's website are for all of Iraq.
http://www.unicef.org/infobycountry/iraq.html
Going back to Unicef's tables, how can you conclude from that webpage that the number is for Iraq ex Kurdistant? It very clearly says Iraq here.
On your point 3:
I don't understand what's so difficult to understand about a sentence like "I e-mailed it in on Sept. 30 under the condition that it came out before the election,"
The election was one month later, peer review at a prestigious journal usually takes six months, if not longer.
On your point 4:
The sample numbers are there to illustrate a different distribution of deaths that is entirely compatible with the Lancet study's raw numbers ex Fallujah. They don't split those in sufficient detail, but take each reported death as representing 3500 real deaths or thereabouts, and look at the gender split ex Fallujah, and the reported number of deaths from crime (though we aren't told how many of those are outside Fallujah), and allow for large uncertainty, and you can pick any number between about 1000 and 40,000 for civilians killed by the coalition, and any number between about 5000 and 50,000 for killed combatants, and anything between about a 1000 and 30,000 for people killed by ordinary crime (or thereabouts).
But the numbers of reported deaths in each category are so low that you can't really conclude very much at all.
Posted by dsquared, Monday, 8 November 2004 11:13 (link):
Heiko, what has happened here is that UNICEF carried out the work between 1995 and 1999 and have used statistical methods to project the child death rates forward. However, they have not done the detailed work that would be needed to assess the impact of oil-for-food, so there is no reason at all to regard them as more definitive for the 2002 death rate than a study carried out on the ground.
Posted by Heiko Gerhauser, Monday, 8 November 2004 13:00 (link):
Hi dsquarend,
it's good that we agree, sort of, what Unicef has done.
My understanding, however, is that they have taken current information into account,
like for example data on malnutrition and sanitation, and have put that into their modelling, which, if I understand them correctly, is well validated based on other countries and time periods.
In particular, note that the figures for child mortality are given with confidence intervals.
They say that they are 97.5% sure that under five mortality of boys in 2002 was at least 101.
By the way, have you followed the discussion at obsidian: http://obsidianorder.blogspot.com/2004/10/pick-number-any-number.html#comments
It (the Lancet paper) is a study on the ground, but that doesn't mean that there is no scope for all sorts of errors, such
as those introduced by only sampling households (leaving out the prison population for example)
as those that might be introduced by selective recall, or changing household composition
people being lied to
To name just a few.
It's a small study and their confidence interval of 8000 to 200,000 or so does not imply that we know with 97.5% certainty that there have been at least 8000 excess deaths.
It's pretty much certain that certain kinds of violence have gone up, but that may be balanced by lower infant mortality, lower violence in prisons and better health care due to greater staff remuneration, wider availability of back-up generators for hospitals and water treatment plants etc. and therefore reduced mortality from for example water borne disease.
The evidence base is poor at the moment. We do know, however, that there was plenty of scope for improvement from the state of affairs we had under Saddam.
Regards, Heiko
Posted by Chris Lightfoot, Monday, 8 November 2004 14:48 (link):
You wrote,
Firstly, you are comparing under-five mortality to infant mortality. Secondly you appear to be holding the two pieces of research to different standards of rigour (or, at least, different interpretations of what a confidence interval is), without giving a reason to do so.
On the remainder of your comment, well, it's obviously true that there are various potential sources of bias and error in the Roberts et al. study; indeed, many of them are discussed in the paper itself! But I haven't yet seen a good reason to disbelieve its basic conclusion, or that the confidence intervals etc. are sensible and correctly reported. As I wrote in my original piece, the take-home message from the paper is that,
I have not yet seen any cause to alter this conclusion.
Posted by Heiko Gerhauser, Monday, 8 November 2004 15:42 (link):
I quoted the number for under 5 mortality, because they've got some measure of the accuracy of the estimate with it.
I would expect the same to apply to infant mortality, but I haven't seen it published. Which means I'd have to write to Unicef and hope for an answer.
However, most under 5 mortality is infant mortality, so I don't see the need from that perspective.
The confidence interval in the Lancet study cannot account for systematic problems (such as say not including the prison population, or army barracks, or people lying to them, or not properly accounting for recall bias and changes in household composition).
I don't know enough about Unicef methodology to be clear what exactly they mean with their measure of certainty, ie what potential sources of error are included and how. I know from the procedure employed by the IPCC (another intergovernmental body, this one concerned with climate change) that it is not uncommon to employ expert judgement for determining measures of probability, but really I'd have to ask them how they've done it here.
While I don't know exactly how Unicef and WHO have arrived at their numbers, I do think it is clear that there is a real difference here between their work and the Lancet study.
Now, suppose, the same team of researchers had been pro-war, and had published only the infant mortality and overall mortality data after the war (the choice of period to investigate after all was the researchers'), and come out with a conclusion that 80,000 children had been saved compared to Unicef estimates of pre-war mortality, while likely 25,000 combatants had been killed by the coalition and the number of civilians killed was so small that the methodology couldn't measure them accurately enough,
what would you have written in your blog?
They could have used the same methodology, in fact the same data (except choosing a shorter time period for study), to come to that completely different conclusion.
Posted by John Simons (Managing Editor, Population Studies), Tuesday, 9 November 2004 10:00 (link):
Estimates based on the most advanced use to date of data from the 1999 UNICEF survey, together with details of methods used and the information needed to evaluate the estimates, appear in the following paper: Ali, Mohamed M., John Blacker, and Gareth Jones. 2003. 'Annual mortality rates and excess deaths of children under five in Iraq, 1991-98'. POPULATION STUDIES 57(2): 217-226. (Ali is at WHO, Jones is at UNICEF, and Blacker is at the London School of Hygiene and Tropical Medicine.)
Confidence intervals for the infant mortality rates are shown in Figures 1 and 2. Incidentally, an error occurred during the production of Figure 1(b) showing the trend of the under-five mortality rate: the y-axis should run from 50 to 270 (like the y-axis in Figure 2(b)) not from 30 to 130 as printed. This does not affect the estimates shown in Table 1. An erratum will appear in the next issue.
JS
Posted by Chris Lightfoot, Thursday, 11 November 2004 18:37 (link):
Right, sorry about the delay in getting back to this. In the end I'm fairly satisfied that the improvement in child nutrition is enough to explain the differences between the 1995-99 infant mortality numbers (even as extrapolated in, e.g., the Ali et al. paper (a preprint is available on London School of Health and Tropical Medicine site). Various indicators of childhood malnutrition in Iraq, (see, for instance figure 2.3 in the Proceedings of CASI's 1999 conference on the sanctions) show falls of one third to two thirds over the period of the oil-for-food programme up to 1999; across countries, indicators of malnourishment are as you would expect well-correlated with infant mortality rates (for instance, see this plot based on data from the UN Millenium Indicators; here, R² ~ 0.4, p-value < 0.001) and so the increasing availability of food could be expected to lead to a -- roughly -- proportional fall in infant mortality; in this case from ~110/1,000 births to ~50±20/1,000 births. By comparison, the Ali et al. paper on which the higher estimates are based assumed that, (from the paper's abstract)
i.e., it's a straight extrapolation based on historical trends (and tables of expected mortality). It doesn't take account of specific changes in malnourishment etc.; nor does it take account of more recent data.
As for,
well, you couldn't conclude very much from such a study (wildly different samples and methodology for the pre- and post-war cases, and covering a period much longer than the war). But if you're going to postulate a hypothetical, why don't you consider the same study, but a different war -- specifically, one in which there were fewer civilian casualties. As has been remarked elsewhere, the striking thing about this study is that the confidence interval for the number of excess deaths does not include zero-- that is, it's very unlikely that the war has saved lives overall so far.
Posted by Heiko Gerhauser, Saturday, 13 November 2004 01:06 (link):
I am aware of the fall in malnutrition, it's in the report by Unicef. And in that report they dismiss all the positive things that did happen in Iraq before the invasion and say that it was merely able to prevent further deterioration. They also rather emphasise other causes of infant mortality besides malnutrition, such as poor sanitation.
I don't think the scatter plot you cite can be taken as evidence that a substantial decline in some measures of malnutrition would yield a two thirds fall within three years (in some countries infant mortality is 100 and malnourishment up at 12, while in others for the same infant mortality it's at 2), or that if this was vaguely credible Unicef and WHO, being well aware of the figures for malnutrition, would publish figures for infant and under 5 mortality that are this seriously removed from reality.
I've asked Unicef now to defend their 2002 and 2001 figures and will await what they come back with.
Remember Unicef and WHO publish figures for every country in the world, and at least for under 5 mortality with confidence intervals. For them to publish figures that are wrong by over a factor three, when they give a confidence interval of +/-20% for under 5 mortality, is something I find very hard to swallow (not to mention, the scatter plot you refer to relies on Unicef/WHO data and would be rather dubious, if those are readily wrong by over a factor 3, after all if they are that wrong for Iraq, you'd seriously have to start questioning data for other poor developing countries - let me be clear, if the source is WHO/Unicef, I believe the data in the scatter plot, I don't believe they show a straightforward relationship between malnutrition and infant mortality, that there is a strong correlation is hardly surprising).
I don't accept your argument about (wildly different samples and methodology for the pre- and post-war cases, and covering a period much longer than the war), they are both measuring the rate for Iraq and the year 2002 is virtually the same time period as the pre-war period looked at in the Lancet study. If the sample is representative of Iraq, we can legitimately compare, if the Lancet sample is not representative, they aren't measuring the value for Iraq.
As long as the pre-war number of 102 is accurate and a study estimates a post-war number of 57 and is accurate, of course, we can compare those two and come up with a number of children saved. If the post-war number isn't accurate, why believe any of the other figures?
I readily believe 30,000 or 60,000 violent deaths in Iraq after the overthrow of Saddam, which is in line with other ways of estimating this number. What I do not believe is that the majority of those were innocent civilians killed by coalition bombing, I believe the coalition and Iraqi government's official pronouncements that most of those killed by them were combatants (and substantial civilian carnage is due to insurgent/terrorist/criminal activity).
I am all in favour of a census and other research to provide good statistics. I don't think military forces can do exact body counts split into civilian/combatants. They can give you estimates of the number killed and that they believe the vast majority to be combatants. That they are doing (and providing exact numbers for their own soldiers, which is easy, they've got good records and can easily identify who is one of their soldiers and who is not).
And the data in the Lancet study do not contradict their general assessment in my opinion. The data for Fallujah are unreliable (how many death certificates do you think did they ask for?, do you realise what proportion of the population of Fallujah would have died if their point estimate for that place was accurate?), and in the rest of Iraq, they've got a very small number of reported/claimed deaths that could be civilians killed by coalition forces. One person lying and making up 2 violent child deaths and 1 violent woman adult death outside of Fallujah would literally double the number of violent child and woman deaths.
100,000 innocent civilians killed by coalition bombing is what is widely being perceived as what this study is reporting, not 60,000 deaths from violence and some increase in overall mortality driven by infant mortality (where I dispute that we've got good enough data to make a judgement on infant mortality or overall mortality, and where deaths from violence are concerned, I think it's clear that several ten thousand have died, but what's in dispute is who was killed and by whom).
Posted by Chris Lightfoot, Saturday, 13 November 2004 11:41 (link):
Among other things, you're mixing up arguments about how the study has been reported with arguments about what the study actually says. It's true that a lot of reporting of this study has been of pretty poor quality (on both sides of the argument), but that's not relevant to the study itself.
I've given the reasons why I don't believe the difference between the UNICEF 2002 figures for infant mortality and those in this study is relevant; you're correct that the claim can't be deduced from the plot I refer to, but this was merely to illustrate the (obvious, to be honest) point that infant mortality is well correlated with rates of infant malnourishment. This will be especially so in a country like Iraq where most infant deaths were caused by malnourishment. (I was not able to find time-series data to test the hypothesis that a sharp fall in infant malnourishment would lead to a similarly sharp fall in infant mortality; however, that seems a reasonable assumption in the first instance.)
Your argument about the UNICEF figures seems to be purely from authority. I'd be more interested in a defence of their methodology, which does not take into account the improvement of the food situation in Iraq over the period of the study. It's all very well for you to say,
but you don't say which report! It specifically isn't in the Ali et al. paper which derives the 2002 estimates of ~107 deaths/1,000 live births that UNICEF's website quotes.
Posted by Heiko Gerhauser, Saturday, 20 November 2004 20:04 (link):
I am still awaiting Unicef's reply (I've only got an automated confirmation that they've received my query).
The accuracy of their figures is an important subject in its own right, and should have nothing to do with whether you are for or against the liberation of Iraq (OR were for or against sanctions before the war).
If they quote figures for 1994-1999, they should clearly say so. If their uncertainty range is more than a factor 3 either way, their figures are next to useless.
If infant mortality was comparable to neighbouring countries in 2002, sanctions didn't kill any children in that year and the potential for any improvement now is minor. If infant mortality was over 100 in 2002, sanctions/Saddam's misrule (take your pick depending on your political views) were killing tens of thousands of children, and there is plenty of scope for improvement and much less for deterioration.
And if Unicef is this wrong for Iraq, what trust should I have in their figures for other developing countries?
I hope you would agree that it is important that we can trust figures published by Unicef for vital statistics, and that these statistics be clear.
http://www.unicef.org/infobycountry/cuba_statistics.html
Is it true that Cuban infant mortality in 1960 was 39 and is now only 7, the same as the US? Wouldn't you want to know?
http://www.unicef.org/infobycountry/zimbabwe_statistics.html
Zimbabwe's infant mortality is 76? Or is it maybe in truth 250? Or 25? Is that for the whole country or maybe only half the country? It's actually describing a survey for the period 1991-1994???
-------------------------------------------
The Lancet editorial says 100,000 civilians, in the summary of the study it says most killed were killed by coalition bombing and most of the victims were women and children.
THIS is what's so inflammatory about the study, not the actual content, which gives a similar order of magnitude of violent deaths as other estimates, and for infant and non-violent mortality is inconclusive.
And THIS (the US has mercilessly slaughtered a hundred thousand innocents for its own nefarious ends) is what the study commonly is quoted for as supporting evidence.
Posted by TallDave, Thursday, 13 January 2005 21:32 (link):
The study is junk. They fail to account for the fact casualties by Coalition action are overreported or that Saddam tended to suppress reporting of casualties inflicted by his gov't prewar.
Why would Iraqis overreport such casualties? Because the Coalition attempts to atone, in some small way, for mistaken killings through financial compensation. (Did Saddam do that pre-war? Besides providing bulldozers for mass graves?)
You're also mistaken about the bombing. Conventional bombs don't vaporize people, they scatter their parts all over. The vast majority of those bodies would be recovered and their deaths would be reported.
This is all besides the hundreds of thousands we know were killed by Saddam in suppressing the Shia and Kurds. Toss in the million or so killed in the Iran-Iraq war, the 500,000 children the UN estimates Saddam killed by not allowing inspections, and the fact we're now holding elections, and sudenly the war to liberate Iraq looks like a humanitarian effort.
Post a new comment.
Comments copyright (c) contributors and available under a Creative Commons License. See also the comments policy.