Lancet death toll bogus?

Kaiser_Sma

I read a letter in the times this morning sugesting that the lancet's post invasion iraq death toll figure as of july 2006 at 655,000 was flawed to a large degree. The writer refered to earlier lancet reports with a predicted "margin of error greater than 90%" and that this report has a "confidence interval of 393,000 to 943,000".
He also stated that "bio-statisticians have criticised the Lancet team's methods for the failure to allow for the possibility of unrepresentative clusters of house holds" and have "voiced doubhts about the randomness of the selection and the rigour of the interview process". He says that they claimed "92% of respondents could produce certificates for claimed death" but also that "fewer than 10% of the 655,000 [have] been officially recorded". He makes the point that "violence capable of killing 655,000 would be expected to leave millions wounded. Where are they?".
He concludes with "The appearence of the Lancet's editor on anti-war platform s suggests that the journal is driven by a political rather than scientific agenda".

Has any body come across any evidence to the contrary or corroborating this?

A quick google search will illustrate just how widley these figures are being used on news and activists sites. There has to remain a certain amount of objectivity in science, its a sad thing when the loose fabric of peer reviewed journals that makes up what we know as scientific truth becomes effected by personal or political sentimentality.

mike65

Truth is nobody knows and nor could one ever expect to make an accurate assessment of the level of fatalities in such an environment. Its a lot -anything else is a guess.

Mike.

sovtek

mike65 wrote:

Truth is nobody knows and nor could one ever expect to make an accurate assessment of the level of fatalities in such an environment. Its a lot -anything else is a guess.

Mike.

Yet it's the only scientific attempt so far, regardless of it's flaws...so therefore the best representation we have.

Sand

Do a search on it Kaiser, there is a very, very, very long thread about it from oh 6 months back or so.

The Lancet report is flawed and there are glaring contradictions - such as the 92% death certificates, but an estimated death toll something like 10 times greater than the recorded death toll.

At the end of the day, as Mike points out no one knows the total deaths in Iraq. It is certainly greater than the official figure. Whether it is 8% greater as Lancets evidence indicates, or 1000% greater as Lancets estimate indicates depends more on how sympathetic you are to the political views of the authors of the Lancet study. Its clear though that very few people have actually sat down and read the report, or looked at its authors. The figure of 655,000 is instead blandly accepted and repeated until it is accepted as fact.

FYI

The debate continues, but no one has yet to provide enough evidence to suggest the figures are wrong.

More latest info here:

http://www.radioopensource.org/les-roberts-weighs-in-on-lancet-controversy/

and it looks as though the study group will release the raw data in the hope it will answer all critics:

http://www.medialens.org/forum/viewtopic.php?t=2414&highlight=lancet

lostexpectation

nope the lacent figures are valid its just despotic wingers who want to use the same lancet methods so they can go invade darfur/kosovo and them use the in the inverse in regard to iraq

bobbyjoe

If they don't bother counting how many they kill while "liberating" then people have to use estimates.

Sand

and it looks as though the study group will release the raw data in the hope it will answer all critics:

Simpler way to do it - answer the criticisms.

The debate continues, but no one has yet to provide enough evidence to suggest the figures are wrong.

nope the lacent figures are valid its just depotic wingers who want to use the same lancet methods so the go invade darfur/kosovo and them use the in the inverse in reagard to iraq

If they don't bother counting how many they kill while "liberating" then people have to use estimates.

Like I said...

Its clear though that very few people have actually sat down and read the report, or looked at its authors. The figure of 655,000 is instead blandly accepted and repeated until it is accepted as fact.

mike65

Did'nt we do Kosovo already lioke?

Mike.

A Dub in Glasgo

Ach sure it is only innocent men, women & children that are being killed. Why bother counting them anyway

Rock Climber

A Dub in Glasgo wrote:

Ach sure it is only innocent men, women & children that are being killed. Why bother counting them anyway

You forgot the word accurately...

John_C

Sand wrote:

The Lancet report is flawed and there are glaring contradictions - such as the 92% death certificates, but an estimated death toll something like 10 times greater than the recorded death toll.

I don't know much about Iraq but I would guess that a lot of Doctors go about their business treating people and issuing death certs either without keeping records or without reporting those records to central government who, I read in the papers, only have control over a fraction of Baghdad and might not be considered relevant in another part of the country or in a part of the country which doesn't recognise the central government.

So there is not necessaraly a contradiction between the number of death certs issued and the number recorded.

Sand

I do not consider it all that likely that doctors issue death certificates but strike any other mention of the death from the official record. Either they do the paperwork, or they dont? So, the Lancet team could helpfully clarify why they take evidence that 92% of deaths are recorded, and yet use that evidence to reach the conclusion that barely 10% of deaths are recorded.

I already know the reason though - its the baseline, pre war death rate they use to compare to the current death rate. Its ludicrously low, lower than any recorded or estimated Iraq death rate prior to the war. Take a baseline figure low or high enough and you can get the result you want. The mechanics of the estimate arent terrible [Estimate current death rate, take away prior death rate makes sense for a rough estimate] , just the variables fed into it by the Lancet team.

FYI

"The Ministry of Health in Iraq has published some numbers from time to time, but these are generally considered to be unreliable. The registration of deaths in Iraq has been an organized process for many years. Death certificates have traditionally been obtained for the deaths of all adults and older children. Death certificates are required for insurance claims, compensation, payment of benefits, and for burial. Cemeteries do not take bodies for burial without certificates. If deaths occurred outside of hospital, the bodies would be transported to the general hospital for the certificate to be issued. If there were doubts about the cause of death, a post-mortem examination would be carried out before issuing a certificate. Copies of the death certificates would go to the national offices managing vital registration. This process has continued through the current conflict, with death certificates being required for burial, and with information from certificates being duly recorded. However, the tabulation of data from registration of deaths in Iraq has suffered from the chaos of the current conflict. Beyond this, there is also a suspicion that records of death, particularly related to violent deaths, is being manipulated and only partially being released for various political reasons. Even with the death certificate system, only about one-third of deaths were captured by the government’s surveillance system in the years before the current war, according to informed sources in Iraq. At a death rate of 5/1,000/year, in a population of 24 million, the government should have reported 120,000 deaths annually. In 2002, the government documented less than 40,000 from all sources. The ministry’s numbers are not likely to be more complete or accurate today.”

http://crookedtimber.org/2006/10/12/death-rates-and-death-certificates/

spank_inferno

Is there any breakdown of casualties between those caused by the "coalition of the willing" and the sectarian violence between sunni/shia?

I'm assuming (in the last 2 years) the vast majority of casualties are caused by the insurgents?
And I agree with the previous point that in any conflict zone there are many injuries for every death, surely wounded iraqi civilians dont amount to millions?!

FYI

spank_inferno wrote:

surely wounded iraqi civilians dont amount to millions?!

'Surely' is not considered an argument.

It also does not constitute a debunking of a peer reviewed scientific study.

And this has been posted numerous times before:

http://news.bbc.co.uk/1/hi/world/middle_east/5052138.stm

bonkey

FYI wrote:

no one has yet to provide enough evidence to suggest the figures are wrong.

Strange as though it is for me to be siding with Sand....

If we are to consider the report as a scientific work, then from a scientific perspective, the onus is first on the authors to show that their assertion is merited.

You don't need to prove it wrong. Valid questions about the sampling methodology have been raised and not answered. Until they are, the scientific perspective should be to not accept the accuracy of the report as being correct, rather than to say that these questions fall short of disproving the hypothesis.

The stance which I personally believe should be taken is that the Lancet study shows conclusively that there is a need for greater accounting and accountability (distasteful though each of those terms may be to some) if we actually wish to be able to determine the true cost of war in future. It also suggests that in Iraq, further study is needed to get an anyway-reliable estimate.

FYI

a piece in response to some of the criticisms:

http://scienceblogs.com/deltoid/2007/03/london_times_hatchet_job_on_la.php

More:

Death toll in Iraq: survey team takes on its critics
Jim Giles, Nature

Raw data should settle arguments over study methods.


It's not often that George W. Bush takes time out to attack a scientific paper on the day that it's released. But then few papers attract as much attention as the one that claimed that more than half a million people, or 2.5% of the population, had died in Iraq as a result of the 2003 invasion. Published last October in the run-up to the US mid-term elections, the interview-based survey attracted huge press interest and controversy.

The media spotlight has moved on, but interest within the scientific community has not. The paper has been dissected online, graduate classes have been devoted to it and critiques have appeared in the literature with more in press. So far, the discussion has created more heat than light. Many of the criticisms that dogged the study are unresolved. For example, Nature has discovered that different authors give conflicting accounts of exactly how the survey was carried out. And although many researchers say the questions hanging over the study are not substantial enough for it to be dismissed, a vocal minority disagrees.

The controversy creates extra interest in the authors' decision, made last week, to release the raw data behind the study. Critics and supporters will finally have access to information that may settle disputes.

On paper, the study seems simple enough. Eight interviewers questioned more than 1,800 households throughout Iraq. After comparing the mortality rate before and after the invasion, and extrapolating to the total population, they concluded that the conflict had caused 390,000–940,000 excess deaths (G. Burnham, R. Lafta, S. Doocy and L. Roberts Lancet 368, 1421–1428; 2006). This estimate was much higher than those based on media reports or Iraqi government data, which put the death toll at tens of thousands, and the authors, based at Johns Hopkins University in Baltimore, Maryland, and Al Mustansiriya University in Baghdad, have found their methods under intense scrutiny.

Much of the debate has centred on exactly how the survey was run, and finding out exactly what happened in Iraq has not been straightforward. The Johns Hopkins team, which dealt with enquiries from other scientists and the media, was not able to go to the country to supervise the interviews. And accounts of the method given by the US researchers and the Iraqi team do not always match up.

Several researchers, including Madelyn Hicks, a psychiatrist at King's College London, recently published criticisms of the study's methodology in The Lancet (369, 101–105; 2007). One key question is whether the interviews could have been done in the time stated. The October paper implied that the interviewers worked as two teams of four, each conducting 40 interviews a day — a very high number given the need to obtain consent and the sensitive nature of the questions.

The US authors subsequently said that each team split into two pairs, a workload that is "doable", says Paul Spiegel, an epidemiologist at the United Nations High Commission for Refugees in Geneva, who carried out similar surveys in Kosovo and Ethiopia. After being asked by Nature whether even this system allowed enough time, author Les Roberts of Johns Hopkins said that the four individuals in a team often worked independently. But an Iraqi researcher involved in the data collection, who asked not to be named because he fears that press attention could make him the target of attacks, told Nature this never happened. Roberts later said that he had been referring to the procedure used in a 2004 mortality survey carried out in Iraq with the same team (L. Roberts et al. Lancet 364, 1857–1864; 2004).

Other arguments focus on the potential for 'main-street bias', first proposed by Michael Spagat, an expert in conflict studies at Royal Holloway, University of London. In each survey area, the interviewers selected a starting point by randomly choosing a residential street that crossed the main business street. Spagat says this method would have left out residential streets that didn't cross the main road and, as attacks such as car bombs usually take place in busy areas, introduced a bias towards areas likely to have suffered high casualties.

The Iraqi interviewer told Nature that in bigger towns or neighbourhoods, rather than taking the main street, the team picked a business street at random and chose a residential street leading off that, so that peripheral parts of the area would be included. But again, details are unclear. Roberts and Gilbert Burnham, also at Johns Hopkins, say local people were asked to identify pockets of homes away from the centre; the Iraqi interviewer says the team never worked with locals on this issue.

Many epidemiologists say such discrepancies are understandable given that Roberts and Burnham could not directly oversee the survey, and do not justify accusations that the process was flawed. For those who disagree, access to the raw data is essential. Although previously reluctant to release them, Roberts and Burnham now say they are removing information that could be used to identify interviewers or respondents and will release the data within the next month to people with appropriate "technical competence".

One researcher keen to see the numbers is Spagat. The 2004 survey used GPS coordinates instead of the main-street system to identify streets to sample, and when Spagat used the limited data available so far to compare the two studies for the period immediately following the invasion, he found that the 2006 study turned up twice as many violent deaths, suggesting that main-street bias may be present.

Roberts and others question Spagat's methods. But the issue could be checked using the raw data. If main-street bias exists, says Spagat, then death rates will fall as the interviews move away from the main street.

The raw data may also help address a fear that some researchers are expressing off the record: that the Iraqi interviewers might have inflated their results for political reasons. That could show up in unusual patterns within the data.

Roberts and Burnham say they have complete confidence in the Iraqi interviewers, after working with them directly for the 2004 study. And supporters say that criticisms should not detract from the fact that the Iraqi team managed to produce a survey under extremely difficult circumstances. Security threats forced the team to change travel plans and at one point to consider cancelling the survey altogether. Since its completion, one interviewer has been killed and another has left Baghdad, although it is not known whether either case is linked to their involvement in the survey. Either way, the continuing violence in the country is enough for the remaining interviewers to say that they are not willing to repeat the exercise.

and more comments here:

http://psychoanalystsopposewar.org/blog/2007/03/01/nature-on-iraq-mortality-study/

John_C

Sand wrote:

I do not consider it all that likely that doctors issue death certificates but strike any other mention of the death from the official record. Either they do the paperwork, or they dont?

Is there any reason not to consider that likely? The two facts available are that the Lancet people found a large number of death certs in their travels around Iraq and that central Iraqi government claim that a much smaller number of certs have been issued. That, to me, screams a gap in the paperwork. The only other possability that I can see is someone telling lies. Iraqi families keep a lot of false death certs handy, the Lancet people made up the numbers or the Iraqi government is making up numbers. While I wouldn't necessaraly rule out the third option, I think a gap in the paperwork is much more likely.

Sand wrote:

I already know the reason though - its the baseline, pre war death rate they use to compare to the current death rate. Its ludicrously low, lower than any recorded or estimated Iraq death rate prior to the war.

I don't see that this will affect an argumet on the current death rate and, specifically, the contradictions in the number of death certs issued. Irrespective of the death rate before the war, the number of death certs issued now does not tally with itself.

gbh

I think its fair to say the Lancet figures are over the top.

The average death toll over the last two years or so and this has been the peak of insurgent/militia activity has been roughly 100 per day. That makes for about 35000 a year give or take. In the two years before that the average death toll was much lower at about 30 per day. That gives about 10000 per year. So post invasion roughly 100000 people have died.

As for the invasion itself, since Saddams regime was toppled in about 6 weeks, its hard to see that several hundred thousand died in this time period.

At most maybe 50000.

So I would imagine about 150000 is closer to the truth since invasion.

FYI

gbh wrote:

I think its fair to say the Lancet figures are over the top.

The average death toll over the last two years or so and this has been the peak of insurgent/militia activity has been roughly 100 per day. That makes for about 35000 a year give or take. In the two years before that the average death toll was much lower at about 30 per day. That gives about 10000 per year. So post invasion roughly 100000 people have died.

As for the invasion itself, since Saddams regime was toppled in about 6 weeks, its hard to see that several hundred thousand died in this time period.

At most maybe 50000.

So I would imagine about 150000 is closer to the truth since invasion.

gbh, I don't see how that little maths exercise backs up your thought: 'its fair to say the Lancet figures are over the top'.

You fail to mention deaths caused by coalition forces over the last 4 years, you fail to mention unreported deaths, you fail to mention deaths due to lack of proper drinking water, deaths due to increased disease due to lack of proper medical facilities and drugs, deaths due to etc etc.

I guess it's 'hard to see that several hundred thousand died in this time period' because you don't want to?

nacho libre

Rock Climber wrote:

You forgot the word accurately...

because it doesn't suit the Liberators?
I imagine if Lancet produced a report compiling deaths in Grozny during the 90's there wouldn't be this much dispute over it on this board. Do any of you dispute the offical findings of the numbers being killed by the Janjaweed? It seems to me those criticising the Lancet report on Iraq are doing so because of their political position not on the basis that it's a flawed report. Does a lesser figure for civilian deaths in Iraq somehow excuse the invasion to some people?

FYI

nacho libre wrote:

because it doesn't suit the Liberators?
I imagine if Lancet produced a report compiling deaths in Grozny during the 90's there wouldn't be this much dispute over it on this board.

Yes that is certainly true, there is a political context to this fervent questioning of the Lancet. An attempt to jump on any deficiency, however tenuous, in order to discredit the damning figures.

Indeed other studies by basically the same people using the same methods, in other conflicts, have been accepted by the 'mainstream' without a single dissenting voice. Go figure.

"The contentious and the uncontentious

An RTE report on the 17 October last, ‘Change to US strategy in Iraq is recommended’, read as if the study have never taken place. It read: "tens of thousands of Iraqis have been killed." I emailed the RTE Online Editor, Bree Treacy, to suggest an alternative phrase: "hundreds of thousands of Iraqis have been killed." [18] [Email to RTE’s online Editor, 17/10/06]

RTE’s Online Editor, Bree Treacy responded:

Dear David

Thanks for your mail and your interest in the site. The section you referred to is re-edited copy from Reuters. There is contention about the number of civilian casualties in Iraq but we strive to be accurate with our News coverage and 'tens of thousands of Iraqis have been killed' is accurate.

Thanks again for your interest.

Bree Treacy [Email, 18/10/06]

I replied:

Dear Ms. Treacy,

Thank you for responding.

However, I don't see how the fact the text was copied from a Reuters piece is relevant to the issue. Contrary to your contention, there is no reasonable or scientific refutation of the study's findings. The 'contention' you referred to is, as you are no doubt aware, politically motivated and should have no bearing on RTE's responsibility to report the facts to the best of their ability. Also contrary to your assertion, the phrase 'tens of thousands of Iraqis have been killed' is about as accurate as writing post 9/11, "tens of Americans have been killed." The latter would never appear in a respectable newspaper and neither should the former.

I remain hopeful that you will address this inaccuracy.

Yours sincerely, [Email, 18/10/06]

Ironically, RTE has seen fit to report other figures compiled using essentially the same methods, and conducted by the same lead author in other less ‘controversial’ war torn regions, without any mention of ‘contention’:

"Congo's elections, the first free elections in the former Belgian colony for more than 40 years, will hopefully put an end to Africa's bloodiest conflict, a civil war that has killed 4 million people since 1998." [RTE 2006] [11]

"The former Zaire is struggling to recover from a wider five-year war that at one stage sucked in six neighbouring countries and, according to an international aid agency, has killed up to four million people." [RTE 2005] [14]

"around 4 million people have died from violence and disease in the Congo over the past five years." [RTE 2003] [12]

"It is part of a wider war held responsible for millions of deaths in Africa's third biggest country over the past five years." [RTE 2003] [13]

"An estimated three million people, including many civilians, have been killed." [RTE 2003] [15]

Media Lens reported in September 2005 with regards to the first Iraq study:

“Les Roberts says, the reaction could not have been more different [to the Congo study]: "Tony Blair and Colin Powell quoted those results time and time again without any question as to the precision or validity."” [16]

The media’s abject failure to highlight this obvious and yet extraordinary disparity signifies yet another home-run for the ‘local highschool team’."

[/email]http://www.mediabite.org/article_A-crime-within-a-crime-within-a-crime_72619352.html

gbh

FYI wrote:

gbh, I don't see how that little maths exercise backs up your thought: 'its fair to say the Lancet figures are over the top'.

You fail to mention deaths caused by coalition forces over the last 4 years, you fail to mention unreported deaths, you fail to mention deaths due to lack of proper drinking water, deaths due to increased disease due to lack of proper medical facilities and drugs, deaths due to etc etc.

I guess it's 'hard to see that several hundred thousand died in this time period' because you don't want to?

Ok, so possibly i didnt address the question of non-frontline deaths, as in the same way you could say that millions died in world war 1 indirectly due to the fighting, ie embargoes etc.

I think i did address somewhat deaths caused by coalition forces or included them implicitly. There were the pre- "mission accomplished" deaths caused by coalition forces which I would say had to be less than fifty thousand. After that phase, I cant see a figure over 10000, because otherwise you would be talking somewhere of the order of coalition troops directly murdering 10 Iraqis every day of the week, which I dont think happens.

As for deaths caused by poor sanitation, i dont think we see on our screens malnourished Iraqi children every day of the week. Despite all the problems, i would wager that generally Iraq civilian infrastructure is much better than the infrastructure of an average African country, and so I dont think access to clean water is a major problem, although I am willing to be corrected by someone who has knowledge of life on the ground there today.

Also remember that millions of Iraqis died because of pre-war sanctions, sanctions that the US often highlighted as failing to weaken Saddams regime while punishing the poor.

Look, I was reasonaly neutral about the start of the war. There were cons such as the possibility of tens of thousands of civilian casualties caused by coalition fighting. But there was also pros such as overthrowing Saddam. So far much of the civilian casualties have been caused by insugrents and revenge death squads made up of Sunni and Shia, an unfortunate and unpredicted consequence of the war. But whatever the rights and wrongs of starting the war, it did start. I think now the coalition should pull out because really its only the Iraqis who can stop further bloodshed.

FYI

gbh wrote:

I would say ... I dont think happens. ... i dont think we see on our screens ... i would wager ... I dont think access to clean water is a major problem

Sounds like if you can't assert anything with any confidence, or crucially provide evidence to back up those assertions, nobody should put any faith in your thoughts and wagers.

gbh wrote:

an unfortunate and unpredicted consequence of the war.

Yes, indeed it is unfortunate, but not unpredicted.

" Up to 500,000 people could suffer serious injuries during the first phase of an attack on Iraq, a confidential United Nations report says."

http://news.bbc.co.uk/2/hi/middle_east/2636835.stm

" The Iraqi population was far more vulnerable to the shocks of war now then at the time of the 1991 Gulf War. The population was isolated from the outside world for 12 years and in a state of dependency on government and international aid."

http://www.mailman.hs.columbia.edu/news/waldman-iraq-b.html

"As many as 260,000 may be killed in the immediate assault, with an additional 200,000 at risk from famine and disease in the aftermath. Experts predict that U.S. invasions could lead to a civil war in Iraq, which could then result in the deaths of an additional 20,000 individuals. The vast majority of those killed would be civilians rather than military personnel."

http://www.citypaper.net/articles/2003-02-27/cover2.shtml

"Civil war? There never was a civil war? It is a tribal, not a sectarian society. Some organisation wants a civil war; oddly, it was an occupation force’s spokesman, a certain Dan Senor, who first warned of civil war in Iraq at an Anglo-American press-conference in 2003. Why? We talk of civil war far more than the Iraqis do. Why? Repeatedly, we are told that Iraqis and westerners are kidnapped by “men wearing police uniforms” or by “men wearing army uniforms”."

http://www.robert-fisk.com/articles571.htm

"Far from facilitating democracy and security, the memo's author fears, U.S. efforts have created an environment rife with corruption and sectarianism likely to result in civil war."

http://www.villagevoice.com/news/0416,vest,52796,1.html

FYI

From the BBC via...

http://members5.boardhost.com/medialens/msg/1174917101.html

Iraqi deaths survey 'was robust'
By Owen Bennett-Jones
BBC World Service

The survey estimated that 601,000 deaths were the result of violence, mostly gunfire.

The British government was advised against publicly criticising a report estimating that 655,000 Iraqis had died due to the war, the BBC has learnt.
Iraqi Health Ministry figures put the toll at less than 10% of the total in the survey, published in the Lancet.

But the Ministry of Defence's chief scientific adviser said the survey's methods were "close to best practice" and the study design was "robust".

Another statistician agreed the method was "tried and tested".

Mortality rates

The Iraq government asks the country's hospitals to report the number of victims of terrorism or military action.

Critics say the system was not started until well after the invasion and requires over-pressed hospital staff not only to report daily, but also to distinguish between victims of terrorism and of crime.

The Lancet medical journal published its peer-reviewed survey last October.

It was conducted by the Johns Hopkins School of Public Health and compared mortality rates before and after the invasion by surveying 47 randomly chosen areas across 16 provinces in Iraq.

Are we really sure the report is likely to be right? That is certainly what the brief implies

Foreign Office official

The researchers spoke to nearly 1,850 families, comprising more than 12,800 people.

In nearly 92% of cases family members produced death certificates to support their answers. The survey estimated that 601,000 deaths were the result of violence, mostly gunfire.

Shortly after the publication of the survey in October last year Tony Blair's official spokesperson said the Lancet's figure was not anywhere near accurate.

He said the survey had used an extrapolation technique, from a relatively small sample from an area of Iraq that was not representative of the country as a whole.

'Not credible'

President Bush said: "I don't consider it a credible report."

One of the documents just released by the Foreign Office is an e-mail in which an official asks about the Lancet report: "Are we really sure the report is likely to be right? That is certainly what the brief implies."

The reply from another official is: "We do not accept the figures quoted in the Lancet survey as accurate. " In the same e-mail the official later writes: "However, the survey methodology used here cannot be rubbished, it is a tried and tested way of measuring mortality in conflict zones."

Asked how the government can accept the Lancet's methodology but reject its findings, the government has issued a written statement in which it said: "The methodology has been used in other conflict situations, notably the Democratic republic of Congo.

'Mainstreet bias'

"However, the Lancet figures are much higher than statistics from other sources, which only goes to show how estimates can vary enormously according to the method of collection. There is considerable debate amongst the scientific community over the accuracy of the figures."

In fact some of the British government criticism of the Lancet report post-dated the chief scientific adviser's report.

Speaking six days after the CSA had approved the study's methods, British foreign office minister Lord Triesman said: "the way in which data are extrapolated from samples to a general outcome is a matter of deep concern...."

It would appear they were only able to sample a small sliver of the country

Dr Michael Spagat

Some scientists have subsequently challenged the validity of the Lancet study. Questions have been asked about the survey techniques and the possibility of "mainstreet bias".

Dr Michael Spagat of Royal Holloway London University says that most of those questioned lived on main streets which are more likely to suffer from car bombs: "It would appear they were only able to sample a small sliver of the country," he said.

Dr Spagat has previously conducted research with Iraq Body Count, an NGO that counts deaths on the basis of media reports and which has produced estimates far lower than those published in the Lancet.

If the Lancet survey is right, then 2.5% of the Iraqi population -- an average of more than 500 people a day - have been killed since the start of the war.

The BBC World Service made a Freedom of Information Request on 28 November 2006. The information was released on 14 March 2007.

http://news.bbc.co.uk/1/hi/uk_politics/6495753.stm

Sand

FYI - youll notice the endorsement was given to the method of the study, not the study itself.

As I said myself....

The mechanics of the estimate arent terrible [Estimate current death rate, take away prior death rate makes sense for a rough estimate] , just the variables fed into it by the Lancet team.

The Lancet study produces a figure far in excess of any other study because the the people behind the study wanted to get a high, headline grabbing figure in time for the US elections. The method is fine and sound, but it is open to abuse by those with political motives like the Lancet team.

bonkey

Sand wrote:

FYI - youll notice the endorsement was given to the method of the study, not the study itself.

Not entirely accurate.

The comments were to the effect that the report could not be dismissed through faulty methodology.

The Lancet study produces a figure far in excess of any other study because the the people behind the study wanted to get a high, headline grabbing figure in time for the US elections. The method is fine and sound, but it is open to abuse by those with political motives like the Lancet team.

You can't have both, Sand. Either the method as implemented is fine and sound, or it isn't. If it was abused, then the vector of abuse is a part of the implementation and therefore makes the implementation open to criticism.

If there is a problem with the implementation of the method, then that problem should be identifiable. Otherwise, the best one can say is that they are of the opinion that an undetectable abuse of best-practice was engaged in.

If you can show why the figure is high, then the methodology used is obviously open to criticism. If the methodology isn't open to criticism, then you can't show why the figure is high and are therefore merely assuming it is.

Science can be used to play politics, but the problem is that science stands and falls by its falsifiability. To be perfectly honest, if someone wants to show the Lancet figures to be deliberately manipulated, then they either need to show how that manipulation was carried out. Alternately, if they wish to seriously call the figures into question, then they need to perform an equivalent study which is of at least the same standard, but which shows contradictary results.

However, simply handwaving and insisting that the figures must be wrong because they disagree with all less accurate studies not based on best-practice methodology isn't a terribly convincing argument.

Akrasia

Sand wrote:

The Lancet study produces a figure far in excess of any other study.

You say that as if any of the other studies that you refer to are worth a ****e. The official figures are just plucked out of mid air. The Iraqi civil service are utterly unable to process proper death statistics at the moment, and even if they did, they have a huge motivation to understate them, The IBC figure is hopelessly flawed.

You can't discredit a scientific study by comparing it other totally discredited studies. You can't disprove evolution by referring to creationism and saying "But it's so different to creationism, evolution can't be true"

Sand

You can't have both, Sand. Either the method as implemented is fine and sound, or it isn't. If it was abused, then the vector of abuse is a part of the implementation and therefore makes the implementation open to criticism.

The Lancet authors based their calculations on an overall, post-invasion, excess mortality rate of 7.8/1000/year. "Pre-invasion mortality rates were 5.5 per 1000 people per year (95% CI 4.3–7.1), compared with 13.3 per 1000 people per year (10.9–16.1) in the 40 months post-invasion."[2] See Table 3 in the Lancet article[2].

I took the above from the wiki article. If you download and read the study youll notice the rate is 5.5

I have absolutely NO issues with the method of the study. Estimating a death rate, comparing it to a prior death rate and assuming the difference to be due to an event that occured between those two rates is not perfect, but its certainly acceptable in difficult circumstances, like Iraq.

The British experts that FYI mentioned above also do not have an issue with the method of the study. The full quote is "The study design is robust and employs methods that are regarded as close to "best practice" in this area, given the difficulties of data collection and verification in the present circumstances in Iraq."

What I have an issue with is the information and variables used by that study. It is current death rate - prior death rate = excess death rate due to war. A prior death rate of 5.5 is ludicrously low.

Dennis Halliday, the UN human rights commissioner resigned in 1998 so he could criticise the UN sanctions, as he said he could no longer continue working for a program that fit the definition of genocide. Noam Chomsky picked up on this and accused the US of engineering genocide in Iraq. In 1995 Lancet [hello, hello....] rushed out a study that claimed an excess mortality of 567,000 children [children alone mind] caused by the UN sanctions - this has since been revised down to about 370,000 or so. When Madeline Albright was confronted with these figures she dismissed them claiming "we think the price is worth it".

Heres the text of a WHO submission made in 2001 regarding the situation in Iraq. One stat leap out at me. First, prior to the first Gulf war, when Iraqs health system was at its height the crude mortality rate was 8 deaths per 1000. Now, given the sort of woe that was unleashed on Iraq during the next 13 years is it all that likely that the crude death rate *fell* to 5.5? The WHO article doesnt note the rise in the adult death rate but the child mortality rate rose significantly by their estimates and disease, malnutrition, lack of clean water and poor medical supplies were all rife. Bu Lancet, defying all reason chose a rate that was less than WHOs crude death rate for Iraq back in the golden age of Iraqi health.

So yes, I can have my cake an eat it. I think the method of the study is fine, I think the base rate they inserted was stupidly low and it was made that low so they could get a nice headline grabbing excess mortality rate. Others have criticised the information gathering, but to my mind the most blatantly manipulated and easily rigged side of the equation is the prior death rate.

You say that as if any of the other studies that you refer to are worth a ****e. The official figures are just plucked out of mid air. The Iraqi civil service are utterly unable to process proper death statistics at the moment, and even if they did, they have a huge motivation to understate them, The IBC figure is hopelessly flawed.

You can't discredit a scientific study by comparing it other totally discredited studies. You can't disprove evolution by referring to creationism and saying "But it's so different to creationism, evolution can't be true"

In all that rambling and attempts at diversion, was there a point? The study is discredited on its own merits or lack thereof. Its a political exercise, it was rigged to meet a political end, and it is so badly rigged that it blatantly contradicts its own sample where 92% of deaths they sampled were recorded, but 92% of deaths they estimated apparently arent.

John_C

Sand wrote:

those with political motives like the Lancet team.

Can you tell us more about this? Is the Lancet not a pretty respected medical journal?