Are benefits and harms in mammography screening given equal attention in scientific articles? A cross-sectional study
© Jørgensen et al; licensee BioMed Central Ltd. 2007
Received: 18 January 2007
Accepted: 30 May 2007
Published: 30 May 2007
The CONSORT statement specifies the need for a balanced presentation of both benefits and harms of medical interventions in trial reports. However, invitations to screening and newspaper articles often emphasize benefits and downplay or omit harms, and it is known that scientific articles can be influenced by conflicts of interest. We wanted to determine if a similar imbalance occurs in scientific articles on mammography screening and if it is related to author affiliation.
We searched PubMed in April 2005 for articles on mammography screening that mentioned a benefit or a harm and that were published in 2004 in English. Data extraction was performed by three independent investigators, two unblinded and one blinded for article contents, and author names and affiliation, as appropriate. The extracted data were compared and discrepancies resolved by two investigators in a combined analysis. We defined three groups of authors: (1) authors in specialties unrelated to mammography screening, (2) authors in screening-affiliated specialties (radiology or breast cancer surgery) who were not working with screening, or authors funded by cancer charities, and (3) authors (at least one) working directly with mammography screening programmes. We used a data extraction sheet with 17 items described as important benefits and harms in the 2002 WHO/IARC-report on breast cancer screening.
We identified 854 articles, and 143 were eligible for the study. Most were original research. Benefits were mentioned more often than harms (96% vs 62%, P < 0.001). Fifty-five (38%) articles mentioned only benefits, whereas seven (5%) mentioned only harms (P < 0.001). Overdiagnosis was mentioned in 35 articles (24%), but was more often downplayed or rejected in articles that had authors working with screening, (6/15; 40%) compared with authors affiliated by specialty or funding (1/6; 17%), or authors unrelated with screening (1/14; 7%) (P = 0.03). Benefits in terms of reduced breast cancer mortality were mentioned in 109 (76%) articles, and was more often provided as a relative risk reduction than an absolute risk reduction, where quantified (45 articles (31%) versus 6 articles (3%) (P < 0.001)).
Scientific articles tend to emphasize the major benefits of mammography screening over its major harms. This imbalance is related to the authors' affiliation.
It is essential to consider both benefits and harms carefully when the merits of any intervention are discussed in the medical literature. This was recently emphasised in an extension to the CONSORT statement on reporting randomised trials , but it also applies to other scientific work, and to information material directed towards healthy citizens and patients . However, the harms of breast cancer screening are often downplayed or left unmentioned in information materials, thus providing an inadequate basis for informed decision-making [3–7]. A similar bias has been found in newspaper articles on breast cancer screening . This focus on benefits is contrary to the ethical imperative that those subjected to any preventive measure or treatment should receive unbiased information , and it is particularly troublesome in screening, as this is directed towards healthy individuals, not patients seeking treatment. The 2002 WHO/IARC report on breast cancer screening notes that "...the vast majority of women undergoing screening do not have breast cancer at the time of the examination, and these women cannot derive a direct health benefit from screening; they can only be harmed" .
This report also notes that "An obvious source of harm associated with any screening programme is unnecessary treatment of cancers that were not destined to cause symptoms"  and that "Overdiagnosis refers to the detection of cancers that would never have been found were it not for the screening test. Patients in whom such indolent cancers are detected do not benefit from screening and can only experience harm: the worry associated with a cancer diagnosis and the complications of therapy" . Overtreatment is the treatment of such lesions. Overdiagnosis is different from identification of false positive cases, which refers to women recalled for further testing who do not have a suspicion of cancer confirmed. We were particularly interested in the issue of overdiagnosis as it markedly influences the life of women who experience it, could affect a substantial number of women , and because it is not mentioned in invitations to screening .
We were also interested in the potential conflicts of interest, including the medical specialties of the authors [11, 12]. A relationship between conclusions and conflicts of interest has been demonstrated for passive smoking , anti-inflammatory drugs in rheumatoid arthritis , spinal manipulation , interventions to reduce allergens from house dust mites  and cardiovascular trials . Authors also prefer references that support their preconceived opinions [18, 19].
We examined articles in medical journals discussing mammography screening to see if they gave equal attention to benefits and harms, as we suspected that information materials and newspaper articles would reflect their presentation of the issues. We also explored the affiliation and source of funding of the authors to examine if they were associated with the likelihood of mentioning, or the framing of, benefits and harms. We experienced some challenges that led to supplementary analyses, which we also report.
We included all articles on mammography screening that described a benefit or a harm, or both, that were published in English in 2004 and were identified in a comprehensive PubMed search; (exp breast neoplasm/all OR "breast cancer" OR exp mammography/all OR mammograph*) AND (exp mass screening/all OR screen*). We excluded articles on mammographic techniques, assessments of methodological approaches to evaluate screening, letters, and articles written by ourselves. The PubMed search was performed in April 2005 and was limited to articles published in 2004 to obtain a manageable sample size that was as up to date as possible. Two investigators read titles and abstracts independently. Potentially eligible articles were collected in hardcopy and read in full.
Data extraction was performed in parallel and independently by the three authors. One author (PG) was blinded to author affiliation when evaluating contents, and blinded to contents when evaluating author affiliation. The other two authors (KJJ and AK) were not blinded. Blinding was obtained by cutting out all information from the article on author names, affiliation and funding, and presenting this information separately from the contents. The blinded author noted all instances where he thought he could guess who the authors were or to which conflict of interest category they belonged (see below). Extracted data from the two unblinded authors were compared and consensus reached where there was disagreement ("unblinded consensus"). Subsequently, unblinded consensus data and data from the blinded extraction were compared and conflicts resolved by two of the authors (KJJ and PG) using the blinded material for reference. We analyzed our data in three ways: (1) data extracted by the blinded author, (2) data extracted by unblinded authors, (3) data combined from all three authors. This analysis will be referred to as the "combined analysis".
Benefits and harms of mammography screening included in our data extraction sheet and mentioned as important in the 2002 WHO/IARC report on mammography screening 
Breast cancer mortality reduction
Relative risk reduction
Absolute risk reduction
Survival time from diagnosis
Number needed to screen
Carcinoma in situ, as positive
Less mastectomies/more tumourectomies
Total mortality reduction
Overdiagnosis and overtreatment
1. Per screen
More mastectomies/more tumourectomies
Increased biopsy rate
Risks associated with additional radiotherapy
Psychological effects related to false positives
Carcinoma in situ, as negative
Pain at mammography
We classified articles in three author affiliation categories: (1) articles where all authors were in specialties unrelated to mammography screening, (2) authors in screening-affiliated specialties (radiology or breast cancer surgery) who were not working with screening, or authors funded by cancer charities, or (3) articles where at least one author worked directly with mammography screening. In many cases, author affiliation was not clear from the articles, or it was not clear if the institution employing the author performed screening. We therefore retrieved additional information from the Internet for our combined analysis. We were not able to find this for all authors, however, and these cases were labelled as not being affiliated with screening.
During our study, we realised that a blinded data extraction would be necessary to guard against bias pertaining to knowledge of author names or affiliation while evaluating if a particular harm was downplayed or acknowledged. Conversely, knowing the contents of an article could potentially influence our evaluation of author affiliation. In a few cases, we knew the author had previously worked with screening. If such articles were sceptical towards screening, we could be prone to label the author affiliation according to current status, and conversely if the article was supportive of screening. We therefore took care not to let our knowledge of the authors influence our judgments, and used the authors' current affiliation in all cases.
We used a chi-square test for trend  to compare the three conflict of interest groups for potential differences in the emphasis on benefits and harms, and a sign test to compare mutually exclusive events in the same group that would be expected to be similarly distributed in the absence of a preference for either when exploring potential differences among author affiliation groups in the number of times the separate 17 items were mentioned and acknowledged.
Distribution of article types, according to author affiliation (blinded consensus)
Work unrelated to screening
Working with screening
No. of articles
No. of articles
No. of articles
Other (e.g. commentaries)
Number and proportion of the 143 articles eligible for review that mention benefits, mention harms, and acknowledge harms of screening
No. of articles
No. of articles
No. of articles
Blinding appeared to be effective. In only 16 cases did the blinded observer guess the identity of the authors, or an author's affiliation, and in only one case might this have influenced the judgments of the reported benefits and harms, as the other 15 cases were unambiguous.
We found that scientific articles tend to emphasize the benefits of mammography screening over the harms. The focus on benefits carries a risk of underestimating harms and is in conflict with guidelines on reporting outcomes of medical interventions . Invitations to screening, information material, and newspaper articles have all been demonstrated to present information biased in favour of screening [3–7] As they are all likely to reflect the views presented in the scientific literature, our results could explain a similar focus on benefits in those areas. Failure to cover harms in invitations has serious ethical implications and violates requirements for informed consent . Guidelines on how to develop balanced decision aids exists, and use of such guidelines could help improve the quality of information materials in the future [21, 22].
Many authors, in particular those working with screening, downplay or reject even those harms that are arguably the most important: overdiagnosis and overtreatment. This is unfortunate, because we know a priori, as stated by the Director of the UK National Screening Committee, that "All screening programmes do harm; some can do good as well" . The most serious harms of screening should therefore have equal priority to the most important benefit – the reduction in breast cancer mortality – if not a higher one, as far more people will experience harm than will benefit . Overdiagnosis and overtreatment were often downplayed as negligible by authors working with screening, but they are not. Assuming a reduction in breast cancer mortality of about 15%, as estimated in the two most recent systematic reviews [24, 25]., and 30% overdiagnosis as indicated by the randomised trials , screening 2000 women over 10 years would prevent one breast cancer fatality but turn 10 healthy women into cancer patients unnecessarily .
Even in comprehensive studies, an evaluation of both benefits and harms of medical interventions is not always performed, or not always published. One recent example is the systematic review on mammography screening performed on behalf of the US Preventive Services Task Force . Although this is the only systematic review performed of all the screening trials after the Cochrane Review of screening was first published in 2001 , it only evaluated the reduction in breast cancer mortality. Many of the important harms were mentioned, including overdiagnosis and overtreatment, but none were quantified. The first Cochrane Review is another example. Although the protocol for the review listed use of surgical interventions (biopsy, tumourectomy and mastectomy) and use of chemotherapy and radiotherapy as outcomes to become assessed, the editors of the Cochrane Breast Cancer Group decided to defer presentation and discussion of these results until further editorial review had been completed. These data – that demonstrated substantial overtreatment – were published elsewhere at the time  and they are now also available in the updated Cochrane Review . The Cochrane Collaboration is aware of the importance of studying and communicating the harms of interventions, but even with this in mind and with the best of intentions, unpleasant findings can delay publication. The inclusion and quantification of harms such as overdiagnosis and overtreatment in the updated version of the Cochrane Review  shows, however, that it is possible to improve reporting and provide a balanced presentation of benefits and harms of interventions. The original review was the first to demonstrate that screening leads to substantial overtreatment, and the estimate of about 30% overdiagnosis and overtreatment has subsequently been supported by large epidemiological studies [28–31]. and with extended data from the Malmö mammography screening trial [32, 33]. Hopefully, the increased focus on harms of interventions will be reflected in information material directed towards the public in the future.
Scientific articles on mammography screening favour information on the mortality reduction, and prefer to present this as a relative risk reduction rather than an absolute risk reduction. A relative risk reduction appears more impressive, but tends to make lay people, as well as health professionals, overestimate the obtainable benefit . This problem is known from scientific articles in general, and is particularly important in a screening setting as so few will benefit of the total number screened .
We experienced some problems while working on this study. Following a pilot data extraction from 10 articles, we realised that simply recording whether or not a harm was mentioned would provide misleading results. The context in which the harm appeared (framing) had a major influence on the impression the reader would get of its meaning and importance. We therefore decided to evaluate whether or not the harms were downplayed or rejected. This decision was made after the completion of our protocol, but before data extraction began. Our evaluation of whether or not a harm was downplayed could have been influenced by our own views, and others might disagree. We have therefore included a sample of quotes to allow the readers to make their own interpretation (see Additional file 1). However, the focus on benefits in our sample is clear, regardless of this analysis.
As the focus of the screening debate changes, the preferred topics may change as well, and our sample may differ from a sample taken in a different year. However, we find it unlikely that this would play any major role, as the inevitable problem of overdiagnosis with cancer screening has been known for decades.
The results of the blinded data extraction generally supported the findings in the unblinded extraction, but the trends were less pronounced, as could be expected. We thought that the results of our combined evaluation would be somewhere in between the two other results, but this was not the case for the relation between author affiliation and the likelihood for acknowledging overdiagnosis and overtreatment. This is mainly due to a change in affiliation for some authors when we did more extensive searches on the Internet for the combined evaluation.
Scientific articles tend to emphasize the major benefits of mammography screening over its major harms. This imbalance is related to the authors' affiliation.
This study did not receive any financial support.
- Ioannidis JP, Evans SJ, Gøtzsche PC, O'Neill RT, Altman DG, Schulz K, Moher D: Better reporting of harms in randomized trials: an extension of the CONSORT statement. Ann Intern Med. 2004, 141: 781-788.View ArticlePubMedGoogle Scholar
- General Medical Council: Seeking patients' consent: the ethical considerations. London: GMC Guidelines, [http://www.gmc-uk.org/guidance/current/library/consent.asp]
- Slaytor EK, Ward JE: How risks of breast cancer and benefits of screening are communicated to women: analysis of 58 pamphlets. BMJ. 1998, 317: 263-264.View ArticlePubMedPubMed CentralGoogle Scholar
- Croft E, Barratt A, Butow P: Information about tests for breast cancer: what are we telling people?. J Fam Pract. 2002, 51: 858-860.PubMedGoogle Scholar
- Rasky E, Groth S: Information materials on mammography screening in Austria – do they help women with informed decision?. Soz Praventiv Med. 2004, 49: 391-397. 10.1007/s00038-004-3092-0.View ArticleGoogle Scholar
- Jørgensen KJ, Gøtzsche PC: Presentation on websites of possible benefits and harms from screening for breast cancer: cross sectional study. BMJ. 2004, 328: 148-151. 10.1136/bmj.328.7432.148.View ArticlePubMedPubMed CentralGoogle Scholar
- Jørgensen KJ, Gøtzsche PC: Content of invitations for publicly funded screening mammography. BMJ. 2006, 332: 538-541. 10.1136/bmj.332.7540.538.View ArticlePubMedPubMed CentralGoogle Scholar
- Wells J, Marshall P, Crawley B, Dickersin K: Newspaper reporting of screening mammography. Ann Intern Med. 2001, 135: 1029-1037.View ArticlePubMedGoogle Scholar
- Vainio H, Bianchini F: IARC Handbooks of Cancer Prevention: Breast Cancer Screening. 2002, Lyon: IARCPress, 7.Google Scholar
- Zahl PH, Strand BH, Mæhlen J: Incidence of breast cancer in Norway and Sweden during introduction of nationwide screening: prospective cohort study. BMJ. 2004, 328: 921-924. 10.1136/bmj.38044.666157.63.View ArticlePubMedPubMed CentralGoogle Scholar
- Als-Nielsen B, Chen W, Gluud C, Kjærgard LL: Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events?. JAMA. 2003, 290: 921-928. 10.1001/jama.290.7.921.View ArticlePubMedGoogle Scholar
- Chalmers TC, Frank CS, Reitman D: Minimizing the three stages of publication bias. JAMA. 1990, 263: 1392-1395. 10.1001/jama.263.10.1392.View ArticlePubMedGoogle Scholar
- Barnes DE, Bero LA: Why review articles on the health effects of passive smoking reach different conclusions. JAMA. 1998, 279: 1566-1570. 10.1001/jama.279.19.1566.View ArticlePubMedGoogle Scholar
- Gøtzsche PC: Methodology and overt and hidden bias in reports of 196 double-blind trials of nonsteroidal antiinflammatory drugs in rheumatoid arthritis. Control Clin Trials. 1989, 10: 31-56. 10.1016/0197-2456(89)90017-2.View ArticlePubMedGoogle Scholar
- Assendelft WJ, Koes BW, Knipschild PG, Bouter LM: The relationship between methodological quality and conclusions in reviews of spinal manipulation. JAMA. 1995, 274: 1942-1948. 10.1001/jama.274.24.1942.View ArticlePubMedGoogle Scholar
- Schmidt LM, Gøtzsche PC: Of mites and men: reference bias in narrative review articles: a systematic review. J Fam Pract. 2005, 54: 334-338.PubMedGoogle Scholar
- Ridker PM, Torres J: Reported outcomes in major cardiovascular clinical trials funded by for-profit and not-for-profit organizations: 2000–2005. JAMA. 2006, 295: 2270-2274. 10.1001/jama.295.19.2270.View ArticlePubMedGoogle Scholar
- Ravnskov U: Cholesterol lowering trials in coronary heart disease: frequency of citation and outcome. BMJ. 1992, 305: 15-19.View ArticlePubMedPubMed CentralGoogle Scholar
- Gøtzsche PC: Reference bias in reports of drug trials. BMJ. 1987, 295: 654-656.View ArticlePubMedPubMed CentralGoogle Scholar
- Altman DG: Practical Statistics for Medical Research. 1991, London: Chapman & HallGoogle Scholar
- Charnock D, Shepperd S, Needham G, Gann R: DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999, 53: 105-111.View ArticlePubMedPubMed CentralGoogle Scholar
- Elwyn G, O'Connor A, Stacey D, Volk R, Edwards A, Coulter A, Thomson R, Barratt A, Barry M, Bernstein S, Butow P, Clarke A, Entwistle V, Feldman-Stewart D, Holmes-Rovner M, Llewellyn-Thomas H, Moumjid N, Mulley A, Ruland C, Sepucha K, Sykes A, Whelan T, The Internantional Patient DecisionAids Standards(IPDAS) Collaboration: Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. BMJ. 2006, 333: 417-422. 10.1136/bmj.38926.629329.AE.View ArticlePubMedPubMed CentralGoogle Scholar
- Gray J: Evidence-Based Healthcare. How to Make Health Policy and Management Decisions. 1997, Edinburgh: Churchill Livingstone, 24.Google Scholar
- Gøtzsche PC, Nielsen M: Screening for breast cancer with mammography. Cochrane Database Syst Rev. 2006, CD001877-4Google Scholar
- Humphrey LL, Helfand M, Chan BK, Woolf SH: Breast cancer screening: a summary of the evidence for the US Preventive Services Task Force. Ann Intern Med. 2002, 137: 347-360.View ArticlePubMedGoogle Scholar
- Olsen O, Gøtzsche PC: Screening for breast cancer with mammography. Cochrane Database Syst Rev. 2001, 4: CD001877.PubMedGoogle Scholar
- Olsen O, Gøtzsche PC: Cochrane review on screening for breast cancer with mammography. Lancet. 2001, 358: 1340-1342. 10.1016/S0140-6736(01)06449-2.View ArticlePubMedGoogle Scholar
- Gøtzsche PC: On the benefits and harms of screening for breastcancer. Int J Epidemiol. 2004, 33: 56-64. 10.1093/ije/dyh014.View ArticlePubMedGoogle Scholar
- Douek M, Baum M: Mass breast screening: is there a hidden cost?. Br J Surg. 2003, 90: 44-45.Google Scholar
- Barratt A, Howard K, Irwig L, Salkeld G, Houssami N: Model of outcomes of screening mammography: information to support informed choices. BMJ. 2005, 330: 936-940. 10.1136/bmj.38398.469479.8F.View ArticlePubMedPubMed CentralGoogle Scholar
- Jonsson H, Johansson R, Lenner P: Increased incidence of invasive breast cancer after the introduction of service screening with mammography in Sweden. Int J Cancer. 2005, 117: 842-847. 10.1002/ijc.21228.View ArticlePubMedGoogle Scholar
- Zackrisson S, Andersson I, Janzon L, Manjer J, Garne JP: Rate of over-diagnosis of breast cancer 15 years after end of Malmo mammographic screening trial: follow-up study. BMJ. 2006, 332: 689-692. 10.1136/bmj.38764.572569.7C.View ArticlePubMedPubMed CentralGoogle Scholar
- Gøtzsche PC, Jørgensen KJ: Estimate of harm/benefit ratioof mammography screening was five times too optimistic. BMJ. 2006, [http://www.bmj.com/cgi/eletters/332/7543/691]Google Scholar
- Fahey T, Griffiths S, Peters TJ: Evidence based purchasing: understanding results of clinical trials and systematic reviews. BMJ. 1995, 311: 1056-1059.View ArticlePubMedPubMed CentralGoogle Scholar
- Schwartz LM, Woloshin S, Dvorin EL, Welch G: Ratio meassures in Leading Medical Journals: Structured review of accessibility of underlying absolute risks. BMJ. 2006, 333: 1248-1253. 10.1136/bmj.38985.564317.7C.View ArticlePubMedPubMed CentralGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1741-7015/5/12/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.