Skip to main content

US public opinion regarding proposed limits on resident physician work hours

Abstract

Background

In both Europe and the US, resident physician work hour reduction has been a source of controversy within academic medicine. In 2008, the Institute of Medicine (IOM) recommended a reduction in resident physician work hours. We sought to assess the American public perspective on this issue.

Methods

We conducted a national survey of 1,200 representative members of the public via random digit telephone dialing in order to describe US public opinion on resident physician work hour regulation, particularly with reference to the IOM recommendations.

Results

Respondents estimated that resident physicians currently work 12.9-h shifts (95% CI 12.5 to 13.3 h) and 58.3-h work weeks (95% CI 57.3 to 59.3 h). They believed the maximum shift duration should be 10.9 h (95% CI 10.6 to 11.3 h) and the maximum work week should be 50 h (95% CI 49.4 to 50.8 h), with 1% approving of shifts lasting >24 h (95% CI 0.6% to 2%). A total of 81% (95% CI 79% to 84%) believed reducing resident physician work hours would be very or somewhat effective in reducing medical errors, and 68% (95% CI 65% to 71%) favored the IOM proposal that resident physicians not work more than 16 h over an alternative IOM proposal permitting 30-h shifts with ≥5 h protected sleep time. In all, 81% believed patients should be informed if a treating resident physician had been working for >24 h and 80% (95% CI 78% to 83%) would then want a different doctor.

Conclusions

The American public overwhelmingly favors discontinuation of the 30-h shifts without protected sleep routinely worked by US resident physicians and strongly supports implementation of restrictions on resident physician work hours that are as strict, or stricter, than those proposed by the IOM. Strong support exists to restrict resident physicians' work to 16 or fewer consecutive hours, similar to current limits in New Zealand, the UK and the rest of Europe.

Peer Review reports

Background

Reduction in the work hours of resident physicians (junior doctors) has been the subject of considerable controversy both in America and Europe. Under current regulations of the Accreditation Council for Graduate Medical Education (ACGME), the private body that accredits all US residency programs, resident physicians in the US work extended duration (≥24 h) shifts 1-2 times per week and work 80-88 h per week, on average. Actual work hours are frequently even longer [1–3]. By contrast, under the European Working Time Directive (EWTD), their counterparts in the UK have gone in the past several years from having no restrictions, to having work hours restricted to 72 then 56 then 48 h per week; shifts are limited to a maximum of 13 h consecutively [4]. Despite these differences in work hour limits, academic medical communities on both sides of the Atlantic have engaged in similar debate regarding the potential merits and problems of work hour reduction [5–26]. Critics of work hour reduction cite concerns about the costs of replacement providers, workforce sufficiency, disrupted continuity of care and resident physicians' training experiences [8, 27]. Conversely, supporters of reform cite the hazards of extended shifts, which have been shown to induce performance impairment comparable to an increased blood alcohol concentration [28, 29], and increase the risk of failures of attention, serious medical errors, fatigue-related preventable adverse events, percutaneous injuries and motor vehicle crashes [2, 3, 30–35]. Chronic sleep loss, induced by long work weeks, and acute sleep loss, induced by extended duration work shifts, interact synergistically to severely degrade performance, especially at night [36]. A meta-analysis of studies from 959 physicians revealed that following 24 h without sleep, their clinical performance declines to the 7th percentile of their rested performance [37].

In 2007, the US House of Representatives Committee on Energy and Commerce requested the Agency for Healthcare Research and Quality to sponsor an evidence-based assessment of the issue by the Institute of Medicine (IOM) [38], prompted by evidence that 20% of physicians in training admitted making a fatigue-related error that injured a patient (preventable adverse event) and 5% admitted making a fatigue-related error that resulted in the death of a patient (fatal preventable adverse event) during their internship [32, 38, 39].

In December 2008, following a year-long investigation, the US Institute of Medicine (IOM) released a report, Resident Duty Hours: Sleep, Supervision and Safety [40], in which it concluded that scheduling resident physicians to work for more than 16 h consecutively without sleep is unsafe both for patients and resident physicians themselves [40]. The IOM recommended that teaching hospitals either: (1) limit work shifts of resident physicians to 16 h; or (2) permit 30-h shifts, but provide 5 h of protected time for sleep in the hospital after 16 h of work, provided that residents do not admit new patients during the second portion of the shift. The IOM also recommended that: resident physicians be provided at least one 24-h interval off every 7 days, without averaging, and at least one continuous 48-h interval off per month; the work week be limited to 80 h, averaged over 4 weeks, including moonlighting; the ACGME ensure adequate, direct, onsite supervision be provided for residents; the ACGME should enhance monitoring of work hour limits, with additional oversight provided by the Centers for Medicare and Medicaid Services and the Joint Commission (a private sector, not-for-profit organization that accredits US hospitals); and teaching hospitals immediately begin providing transportation home for resident physicians impaired by fatigue.

The ACGME convened a Duty Hours Congress in 2009, at which considerable opposition to the IOM recommendations was voiced, and has since surveyed the opinions of residents, faculty and program directors, commissioned three external reviews of the literature and formed a Duty Hours Task Force. In November 2009, the American Medical Association adopted a policy opposing the IOM recommendation that protected time for sleep be provided during 30-h shifts [41, 42].

Despite the vigorousness of the debate, the views of the American public with regard to resident physician work hours, disclosure of resident physician work hours to individual patients, and support for the 2008 IOM recommendations remained largely unknown. Indeed, we are aware of only two prior American public opinion surveys that, tangentially, addressed the issue of resident work hours. In the first, 86% of patients stated that they would be anxious if they knew their surgeon had been awake for 24 h and 70% would ask for a different doctor in that circumstance [43]. In the second study, 74% of respondents listed overwork, stress or fatigue of health professionals as a 'very important cause of medical errors' and 66% felt that reducing the work hours of resident physicians would be very effective in reducing preventable errors [44, 45]. To our knowledge, no public surveys on this topic have been conducted elsewhere to date. Understanding the public's views on the IOM's resident work hour recommendations is essential, as it is the safety of the care that they receive while hospitalized that is ultimately at stake. As indicated in the IOM's prior landmark report, Crossing the Quality Chasm, a leading aim in efforts to improve the quality of the healthcare system is to better incorporate the beliefs, preferences, and experiences of patients into decisions about the manner in which healthcare is delivered [46].

To better understand the views of the American public on the hours resident physicians currently work and to gauge their reactions to the IOM's proposed limits on resident physician work hours, we administered a public opinion survey to a representative sample of the US population, the most comprehensive survey of public opinion on this issue to date.

Methods

On 17-22 November 2009 and on 21-30 January 2010, 1,200 members of the general American public participated in an 18-min telephone survey (Figure 1) that was conducted by an independent public opinion research firm (Lake Research Partners, Berkeley, CA, USA). Live interviewers provided by McGuire Research Services LLC (Denver, CO, USA) dialed randomly generated landline telephone numbers, stratified by region using area codes and phone exchanges, and either conducted a full interview or noted the reasons for failure to complete.

Figure 1
figure 1

Flowchart for participation in national US public opinion survey.

In all, 24,082 numbers were called, with calling patterns adjusted based on response rates to capture a sample of the US adult population comparable to US Census estimates by region, gender, age and race. 4,048 calls were answered by a person; interviewers then asked to speak to the adult (at least 18 years old) member of that household with the most recent birthday. Of the 4,048 calls answered by a person, 2,152 were determined to be qualified respondents. Of these, 1,200 completed the interview, 876 refused to participate and 76 terminated the call before completing the interview. The response rate (1,200/2,152) was 55.8% (Figure 1). The respondents interviewed between 17-22 November 2009 (N = 800) and the respondents interviewed between 21-30 January 2010 (N = 400) were comparable with respect to both their demographics and their answers to the survey questions, which are combined in the results section.

Participation was strictly voluntary and no personal identifiers were recorded. No financial incentives were provided for participation. Respondents were asked questions about demographic, medical service use, their view on resident physician work hours and the IOM recommendations (see Additional file 1).

The margin of error for the full sample was ± 2.8 percentage points. The data were weighted by gender, race, age, education, and region to ensure that the demographics of the sample matched the US adult population more exactly. Statistical analysis was performed using SAS v.9.1.3 (Cary, NC, USA).

Results

Demographic data

The demographic characteristics of the 1,200 study participants were similar to the US adult population (Table 1). The mean age of the survey sample was 47 ± 18.5 years and 51% were female. The participants were well distributed among the four Census Bureau regions, with 91% of participants reporting that they were registered voters. US Census data indicate that 72.1% of voting-age US citizens are registered to vote nationwide [47]; it has been previously documented that survey respondents typically self-report a higher than actual rate of voter registration, which may be due to a social desirability bias [48]. The sample reflected the ethnic and political make-up of the adult population of the US (Table 1). A large proportion of the respondents' households included a health care consumer: 91% reported that someone in their household had seen a doctor in the last year, with 68% reporting three or more visits in the past year (see Additional file 1); 44% reported that someone in their household had been hospitalized in the past 3 years, with 23% reporting that a household member was hospitalized within the past year.

Table 1 Characteristics of the survey respondents

Resident work hour limits

Work hours per shift

Respondents estimated that resident physicians currently work an average shift duration of 12.9-h shifts (95% CI 12.5 to 13.3 h) (for detailed responses, see Table 2). Among the respondents, 82.2% (95% CI 80% to 84.5%) estimated that resident physicians currently average 16 h or less per shift; 11.6% (95% CI 9.7% to 13.6%) estimated the average to be longer than 16 h; and 2% estimated the average to be longer than 24 h. When asked the maximum number of hours resident physicians should work on any given shift (Table 2), responses averaged 10.9 h (95% CI 10.6 to 11.3 h), with 90% (95% CI 89% to 92%) setting the maximum shift duration at 16 h or less. In fact, 85% of the respondents set the maximum shift duration at 12 h or less consecutively (95% CI 82% to 87%). A total of 4% (95% CI 3% to 5.6%) of the respondents set the maximum shift duration at more than 16 h, and 1% (95% CI 0.6% to 2%) set the limit at more than 24 h.

Table 2 Beliefs of survey respondents regarding current resident work hours and opinions about appropriate work hours

Weekly work hours

Respondents estimated that resident physicians currently average 58.3 h (95% CI 57.1 to 59.5 h) of work per week (Table 2). Among the respondents, 67% (95% CI 65% to 71%) thought that resident physicians currently average less than 80 h of work per week. The maximum number of hours per week that respondents believed resident physicians should be allowed to work averaged 50 h (95% CI 49.4 to 50.8 h), with 86% (95% CI 84% to 88%) setting the weekly maximum at less than 80 h and 5.5% (95% CI 4% to 7%) at 80 h or more per week.

Resident work hours and medical errors

More than four out of five respondents believed that reducing resident work hours would be 'effective' or 'somewhat effective' in reducing medical errors (Table 3). When stratified by political party, the support for this statement was maintained across political party lines.

Table 3 Support for selected resident work hour reforms and related beliefs, stratified by political party affiliation, geographic region and race

Public opinions on resident physician work hour regulation

When asked about specific IOM proposal packages, four out of five respondents supported a proposal that included limiting the duration of individual work shifts to 16 h, capping weekly work hours at a maximum of 80 h in any single week and ensuring that medical residents have at least 1 day off per week (Table 3). Of note, 67% of the subgroup of respondents who were health care workers supported this proposal, as did 83% of the subgroup who were health care workers' family members. Among the 20% of the overall respondents who withheld their support for this proposal (opposed or undecided), 44% did so because they thought that the proposal did not go far enough in limiting resident physician work hours. Thus, 89% either supported a proposal with this or stricter work hour limits.

When asked about individual components of the IOM recommendations (Table 4), three-quarters or more of respondents: (1) supported the IOM-proposed 16 h per shift limitation; (2) supported the alternative IOM recommendation requiring that at least 5 h of protected sleep be provided during a 30-h shift; (3) supported the IOM proposal that work hours be capped at no more than 80 h in any single week; (4) supported the IOM proposal that hospitals be required to provide safe transportation home, such as a taxi or public transit ticket, for residents too fatigued to drive home safely; and (5) did not believe that resident physicians should be allowed to work jobs outside of the hospital in their off hours during their training (Table 4). Two-thirds of respondents favored a 16-h maximum shift duration over the alternative IOM recommendation requiring 5 h of protected time for sleep. More than 90% of respondents supported the establishment of strict rules to ensure that medical residents are provided with direct, on-site supervision by more experienced doctors (Table 4).

Table 4 US public opinion regarding individual components of Institute of Medicine (IOM) recommendations for resident physician work hour limits

Regulatory enforcement

Among the respondents, 49% (95% CI 46% to 52%) favored letting the ACGME be responsible for regulating the hours worked by medical residents with oversight by a federal health agency, whereas 37% (95% CI 34% to 40%) favored letting the ACGME alone be responsible for such regulation. Support for letting the ACGME alone have this responsibility varied by political party affiliation: Democrats, 28%, Republicans, 47%, and Independents, 40%.

Patient reactions to extended resident physician work shifts

Among the respondents, more than four out of five respondents believed that "patients should be informed if a medical resident who is treating them has been working for more than 24 hours" (Table 3). If they learned that their doctor had been awake for more than 24 h, 85% (95% CI 82% to 87%) reported that they would "feel anxious about the safety of [their] medical care" and 80% (95% CI 78% to 83%) would "want to be treated by a different doctor" (Additional file 1: Table 4).

Discussion

Only 1% of the American public believes that resident physicians should be allowed to work more than 24 h consecutively. This judgment contrasts with current ACGME policy, which authorizes work shifts of 30 h consecutively without sleep twice each week. Most resident physicians routinely work shifts of this duration [1, 2]. As it is the public that funds both residency training (through the Centers for Medicare and Medicaid Services) and healthcare costs, and the public that ultimately suffers the potential safety consequences of healthcare provider impairment, members of the public should from an ethical standpoint have the right to make voluntary and informed health care decisions regarding the care that they receive [46]. The stark discrepancy between the views of the public and current practices regarding resident work hours is therefore of considerable concern. Most resident physicians routinely work shifts exceeding those that the public believes to be safe [1, 2]; there is considerable evidence that the public's concerns regarding the patient safety consequences of extended duration shifts and provider sleep deprivation are justified [23, 29, 31, 32, 37, 49, 50]. We found that 89% of the public supports work hour limits that go beyond the IOM recommendations, including a requirement for a shift duration of 16 h or less, as has been in place for resident physicians in New Zealand for 20 years [51]; 85% support an even more stringent 12-h limit, similar to Europe's 13-h limit [52]. On average, the American public supports a 50-h work week limit, slightly higher than the 48-h limit in Europe, but considerably less than the ACGME's 80-h average weekly limit.

Public support was very strong for each of the core components of the IOM recommendations, ranging from 75 to 91%, and held true across political party lines, regions, racial/ethnic groups and gender. When asked to choose, nearly two-thirds (64%) preferred the IOM recommendation that resident physicians not be allowed to work more than 16 h per shift in patient care to the IOM alternative that resident physicians working a 30-h shift be provided at least 5 h of protected sleep time. Two-thirds of the American public believe that resident physicians should not work more than 60 h per week and 86% believe that resident physicians should not work more than 80 h in any week.

Substantial majorities believe reductions in resident work hours would be effective in reducing medical errors, a belief with empirical support [31, 32], and that patients should be informed if their treating resident physician had worked for more than 24 h. Support for this disclosure requirement held across political party lines, regions, racial/ethnic group and gender. Survey respondents claimed that they would act on such disclosure: 80% stated that they would request another physician if their physician was working longer than 24 h, an increase from the 70% figure reported in 2002 [43]. Whether or not patients would do so cannot be determined from this study; indeed, patients may not be in a position to refuse care on a case by case basis, which makes determining their views on this issue beforehand and developing appropriate policies in response all the more important. Incorporating the voice of the patient in health care decisions is necessary to respect the ethical principle of autonomy. Patients have the right to guide decisions regarding their care, and should be the ultimate arbiters of what care they receive [3, 53].

The public estimate of the average weekly work hours of resident physicians is remarkably accurate [54]. Yet most Americans believe that the average work shift of resident physicians is currently less than 12 h in duration, with only 2% estimating the average shift to be greater than 24 h, perhaps due to the well publicized work hour limits that the ACGME implemented in 2003. Given that the public appears to be unaware that the ACGME set the work shift limit for resident physicians at 30 h consecutively in 2003, a duration that raises safety concerns for 85% of respondents, the failure to disclose whenever a treating physician has been working more than 24 h in the face of the now documented assertion by patients that most would seek alternate medical care on the basis of that information raises serious ethical and policy concerns [3, 53].

The public plays a large role in funding the training of physicians; Medicare funding for graduate medical education was nearly US $10 billion in 2008 [55]. The IOM report estimated that an additional $1.7 billion would be needed to fund its recommendations. However, if preventable adverse events were reduced by 11.3%, a reduction well within that observed in one study in the intensive care unit [31], the additional expenditures would be recovered by society [56].

Our study had a number of limitations. First, our response rate was 55.8%. It is thus possible that our population differs from the general population. However, this response rate is typical of rigorously conducted public social science polls, which average 50% [57]. Interviewers did not reveal the topic of the survey before determining whether or not the respondent was willing to participate. Rigorously conducted public opinion surveys with a 50% response rate reached a sample that, though somewhat younger and more educated than the general population, was generally representative of the US population [57, 58], To account for this potential bias, we adjusted dialing patterns to ensure that the age distribution of our survey population was representative of the US population, and weighted the data analysis by gender, race, age, education, and region to match the US adult population more exactly. Since the makeup of our sample closely reflected that of the US adult population and there were minimal differences in responses to the survey questions with respect to age, gender, geographic region and race/ethnicity, the results of both the weighted and unweighted data analysis were comparable. Moreover, the Pew Research Center only found small differences between responses from a standard survey, with a typical response rate of 30%, and responses from a rigorous survey with a response rate of 50%, including responses from those interviewees who were the hardest to reach. This is consistent with the comparability in our survey results between the first 800 respondents interviewed in November 2009 and the final 400 respondents interviewed 8 weeks later [57, 58]. Finally, the definitive results of this survey are such that it is highly unlikely that an even larger sample would yield substantially different results.

A second limitation was that we relied on landline telephone numbers. Those who maintain landlines may differ from those with only mobile telephones. However, a representative geographic distribution of residences is difficult to achieve using mobile telephone numbers, which often remain unchanged after relocation. Therefore, most public opinion surveys use landlines only. To avoid potential demographic biases related to the use of landlines, calling patterns were adjusted to capture a representative demographic population. A third limitation is that the interviews were conducted solely in English; however, 14% of our respondents identified themselves as Latino, which is comparable with that observed in the US Current Population Survey [57].

Conclusions

The IOM recommendations represent one of the first efforts to substantively reform resident work hours initiated by the leadership of academic medicine, rather than by legislative bodies. The majority of the American public endorses restrictions on resident work hours that are as stringent as or even more stringent than those put forth by the IOM and endorsed by a coalition of more than 40 public interest and patient safety groups [59]. Our data indicate that only 1% of the American public approves of the 30-h shifts currently authorized by the ACGME for residents working in teaching hospitals. Most respondents also support the addition of federal oversight to the role of the ACGME in regulating resident work hours, as recommended by the IOM. Furthermore, most respondents claim they would request a different doctor if their treating physician were awake more than 24 h. Honoring such requests could have a profound impact on the American health care delivery system, given that the 108,000 resident physicians provide much of the direct medical care in US teaching hospitals [60] and that teaching hospitals, which represent 22% of all US hospitals, provide care for 53% of all hospitalized patients and charity care for 71% of all hospitalized patients who lack adequate medical insurance and are unable to pay for their own care [61]. As the public ultimately bears the consequences of decisions affecting the safe delivery of healthcare, their opinions should be seriously considered by policy makers who regulate resident physician work hours. In other countries struggling to balance competing concerns about resident physician work hour reform, the perspectives of the public could likewise be important to consider.

Author information

ABB is a recipient of a Ruth L. Kirschstein National Research Service Award. ABB is currently a Health and Evidence Policy Fellow at Mount Sinai School of Medicine and is matriculated in the Masters in Public Health Program at The Johns Hopkins Bloomberg School of Public Health. ABB is a pediatrician who trained at University of California Los Angeles. He also serves as a consultant to CIR.

FR is currently a fellow in cardiovascular medicine at Montefiore Medical Center, the University Hospital of Albert Einstein College of Medicine. He holds board certification in internal medicine, having completed residency training at St Luke's Roosevelt Hospital Center, an affiliate of Columbia University College of Physicians and Surgeons. He also holds a PhD degree from Cornell University, an MD degree from Tehran University of Medical Sciences, and is President of CIR.

SS serves as policy director for the Committee of Interns and Residents/SEIU, a national union of resident physicians with 13,000 members in more than 50 public and private teaching hospitals in Massachusetts, New York, New Jersey, District of Columbia, Florida, New Mexico and California. SS is also the Program Director of the non-profit CIR Policy and Education Initiative. SS has more than 30 years of experience with medical residency training issues, with a consistent focus on resident work hours, and has organized two national conferences on the design of resident work to enhance patient and physician safety, cosponsored by CIR and the American Medical Association Resident Fellow Section.

DM is a partner at Lake Research Partners (LRP) and heads the firm's Bay Area office. LRP is the national research firm established in 1995 by Celinda Lake, one of the country's leading political strategists. LRP advises dozens of incumbents and challengers at all levels of the electoral process, as well as a wide range of non-partisan advocacy organizations, foundations, and labor unions. LRP has nationally recognized expertise on women voters, health care issues, the environment, and the labor movement, and conducts the bipartisan Battleground Poll, a widely respected barometer of national political trends. LRP has offices in Washington, DC and Berkeley, CA. Formerly, DM worked at The Harwood Group, a non-partisan public issues research and consulting firm, and for the Health Care Financing Administration.

PL is Senior Advisor in the Office of Policy, Planning and Budget in the Office of the Commissioner at the Food and Drug Administration. The views presented here do not represent those of the FDA or the US government. Formerly, PL was Deputy Director of Public Citizen's Health Research Group in Washington, DC, and on the faculties of the University of California, San Francisco and the University of Michigan.

CPL is director of Sleep and Patient Safety at the Brigham and Women's Hospital, Assistant Professor of Pediatrics and of Medicine at Harvard Medical School, a pediatric hospitalist at Children's Hospital Boston and a health services researcher with 12 years' experience studying the quality of inpatient care, patient safety, and the effects of healthcare providers' work hours and sleep deprivation on safety. His research has evaluated the efficiency and quality of care in pediatric hospitalist systems, as well as the safety of hospital care. CPL helped design and conduct some of the first studies to determine rates of medication errors in pediatrics, and over the past several years he has led a series of studies evaluating the relationship between resident sleep deprivation and patient safety; resident depression and patient safety; the effectiveness of computerized order entry systems; and the relationship between resident handoffs and error. CPL is an Executive Council Member and Immediate Past Chair of the Pediatric Research in Inpatient Settings (PRIS) Research Network, a collaboration of over 300 hospitalists who are beginning to study quality and variation in the care of hospitalized children, with the goal of developing and disseminating improvements.

CAC is Director of the Division of Sleep Medicine and the Harvard Work Hours, Health and Safety Group, and the Baldino Professor of Sleep Medicine at the Harvard Medical School; and is Chief of the Division of Sleep Medicine in the Department of Medicine at the Brigham and Women's Hospital. CAC serves as a member of the Steering Committees of the Sleep Research Network and the Academic Alliance for Sleep Research; is a member of the National Center for Sleep Disorders Research Advisory Board of the National Heart Lung and Blood Institute; was President of the Sleep Research Society; and served as a member of the Drowsy Driving Commission for the Commonwealth of Massachusetts. CAC has three decades of experience conducting translational research in circadian physiology, sleep science and sleep medicine, and pioneered the application of sleep and circadian physiology and pathology to occupational medicine. In recognition for his research on the consequences of resident physician work hours, CAC has been inducted as an Honorary Fellow of the Royal College of Physicians, London (FRCP); has received the National Institute of Occupational Safety and Health (NIOSH) Director's Award for the most meritorious research supported by NIOSH in 2006; and will receive the Senator Mark O Hatfield Public Policy Award from the American Academy of Sleep Medicine in 2010.

References

  1. Landrigan CP, Barger LK, Cade BE, Ayas NT, Czeisler CA: Interns' compliance with accreditation council for graduate medical education work-hour limits. JAMA. 2006, 296: 1063-1070. 10.1001/jama.296.9.1063.

    Article  CAS  PubMed  Google Scholar 

  2. Barger LK, Cade BE, Ayas NT, Cronin JW, Rosner B, Speizer FE, Czeisler CA: Extended work shifts and the risk of motor vehicle crashes among interns. N Engl J Med. 2005, 352: 125-134. 10.1056/NEJMoa041401.

    Article  CAS  PubMed  Google Scholar 

  3. Czeisler CA: Medical and genetic differences in the adverse impact of sleep loss on performance: ethical considerations for the medical profession. Trans Am Clin Climatol Assoc. 2009, 120: 249-285.

    PubMed  PubMed Central  Google Scholar 

  4. Hickner RC, Racette SB, Binder EF, Fisher JS, Kohrt WM: Suppression of whole body and regional lipolysis by insulin: effects of obesity and exercise. J Clin Endocrinol Metab. 1999, 84: 3886-3895. 10.1210/jc.84.11.3886.

    CAS  PubMed  Google Scholar 

  5. Willis RE: Views of surgery Program Directors on the current ACGME and proposed IOM duty-hour standards. J Surg Educ. 2009, 66: 216-221. 10.1016/j.jsurg.2009.06.005.

    Article  PubMed  Google Scholar 

  6. Blanchard MS, Meltzer D, Polonsky KS: To nap or not to nap? Residents' work hours revisited. N Engl J Med. 2009, 360: 2242-2244. 10.1056/NEJMe0901226.

    Article  CAS  PubMed  Google Scholar 

  7. Rybock JD: Residents' duty hours and professionalism. N Engl J Med. 2009, 361: 930-931. 10.1056/NEJMc0905152.

    Article  CAS  PubMed  Google Scholar 

  8. Britt LD, Sachdeva AK, Healy GB, Whalen TV, Blair PG: Resident duty hours in surgery for ensuring patient safety, providing optimum resident education and training, and promoting resident well-being: a response from the American College of Surgeons to the Report of the Institute of Medicine, "Resident Duty Hours: Enhancing Sleep, Supervision, and Safety". Surgery. 2009, 146: 398-409. 10.1016/j.surg.2009.07.002.

    Article  CAS  PubMed  Google Scholar 

  9. Association of Pediatric Program Directors: Association of Pediatric Program Directors (APPD) Position Statement in Response to the IOM Recommendations on Resident Duty Hours. 2009, [http://www.appd.org/PDFs/APPD_IOM%20_Duty_Hours_Report_Position_Paper_4-30-09.pdf]

    Google Scholar 

  10. Nasca T: Resident Issues Need Close Examination. AAMC Reporter. 2009, [http://www.aamc.org/newsroom/reporter/july09/viewpoint.htm]

    Google Scholar 

  11. Jagannathan J, Vates GE, Pouratian N, Sheehan JP, Patrie J, Grady MS, Jane JA: Impact of the Accreditation Council for Graduate Medical Education work-hour regulations on neurosurgical resident education and productivity. J Neurosurg. 2009, 110: 820-827. 10.3171/2009.2.JNS081446.

    Article  PubMed  Google Scholar 

  12. Chen PW: Does more sleep make for better doctors?. The New York Times. December 4, 2008, [http://www.nytimes.com/2008/12/05/health/05chen.html?_r=1&scp=1&sq=Does%20more%20sleep%20make%20for%20better%20doctors?&st=cse]

    Google Scholar 

  13. Ingle J: Resident work hours: standing up for patient safety is not "whining". The Huffington Post. December 19, 2008, [http://www.huffingtonpost.com/john-ingle-md/resident-work-hours-stand_b_152458.html]

    Google Scholar 

  14. New York Times Editorial Board: Napping during hospital shifts. The New York Times. 2008, A34-[http://www.nytimes.com/2008/12/09/opinion/09tue3.html?scp=1&sq=Napping+during+hospital+shifts&st=nyt]

    Google Scholar 

  15. USA Today Editorial Board: Our view on medical care: sleep-deprived residents still pose risk for patients. USA Today. 2008, 12A.

    Google Scholar 

  16. Nasca T: Opposing view: teaching hospitals excel. USA Today. 2008, 12A.

    Google Scholar 

  17. Moalem J, Salzman P, Ruan DT, Cherr GS, Freiburg CB, Farkas RL, Brewster L, James TA: Should all duty hours be the same? Results of a national survey of surgical trainees. J Am Coll Surg. 2009, 209: 47-54. 10.1016/j.jamcollsurg.2009.02.053.

    Article  PubMed  Google Scholar 

  18. Lopez L, Katz JT: Creating an ethical workplace: reverberations of resident work hours reform. Acad Med. 2009, 84: 315-319. 10.1097/ACM.0b013e3181971ee1.

    Article  PubMed  Google Scholar 

  19. Higginson JD: Limiting resident work hours is a moral concern. Acad Med. 2009, 84: 310-314. 10.1097/ACM.0b013e3181971bf2.

    Article  PubMed  Google Scholar 

  20. Czeisler CA: It's time to reform work hours for resident physicians. Science News. 2009, 176: 36-10.1002/scin.5591760924. [http://www.sciencenews.org/view/generic/id/48153/title/Comment__It%E2%80%99s_time_to_reform_work_hours_for_resident_physicians]

    Article  Google Scholar 

  21. Czeisler CA: Operating over the limit. The Boston Globe. 2009, A17-[http://www.boston.com/news/health/articles/2009/11/19/operating_over_the_limit/]

    Google Scholar 

  22. Warshaw AL: Walking a tightrope on MGH's training hours. The Boston Globe. 2009, A11-[http://www.boston.com/bostonglobe/editorial_opinion/oped/articles/2009/11/21/walking_a_tightrope_on_mghs_training_hours/]

    Google Scholar 

  23. Cappuccio FP, Bakewell A, Taggart FM, Ward G, Ji C, Sullivan JP, Edmunds M, Pounder R, Landrigan CP, Lockley SW, Peile E, Warwick EWTD Working Group: Implementing a 48 h EWTD-compliant rota for junior doctors in the UK does not compromise patients' safety: assessor-blind pilot comparison. Q J Med. 2009, 102: 271-282.

    Article  CAS  Google Scholar 

  24. Cappuccio FP, Lockley SW, Landrigan CP: Cappuccio response to correspondence. QJM. 2009, 102: 363-364. 10.1093/qjmed/hcp032.

    Article  PubMed  Google Scholar 

  25. BBC News: Doctor hour cuts 'will harm care'. [http://news.bbc.co.uk/2/hi/health/7328448.stm]

  26. Reid W, Coakley J: Junior doctors' working hours: 48 hours in 2009. Meeting the challenge. Clin Med. 2008, 8: 346-347.

    Article  Google Scholar 

  27. Fischer JE: Surgeons: employees or professionals?. Am J Surg. 2005, 190: 1-3. 10.1016/j.amjsurg.2005.03.023.

    Article  PubMed  Google Scholar 

  28. Dawson D, Reid K: Fatigue, alcohol and performance impairment. Nature. 1997, 388: 235-10.1038/40775.

    Article  CAS  PubMed  Google Scholar 

  29. Arnedt JT, Owens J, Crouch M, Stahl J, Carskadon MA: Neurobehavioral performance of residents after heavy night call vs after alcohol ingestion. JAMA. 2005, 294: 1025-1033. 10.1001/jama.294.9.1025.

    Article  CAS  PubMed  Google Scholar 

  30. Lockley SW, Cronin JW, Evans EE, Cade BE, Lee CJ, Landrigan CP, Rothschild JM, Katz JT, Lilly CM, Stone PH, Aeschbach D, Czeisler CA, Harvard Work Hours, Health and Safety Group: Effect of reducing interns' weekly work hours on sleep and attentional failures. N Engl J Med. 2004, 351: 1829-1837. 10.1056/NEJMoa041404.

    Article  CAS  PubMed  Google Scholar 

  31. Landrigan CP, Rothschild JM, Cronin JW, Kaushal R, Burdick E, Katz JT, Lilly CM, Stone PH, Lockley SW, Bates DW, Czeisler CA: Effect of reducing interns' work hours on serious medical errors in intensive care units. N Engl J Med. 2004, 351: 1838-1848. 10.1056/NEJMoa041406.

    Article  CAS  PubMed  Google Scholar 

  32. Barger LK, Ayas NT, Cade BE, Cronin JW, Rosner B, Speizer FE, Czeisler CA: Impact of extended-duration shifts on medical errors, adverse events, and attentional failures. PLoS Med. 2006, 3: e487-10.1371/journal.pmed.0030487.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Ayas NT, Barger LK, Cade BE, Hashimoto DM, Rosner B, Cronin JW, Speizer FE, Czeisler CA: Extended work duration and the risk of self-reported percutaneous injuries in interns. JAMA. 2006, 296: 1055-1062. 10.1001/jama.296.9.1055.

    Article  CAS  PubMed  Google Scholar 

  34. Czeisler CA: The Gordon Wilson Lecture: work hours, sleep and patient safety in residency training. Trans Am Clin Climatol Assoc. 2006, 117: 159-189.

    PubMed  PubMed Central  Google Scholar 

  35. Institute of Medicine: Sleep disorders and sleep deprivation: An unmet public health problem. [http://www.iom.edu/Reports/2006/Sleep-Disorders-and-Sleep-Deprivation-An-Unmet-Public-Health-Problem.aspx]

  36. Cohen DA, Wang W, Wyatt JK, Kronauer RE, Dijk D, Czeisler C, Klerman EB: Uncovering residual effects of chronic sleep loss on human performance. Sci Trans Med. 2010, 2: 14ra3.

    Article  Google Scholar 

  37. Philibert I: Sleep loss and performance in residents and nonphysicians: a meta-analytic examination. Sleep. 2005, 28: 1392-1402.

    PubMed  Google Scholar 

  38. Iglehart JK: Revisiting duty-hour limits - IOM recommendations for patient safety and resident education. N Engl J Med. 2008, 359: 2633-2635. 10.1056/NEJMp0808736.

    Article  CAS  PubMed  Google Scholar 

  39. Dingell JD, Barton J, Stupak B, Whitfield E: Letter written to William Munier, Agency for Healthcare, Research and Quality. 2007, Washington, DC, USA: House of Representatives Committee on Energy and Commerce, [http://energycommerce.house.gov/Press_110/110-ltr.032907.HHS.Munier.pdf]

    Google Scholar 

  40. Institute of Medicine: Resident duty hours: enhancing sleep, supervision, and safety. [http://www.iom.edu/Reports/2008/Resident-Duty-Hours-Enhancing-Sleep-Supervision-and-Safety.aspx]

  41. Goldstein J: AMA: Don't guarantee naps for residents on overnight shifts. Wall Street Journal Health Blog. 2009, [http://blogs.wsj.com/health/2009/11/23/ama-dont-guarantee-naps-for-residents-on-overnight-shifts/]

    Google Scholar 

  42. O'Reilly KB: AMA meeting: nap time mandate for on-call residents rejected. American Medical News. 2009, [http://www.ama-assn.org/amednews/2009/11/23/prsh1123.htm]

    Google Scholar 

  43. National Sleep Foundation: 2002 "Sleep in America" poll. [http://www.sleepfoundation.org/article/sleep-america-polls/2002-adult-sleep-habits]

  44. Kaiser Family Foundation: Public opinion on medical errors. [http://www.kff.org/spotlight/mederrors/index.cfm]

  45. Altman DE, Clancy C, Blendon RJ: Improving patient safety - five years after the IOM report. N Engl J Med. 2004, 351: 2041-2043. 10.1056/NEJMp048243.

    Article  CAS  PubMed  Google Scholar 

  46. Institute of Medicine: Crossing the quality chasm: a new health system for the 21st century. 2001, Washington, DC, USA: National Academic Press, [http://www.iom.edu/Reports/2001/Crossing-the-Quality-Chasm-A-New-Health-System-for-the-21st-Century.aspx]

    Google Scholar 

  47. US Census Bureau: Voting and registration in the election of November. 2004, [http://www.census.gov/prod/2006pubs/p20-556.pdf]

    Google Scholar 

  48. Fullerton AS, Dixon JC, Borch C: Bringing registration into models of vote overreporting. Publ Opin Quarter. 2007, 71: 649-660. 10.1093/poq/nfm040.

    Article  Google Scholar 

  49. Rothschild JM, Keohane CA, Rogers S, Gardner R, Lipsitz SR, Salzberg CA, Yu T, Yoon CS, Williams DH, Wien MF, Czeisler CA, Bates DW, Landrigan CP: Risks of complications by attending physicians after performing nighttime procedures. JAMA. 2009, 302: 1565-1572. 10.1001/jama.2009.1423.

    Article  CAS  PubMed  Google Scholar 

  50. Gordon JA, Alexander EK, Lockley SW, Evans EE, Venkatan SK, Landrigan CP, Czeisler CA: Does simulator-based clinical performance correlate with actual hospital behavior? The effect of extended work hours on patient care provided by medical interns. Acad Med.

  51. Gander PH, Purnell HM, Garden A, Woodward A: Work patterns and fatigue-related risk among junior doctors. Occup Environ Med. 2007, 64: 733-8. 10.1136/oem.2006.030916.

    Article  PubMed  PubMed Central  Google Scholar 

  52. British Medical Association: Time's up: a guide on the European Working Time Directive for junior doctors. [http://www.eastmidlandsdeanery.nhs.uk/document_store/12656387591_times_up_-_bma_guide_to_working_time_regulations.pdf]

  53. Czeisler CA: Ethical considerations for the scheduling of work in continuous operations: physicians in training as a case study. Sleep Epidemiology - From Aetiology to Public Health. Edited by: Cappuccio FP, Miller M, Lockley SW. 2010, Oxford, UK: Oxford University Press.

    Google Scholar 

  54. Staiger DO, Auerbach DI, Buerhaus PI: Trends in the work hours of physicians in the United States. JAMA. 2010, 303: 747-753. 10.1001/jama.2010.168.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  55. Medicare Payment Advisory Commission: Report to the Congress: Improving incentives in the medicare program. [http://www.medpac.gov/documents/jun09_entirereport.pdf]

  56. Nuckols TK, Bhattacharya J, Wolman DM, Ulmer C, Escarce JJ: Cost implications of reduced work hours and workloads for resident physicians. N Engl J Med. 2009, 360: 2202-2215. 10.1056/NEJMsa0810251.

    Article  CAS  PubMed  Google Scholar 

  57. Keeter S, Kennedy C, Dimock M, Best J, Craighill P: Gauging the impact of growing nonresponse on estimates from a national RDD telephone survey. Publ Opin Quart. 2006, 70: 759-779. 10.1093/poq/nfl035.

    Article  Google Scholar 

  58. Pew Research Center: Polls face growing resistance, but still representative. [http://people-press.org/report/211/]

  59. WakeUpDoctor.org: Safe work hours, safe patients petition. [http://www.wakeupdoctor.org]

  60. Volpp KG, Grande D: Residents' suggestions for reducing errors in teaching hospitals. N Engl J Med. 2003, 348: 851-855. 10.1056/NEJMsb021667.

    Article  PubMed  Google Scholar 

  61. Association of American Medical Colleges: Teaching hospitals and charity care. [http://www.aamc.org/newsroom/presskits/teachinghospitalscharitycare.pdf]

Pre-publication history

Download references

Acknowledgements

The nationwide survey was sponsored financially by the Committee of Interns and Residents, which is the largest union of resident physicians in the US and is affiliated with the Service Employees International Union; and by Public Citizen, a national non-profit consumer advocacy organization based in Washington, DC. The Committee of Interns and Residents contracted with an independent public opinion survey firm, Lake Research Partners, to design, administer and analyze this nationwide public opinion survey, and did not impose any limitations on the publication of the study's results. Dan Cooper, Matt Eberle and John Norris from Lake Research Partners conducted an independent analysis of the raw data. Their tabulated results of the respondents' answers to the survey questions were the same as those reported by the authors in this article. The authors wish to thank the American Medical Student Association (AMSA), a student-governed, national organization committed to representing the concerns of physicians in training in the US and Mothers Against Medical Error (MAME), an advocacy group supporting victims of medical harm, for their participation in the planning of this initiative. We thank Hillary Peabody for assistance with survey development; Dan Cooper and Matt Eberle for data programming; John Norris for overseeing the survey administration and independent analysis; and Lorna Preston, Christine Gordon and Timothy Foley for their assistance in preparation of the manuscript. Financial support for this study was provided in part by the Committee of Interns and Residents, Public Citizen, the American Medical Student Association (AMSA) and the Harvard Medical School Division of Sleep Medicine. The respondents' answers to the survey questions, including a technical appendix, statistical code, and dataset, are available from the corresponding author.

The opinions expressed herein are those of the authors and do not represent the official position of the US Food and Drug Administration or the US government.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Charles A Czeisler.

Additional information

Competing interests

The nationwide survey was sponsored by the Committee of Interns and Residents, which is the largest union of resident physicians in the US and is affiliated with the Service Employees International Union, by Public Citizen, a national non-profit consumer advocacy organization, and by the American Medical Student Association (AMSA), a student-governed, national organization committed to representing the concerns of physicians in training in the US. The Committee of Interns and Residents contracted with an independent public opinion survey firm, Lake Research Partners, to design and analyze the survey, and did not impose any limitations on the publication of the study's results. All authors have completed the Unified Competing Interest form at http://www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author). ABB is an employee of Mount Sinai School of Medicine and a paid consultant to the Committee of Interns and Residents of the Service Employees International Union (CIR/SEIU). FR is an employee of Montefiore Medical Center and is President of CIR/SEIU. SS is an employee of CIR/SEIU. DM is an employee of Lake Research Partners, an independent public opinion research firm that was provided a contract for services provided in designing, conducting and analyzing the nationwide survey reported herein. PL was previously an employee of Public Citizen. CPL is an employee of Children's Hospital Boston and of Brigham and Women's Hospital, a subsidiary of Partners HealthCare, and is on the faculty of Harvard Medical School. CPL also reports serving as a paid consultant to the District Health Boards of New Zealand, providing recommendation on how to improve the scheduling and working conditions for junior doctors in New Zealand; Vital Issues in Medicine, developing an educational course for physicians on Shift Work Disorder (supported by an unrestricted educational grant from Cephalon Inc. to Vital Issues in Medicine); and AXDev, to assist in the development of a study of Shift Work Disorder (supported by an unrestricted research grant from Cephalon Inc. to AXDev). In addition, CPL reports receiving monetary awards, honoraria, and travel reimbursement from multiple academic and professional organizations for delivering lectures on sleep deprivation and safety. CAC is an employee of Brigham and Women's Hospital, a subsidiary of Partners HealthCare, which employs more than 1,000 resident physicians, and is on the faculty of Harvard Medical School. CAC has received consulting fees from or served as a paid member of scientific advisory boards for: Actelion, Ltd; Bombardier, Inc.; Boston Celtics; Cephalon, Inc.; Columbia River Bar Pilots, Delta Airlines; Eli Lilly and Co.; Fedex Kinko's; Federal Motor Carrier Safety Administration (FMCSA), US Department of Transportation; Fusion Medical Education, LLC; Garda Síochána Inspectorate (Dublin, Ireland); Hypnion, Inc. (acquired by Eli Lilly and Co. in April 2007); Global Ground Support; Johnson & Johnson; Koninklijke Philips Electronics, NV; Minnesota Timberwolves; Morgan Stanley; Philips Respironics, Inc.; Portland Trail Blazers; Sanofi-Aventis Group; Sepracor, Inc.; Sleep Multimedia, Inc.; Sleep Research Society (for which CAC served as president); Somnus Therapeutics, Inc.; Takeda Pharmaceuticals; Vanda Pharmaceuticals, Inc.; Vital Issues in Medicine; Warburg-Pincus and Zeo Inc. CAC owns an equity interest in Lifetrac, Inc.; Somnus Therapeutics, Inc.; Vanda Pharmaceuticals, Inc., and Zeo Inc., and received royalties from McGraw Hill, the New York Times and Penguin Press. CAC has received lecture fees from the Accreditation Council of Graduate Medical Education; Alfresa; the American Academy of Allergy, Asthma and Immunology Program Directors; American Physiological Society; Association of University Anesthesiologists; Baylor College of Medicine; Beth-Israel Deaconess Medical Center; Brown Medical School/Rhode Island Hospital; Cephalon, Inc.; Clinical Excellence Commission (Australia); Dalhousie University; Duke University Medical Center; Harvard School of Public Health, Harvard University; Institute of Sleep Health Promotion (NPO); London Deanery; Morehouse School of Medicine; Mount Sinai School of Medicine; National Emergency Training Center; National Institutes of Health; North East Sleep Society; Osaka University School of Medicine; Partners HealthCare, Inc.; Sanofi-Aventis, Inc.; St Lukes Roosevelt Hospital; Takeda; Tanabe Seiyaku Co., Ltd; Tokyo Electric Power Company (TEPCO); University of Michigan; University of Pennsylvania; University of Pittsburgh; University of Tsukuba; University of Virginia Medical School; University of Washington Medical Center; University of Wisconsin Medical School; World Federation of Sleep Research and Sleep Medicine Societies. CAC has also received research prizes with monetary awards from the American Academy of Sleep Medicine; American Clinical and Climatological Association; Association for Patient-Oriented Research; National Institute for Occupational Safety and Health; National Sleep Foundation; and Sleep Research Society; clinical trial research contracts from Cephalon, Inc., Merck & Co., Inc., and Pfizer, Inc.; an investigator-initiated research grant from Cephalon, Inc.; and his research laboratory at the Brigham and Women's Hospital has received unrestricted research and education funds and/or support for research expenses from Cephalon, Inc., Koninklijke Philips Electronics, NV, ResMed, and the Brigham and Women's Hospital. The Harvard Medical School Division of Sleep Medicine (HMS/DSM), which CAC directs, has received unrestricted research and educational gifts and endowment funds from: Boehringer Ingelheim Pharmaceuticals, Inc., Cephalon, Inc., George H Kidder, Gerald McGinnis, GlaxoSmithKline, Herbert Lee, Hypnion, Jazz Pharmaceuticals, Jordan's Furniture, Merck & Co., Inc., Peter C Farrell, Pfizer, ResMed, Respironics, Inc., Sanofi-Aventis, Inc., Sealy, Inc., Sepracor, Inc., Simmons, Sleep Health Centers LLC, Spring Aire, Takeda Pharmaceuticals and Tempur-Pedic. The HMS/DSM has received gifts from many outside organizations and individuals including: Axon Sleep Research Laboratories, Inc., Boehringer Ingelheim Pharmaceuticals, Inc., Catalyst Group, Cephalon, Inc., Clarus Ventures, Eli Lilly and Co., Farrell Family Foundation, Fisher & Paykel Healthcare Corporation, George H Kidder, GlaxoSmithKline, Hypnion, Inc., Jordan's Furniture, Merck Research Laboratories, Park Place Corporation, Respironics, Inc., Sanofi-Aventis, Inc., Select Comfort Corporation, Sepracor, Inc., Sleep Health Centers LLC, Takeda Pharmaceuticals, Tempur-Pedic Medical Division, Total Sleep Holdings, Vanda Pharmaceuticals, Inc. The HMS/DSM Sleep and Health Education Program has received Educational Grant funding from Cephalon, Inc., Takeda Pharmaceuticals, Sanofi-Aventis, Inc. and Sepracor, Inc. CAC is the incumbent of an endowed professorship provided to Harvard University by Cephalon, Inc. and holds a number of process patents in the field of sleep/circadian rhythms (for example, photic resetting of the human circadian pacemaker). Since 1985, CAC has also served as an expert witness on various legal cases related to sleep and/or circadian rhythms.

Authors' contributions

ABB, SS, DM, PL, CPL, and CAC conceived of and created the study design, and together with FR designed the survey instrument. SS and PL obtained funding for the project. DM, CPL and CAC supervised the overall conduct of the study. DM oversaw survey data acquisition and data analysis through Lake Research Partners. ABB, FR, CPL and CAC prepared the first draft of the manuscript. All authors contributed to the content of the manuscript (including the authors employed by the study's sponsors) and to the design and conduct of the study; the analysis and interpretation of the data; and the preparation of the manuscript. All authors had full access to the statistical reports and data tables in the study. FR performed the statistical analyses reported in the article. All authors read and approved the final manuscript.

Alexander B Blum, Farbod Raiszadeh contributed equally to this work.

Electronic supplementary material

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Rights and permissions

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Blum, A.B., Raiszadeh, F., Shea, S. et al. US public opinion regarding proposed limits on resident physician work hours. BMC Med 8, 33 (2010). https://doi.org/10.1186/1741-7015-8-33

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1741-7015-8-33

Keywords