- Open Access
How should we manage information needs, family anxiety, depression, and breathlessness for those affected by advanced disease: development of a Clinical Decision Support Tool using a Delphi design
BMC Medicine volume 13, Article number: 263 (2015)
Clinicians request guidance to aid the routine use and interpretation of Patient Reported Outcome Measures (PROMs), but tools are lacking. We aimed to develop a Clinical Decision Support Tool (CDST) focused on information needs, family anxiety, depression, and breathlessness (measured using the Palliative care Outcome Scale (POS)) and related PROM implementation guidance.
We drafted recommendations based on findings from systematic literature searches. In a modified online Delphi study, 38 experts from 12 countries with different professional backgrounds, including four patient/carer representatives, were invited to rate the appropriateness of these recommendations for problems of varying severity in the CDST. The quality of evidence was added for each recommendation, and the final draft CDST reappraised by the experts. The accompanying implementation guidance was built on data from literature scoping with expert revision (n = 11 invited experts).
The systematic literature searches identified over 560 potential references, of which 43 met the inclusion criteria. Two Delphi rounds (response rate 66 % and 62 %; n = 25 and 23) found that good patient care, psychosocial support and empathy, and open communication were central to supporting patients and families affected by all POS concerns as a core requirement. Assessment was recommended for increasing problems (i.e. scores), followed by non-pharmacological interventions and for breathlessness and depression, pharmacological interventions. Accompanying PROM implementation guidance was built based on the 8-step International Society for Quality of Life Research framework, as revised by nine (response rate 82 %) experts.
This CDST provides a straightforward guide to help support clinical care and improve evidence-based outcomes for patients with progressive illness and their families, addressing four areas of clinical uncertainty. Recommendations should be used flexibly, alongside skilled individual clinical assessment and knowledge, taking into account patients’ and families’ individual preferences, circumstances, and resources. The CDST is provided with accompanying implementation guidance to facilitate PROM use and is ready for further development and evaluation.
Providing optimal care to patients facing life-threatening and progressing illnesses requires a focus on patients’ and their family caregivers’ physical, psychological, emotional, and spiritual needs . It is not solely a task of experts in palliative care. The number of people affected by chronic and progressive illness is escalating, fueled by population ageing and improved life expectancy . While specialist palliative care services, such as hospices and home care teams, provide an extra layer of support for those with the most complex needs, patients and their families with progressive and life threatening illness come into contact with health and social care workers in all settings – primary, secondary, and tertiary. Providing holistic care for seriously ill patients and their families is now a central component of health care . This can, however, be daunting for many doctors, nurses and other staff, especially with limited formal (undergraduate and postgraduate) training [4, 5].
An important starting point in effective clinical management is the assessment and identification of problems . Unfortunately, practitioners often miss many of patients’ needs and symptoms, especially when these are non-physical [7–11]. This problem is compounded because patients have multiple symptoms and concerns [12–14]. Asking patients to report their concerns using Patient Reported Outcome Measures (PROMs) has been proposed as a way to overcome this . When repeated over time, such assessments can become measurable outcomes, revealing how patients’ health status changed following the care provided .
Although practitioners express positive attitudes towards measuring PROMs [17, 18], routine measurement of concerns is hampered by a lack of training and guidance on how to use and respond to PROMs in clinical care [17–19]. Published guidance on using PROMs [20–22] is fragmented; it lacks specified steps and recommendations to follow when implementing PROMs [20, 21] or does not focus on advance disease . Even more pressing, there is a lack of guidance on how to respond to specific PROM scores. There is often an unfamiliarity with the score interpretation , while PROM scores are potentially not fed back to the right person, or reported as often as needed . Clinical significance of health-related quality of life scores are not always reported in studies [25, 26]. Unsurprisingly, therefore, clinical decision-making is not often based on outcome scores  and measuring PROMS seems to have a stronger impact on process-aspects of care, i.e. detection of symptoms, than on outcome-aspects of care, i.e. patients’ health status (e.g. [15, 23, 28, 29]). Ideally, PROMs should routinely be used and assist in detecting problems, planning treatment, and monitoring how well concerns are alleviated.
To fill this gap we need specific clinical decision support aids for specific PROM scores. This should be linked with better focused guidance on how to implement PROMs in routine clinical practice. It is important that both sets of guidance are developed together. This will ensure specific PROMs are successfully implemented in clinical care, while at the same time clinicians are supported in using the PROMs when responding to concerns.
One of the most widely used outcome measures in clinical practice in advanced illness is the Palliative (or Patient) care Outcome Scale (POS) family of measures (consisting of the POS, Integrated POS (IPOS), African Palliative Care Association African POS (APCA African POS), and POS-Symptoms (POS-S)) . It assesses physical symptoms, emotional, psychological, and spiritual concerns, and needs for information and support . It is brief (<10 minutes to complete), widely validated, able to transfer across settings, has good responsiveness to change, and has been translated and/or culturally adapted and revalidated in many different languages and cultures (e.g. [32–34]). A full suite of free user support resources is available at www.pos-pal.org . During a training day on POS in 2013, we identified clinicians’ need for guidance in the interpretation of and responding to POS scores. Most difficulties were encountered with interpreting the questions regarding psychological functioning (i.e. depression/feeling worthwhile, n = 18/36 comments), information provision (n = 7/36 comments), and family anxiety (n = 4/36 comments). Breathlessness is a very disturbing symptom for patients [36, 37] and their families . Creation of evidence-based clinical guidance on how to respond to these concerns is needed.
The aim of this study was therefore to develop a Clinical Decision Support Tool (CDST) for the specific POS items of most concern to clinicians: information needs, family anxiety, depression, and breathlessness. The guidance is aimed for all practitioners and settings, and is applicable to all patients/family caregivers with complex needs and progressive, life-threatening and serious disease. Because successful implementation of PROMs in clinical practice is a precondition to using a CDST, but current guidance is fragmented, in parallel we also developed accompanying guidance on PROM implementation. This article reports the development and final formats of the CDST and implementation guidance. It also offers a novel methodological approach for CDSTs in other areas.
A systematic literature search and modified Delphi study were conducted to develop the CDST. A literature search and expert consultations were used to create the PROMs implementation guidance. Ethical approval was granted by the Research Ethical Committee of King’s College London (BDM/13/14-3). Participants were explained that participation implied informed consent.
Clinical Decision Support Tool (CDST)
The CDST on how to respond to different levels of reported POS scores on information needs, family anxiety, depression, and breathlessness was created using a systematic literature search to develop preliminary recommendations for clinical care, followed by a modified Delphi approach to determine how the recommendations should be applied for different levels of POS score severity.
Systematic literature search
To develop draft recommendations we searched for guidelines and systematic reviews on the aforementioned topics in PubMed, Google Scholar (first 4 pages), Cochrane Database, and the York DARE database (2000 to end June 2013). The websites of NICE (UK), National Guideline Clearinghouse (US), the Canadian Medical Association, and google.com (first 4 pages) were hand searched for relevant guidelines. Three guides on using PROMS were screened for relevant information [20–22] (see Additional file 1 for search strategies).
Inclusion criteria were:
Guidelines/systematic reviews focusing on general palliative care, or focusing on information needs, family anxiety, depression, or breathlessness in palliative care (sources focusing on palliative care in a specific setting were included)
Guidelines/systematic reviews published in English/Dutch/German/Italian (languages available in the study team)
Guidelines from national (disease) organizations
Guidelines providing an evidence-base for created recommendation
Systematic reviews published in peer-reviewed journals
Exclusion criteria were guidelines/systematic reviews:
Not focusing solely on palliative care (but on the entire trajectory of disease(s))
Focusing on a specific disease
Focusing solely on pediatric care
Of which an updated version was available
Of which no full-text was available
Data from included sources on how to clinically respond to information needs, family anxiety, depression, and breathlessness was extracted (by LV).
LV drafted recommendations based on the extracted evidence. These were then critically revised by the co-authors (RH, CB, SP). Two authors (LV/IH) finalized the draft recommendations which were then taken forward into the subsequent modified Delphi study.
Modified Delphi study
We conducted a two-round online Delphi-study [39, 40] (hosted via internet platform Keypoint), to appraise and revise the draft recommendations and agree an expert consensus on the appropriateness of all recommendations for all scores on the POS items.
Overall, 38 experts were purposefully sampled (we approached 48, of which 10 declined beforehand); comprising 26 clinicians, 24 researchers (some had a dual role), and 4 patient/family representatives (as experts by experience, recruited via the UK National Council of Palliative Care and the European Cancer Patient Coalition). Experts came from the UK, US, Netherlands, Italy, Germany, Australia, South Africa, Belgium, Greece, Sweden, Poland, and Switzerland. Experts were chosen based on their expertise in the field of the topics under study.
In the Delphi round 1, experts rated the appropriateness of recommendations for all the different answer categories (0–4) for each POS-item and provided comments and/or suggested revisions. Appropriateness was rated on a 1–9 scale (‘not at all appropriate’ to ‘extremely appropriate’) with a ‘do not know’ option and space for comments. In round 2, a summary of the results of round 1 was provided (median, range, interquartile ranges, summary qualitative remarks) which participants were asked to take into account when rating the recommendations again. The results of round 1 were circulated. If recommendations in round 2 achieved a median ranging 7–9 and did not have >30 % of scores in the 1–3 and 7–9 range, they were deemed as appropriate [41–43] and included in the CDST. Lastly, the draft CDST was commented upon by (the same, plus one additional) experts via email and in a face-to-face research meeting (with different researchers).
Quality of evidence
For each recommendation, the quality of evidence was determined, using an adapted GRADE approach  with the classifications: A (e.g. meta-analysis, systematic review of randomized controlled trials (RCTs), RCT), B (e.g. cohort studies, case–control studies), C (e.g. retrospective, poor quality cohort studies), and D (qualitative studies, expert opinion) quality evidence. We rated each recommendation using the most recent and highest level of known evidence (based on design). If sources already provided a quality rating, their rating was used. Additional file 2 depicts how quality ratings from each source were mapped to the ABCD framework.
Accompanying guidance on implementing PROMs
We scoped the literature for relevant guidance on implementing PROMs. Data was extracted from these sources to develop (with assistance of several experts in the field) a draft implementation guidance linked to our specific PROM, the POS. Next, we invited comments and critical revisions via email from the 10 members of the European Association for Palliative Care (EAPC) Taskforce Outcome Measures (we approached 13, of which three declined beforehand), plus one additional expert who agreed to participate, representing the countries of the UK, US, Germany, Australia, Italy, Belgium, and South Africa. A finalized version was created with comments from one additional expert.
Clinical Decision Support Tool (CDST)
Systematic literature search
Our search of the online databases revealed 703 sources (564 after duplication removal) of which 31 were included (see Fig. 1 for the flowchart of inclusion) [45–75]. Two additional guidelines [76, 77] were added as they were referred to by two included references [49, 73]. In addition, nine extra guidelines were included from the screened websites [78–86]. As panic/anxiety is often associated with breathlessness, we screened the NICE and Clearinghouse websites and all included guidelines for additional sources, revealing one additional source . Therefore, ultimately 43 sources were included.
All included sources were scrutinized for evidence-based conclusions, data and recommendations on how to clinically respond to information needs, family anxiety, depression, and breathlessness in palliative care. Overall, 47 recommendations were drafted to be rated in the Delphi Round 1. Not all included sources provided information that could be used to draft recommendations, e.g. one review focusing on the effect of home-based palliative care found inconsistent results. As no other source focused on this topic, no recommendations about home-based care were made .
Modified Delphi study
In total, 25/38 (66 %) of our experts participated in round 1. Open comments were used to adjust wordings of several recommendations (services should be ‘offered’ instead of ‘provided’; pharmacological interventions should be ‘offered, alongside non-pharmacological interventions’). For round 2, experts were asked to take into account a summary of the qualitative and quantitative results of round 1. In this second round, 62 % (23/37, one person withdrew from the project) of the experts participated. Demographic characteristics of participating experts in rounds 1 and 2 are displayed in Additional file 3. Open comments (services should be provided, ‘depending on resources’; psycho-education, ‘i.e. teaching, explanation’) were used to finalize the recommendations, while the predefined cut-offs were used to draft the CDST. Quality of evidence was added for each recommendation, ranging from high to very low (later adapted to A–D). One recommendation (regarding chest-wall vibration to treat breathlessness) was downgraded from A to B as evidence was based on laboratory (as opposed to clinical) studies.
Final clinical decision support tool
Two formats of a decision-diagram for each POS item and a manual of the CDST were created and again sent out to all experts, of which 41 % (15/37) provided comments, in addition to one other expert in the field. Main comments focused on making it clear that for higher POS scores, lower recommendations still apply and creating both a short and long manual, while both formats of the decision diagrams were equally preferred (also in the face-to-face researchers meeting). The final decision-diagrams are depicted in Figs. 2, 3, 4, 5, 6, 7, 8 and 9. Additional file 4 depicts the short manual while the long manual can be found at the Palliative care Outcome Scale website . Independent of POS score, core recommendations center on: i) good patient care, ii) the provision of psychosocial support and empathy, and iii) the use of open communication. Proper assessment is needed for increasing scores, followed by non-pharmacological interventions, and pharmacological interventions for high levels of depression and breathlessness.
Our scoping identified four major guides which had complementary approaches: i) the PROMs guidelines of the International Society for Quality of Life Research (ISOQOL) including an 8-step framework , ii) the Outcome Measures in Palliative Care booklet , iii) the White paper on Outcome Measures of the EAPC , and iv) for the POS family of measures, the Guidelines for using the POS . To provide guidance on implementing PROMs in clinical practice, the ISOQOL’  8-step framework was followed. The 8 steps include i) identify goals for collecting PROMs; ii) select patients, setting, and timing of assessment; iii) determine which questionnaire to use; iv) choose a mode for administering/scoring the questionnaire; v) design processes for reporting results, vi) identify aids to facilitate score interpretation; vii) develop strategies for responding to identified issues; and 8) evaluate the impact of measuring PROMs on practice. We amalgamated consistent recommendations from the different sources into this framework.
In total, 9/11 (82 %) of the invited participants provided comments on the guidance (in addition to one other expert in the field). The comments stressed the need to define the target group, present tables/boxes, and introductory passages, and these were integrated into the final guidance. The key recommendations of the implementation guidance are summarized in Table 1 for each step (the complete guidance can be found at ).
This is the first study to develop a CDST for some of the complex problems faced in advanced disease, in particular information needs, family anxiety, depression, and breathlessness, linking these to specific actions. The CDST has an evidence-based approach, responding to different levels of patient-reported symptoms. We found it was necessary to have both core recommendations, applicable for all symptom levels, as well as responses that could be titrated up with increasingly severe symptom scores. Accompanying guidance on PROM implementation was also developed, using an 8-step approach. By producing both types of evidence, PROMs (and specifically POS, or similar tools) might be more widely integrated in clinical care and their effects on patients’ outcomes can be strengthened.
The potential of our guidance and CDST to improve provided care and patient outcomes is supported by accumulating evidence. The Australian Palliative Care Outcome Collaboration showed that implementing PROMs nationally in palliative care improved patients’ reported symptoms year after year [89, 90]. Expert consensus underlines the importance of coming to a routinely collected set of outcome measures, which can be used to make comparisons across services and countries . Our provided guidance might assist in this development while overcoming some of the perceived barriers of implementing PROMs, most notably a lack of knowledge and education [6, 91].
The CDST can assist clinicians in responding to the psychosocial aspects of clinical care which they may feel less confident in delivering. Across the several symptoms included in our Delphi, for higher scores, more intensive clinical responses seem warranted. Ensuring the core recommendations (good patient care; providing psychosocial support and empathy; the use of open communication) are in place remains vital in these situations. These core recommendations reflect previous study findings, showing that patients expect their clinicians to show both technical competence  as well as seeing them as an individual person . Empathy becomes increasingly important in progressing disease , while most medical complaints are related to communication deficits [95, 96], adding not only to patient but also caregiver distress . Our CDST postulates that proper assessment should be followed by non-pharmacological interventions which can be progressed to pharmacological interventions in the more medical oriented symptoms of breathlessness and depression. However, also in these domains, there seems a preference for psychosocial interventions. Indeed, a recent RCT showed that a short-term service provided by palliative care, respiratory, physiotherapy, and occupational therapy improved breathless patients’ outcomes . This service combined pharmacological review with a focus on non-pharmacological interventions such as pacing and relaxation.
Although originally developed for the POS family of measures, the core recommendations point towards the potential usefulness of our CDST for other PROMs covering similar domains (e.g. the Edmonton Symptom Assessment System , the McGill Quality of Life Questionnaire , or quality of life measures such as the EORTC-QLQC30 ). It is a wider problem that in advanced illness patients’ and caregivers’ psychosocial needs often remain unmet [102, 103]. A recent systematic review, including but not limited to POS, concluded that feeding back PROM data in palliative care influenced patients’ psychosocial outcomes the most . Our CDST might strengthen this finding by generalizing its applicability beyond POS.
Although the CDST proposals are based on scientific evidence combined with expert opinion, this tool is not intended to be prescriptive. Instead, it aims to help practitioners think through the best decision towards difficult and complex encountered problems. Being too prescriptive can be counterproductive, especially because in advanced illness patients and families have complex and often quite individual needs, circumstances, and trajectories, which interact [1, 105]. Instead the CDST should be used as the name implies, as a support, to aid a wider range of aspects to be considered when making treatment choices, alongside skilled individual clinical assessment and knowledge, taking into account patients’ and families’ individual preferences, circumstances and available resources. In areas of uncertainty or conflict, specialist support or a second opinion should be obtained. The CDST should not be used as an endpoint (‘tick-box’ exercise) in itself but as a starting point to achieve high quality person-centered care and good clinical practice. As the implementation guide makes clear, training and ongoing support will be an essential component.
This study has limitations. Our sample size was relatively small, although it should be noted that the responses rates in the online Delphi (66 % vs 62 %) are in line with a previous similar study in this field . Next, we asked participants in the Delphi study to review a wide area of topics, which might have been challenging, especially surrounding the specific topic of breathlessness. However, as palliative care focuses on physical, psychological, emotional, and spiritual issues, we anticipated most participants to have (some) knowledge in all domains and added a ‘do not know’ option in the Delphi. The quality rating should also be interpreted with caution as we used an adapted GRADE approach. Many sources used different rating systems and we followed sources’ own evidence-levels if already provided. No strength of recommendation (representing whether desirable effects of recommendations clearly outweigh (or not) undesirable effects ) was provided due to the heterogeneity of used quality ratings and the envisaged applicability of our recommendations in different settings and countries. Lower ratings do not necessarily imply that these recommendations are not important, as they were among the highest rated recommendations by our expert panel (e.g. the provision of emotional support and respect for cultural/religious traditions when handling family anxiety). These ratings also highlight the pressing need for more high-quality research studies to build the evidence-base of these and other key components of palliative care. For example, the use of a fan was labelled as ‘no evidence’ based on a Cochrane review . However, this is still in the early stages of evaluation, with recent studies showing conflicting effects (e.g. [107, 108]). As fans are inexpensive, unlike to cause any harm and have low side effects, they can be worth trying. Another limitation entailed that we only included sources that focused on palliative care in general and not on specific diseases as we sought recommendations that were applicable across diseases. For some diseases, the evidence might propose alternative strategies; however, we would expect experts in those fields to be aware of this evidence to take it into consideration. Although we used a broad search strategy, relevant sources might have been missed. Furthermore, we refer in footnotes to the Cicely Saunders Institute’s Breathlessness Intervention Service and the Vitaltalk organization for more resources on breathlessness and information needs. These were not found via our search strategies (as they are very recent), but we believed them to be of importance to clinical care. Finally, due to time and resource constraints, only one author identified resources and extracted data of included sources. That being said, a major strength of this work includes the involvement of patients and families in the development of the CDST and the combination of evidence-based recommendations with clinician expertise. Future studies can use this study as a starting point in developing evidence-based CDST for advanced disease, and should now refine our proposed CDST, test its effect on patient and family outcomes, and develop CDST for other POS items.
Our findings underline the importance of providing good patient care, psychosocial support and empathy, and communication in advanced disease for all patients and families, irrespective of POS scores, in the domains of information needs, family anxiety, depression, and breathlessness. For increasing scores, patients’ symptoms should be assessed and responded to with non-pharmacological interventions, followed by pharmacological interventions (for breathlessness and depression). We were able to develop these recommendations into a CDST. Our novel evidenced-based approach to develop a CDST offers a replicable method for other areas. By using the presented recommendations alongside skilled clinical knowledge and patient preferences and ongoing training, it aims to help support clinicians provide the best possible patient-centered care and patients and their families achieve the best possible outcomes in highly threatening times. Systematically following the 8-step framework can support successful implementation of PROMS, and POS, in advanced disease.
Clinical decision support tool
European Association for Palliative Care
International Society for Quality of Life Research
Palliative (or patient) Outcome Scale
Patient reported outcome measures
Higginson IJ, Costantini M. Dying with cancer, living well with advanced cancer. Eur J Cancer. 2008;44:1414–24.
Gomes B, Higginson IJ. Where people die (1974—2030): past trends, future projections and implications for care. Palliat Med. 2008;22:33–41.
Quill TE, Abernethy AP. Generalist plus specialist palliative care – creating a more sustainable model. New Engl J Med. 2013;368:1173–5.
Frey R, Gott M, Raphael D, O’Callaghan A, Robinson J, Boyd M, et al. Clinical staff perceptions of palliative care-related quality of care, service access, education and training needs and delivery confidence in an acute hospital setting. BMJ Support Palliat Care. 2014;4:381–9.
Gibbins J, McCoubrie R, Forbes K. Why are newly qualified doctors unprepared to care for patients at the end of life? Med Educ. 2011;45:389–99.
Antunes B, Harding R, Higginson IJ. Implementing patient-reported outcome measures in palliative care clinical practice: a systematic review of facilitators and barriers. Palliat Med. 2014;28:158–75.
Stromgren AS, Groenvold M, Pedersen L, Olsen AK, Spile M, Sjogren P. Does the medical record cover the symptoms experienced by cancer patients receiving palliative care? A comparison of the record and patient self-rating. J Pain Symptom Manage. 2001;21:189–96.
Brunelli C, Costantini M, DiGiulio P, Gallucci M, Fusco F, Miccinesi G, et al. Quality-of-life evaluation: when do terminal cancer patients and health-care providers agree? J Pain Symptom Manage. 1998;15:151–8.
Petersen MA, Larsen H, Pedersen L, Groenvold M. Assessing health-related quality of life in palliative care: comparing patient and physician assessments. Eur J Cancer. 2006;42:1159–66.
Hladschik-Kermer B, Kierner KA, Heck U, Miksovsky A, Reiter B, Zoidl H, et al. Patients and staff perceptions of cancer patients’ quality of life. Eur J Oncol Nurs. 2013;17:70–4.
Janse A, Gemke R, Uiterwaal C, van der Tweel I, Kimpell JL, Sinnema G. Quality of life: patients and doctors don’t always agree: a meta-analysis. J Clin Epidemiol. 2004;57:653–61.
Moens K, Higginson IJ, Harding R. Are there differences in the prevalence of palliative care-related problems in people living with advanced cancer and eight non-cancer conditions? A systematic review. J Pain Symptom Manage. 2014;48:660–77.
Solano JP, Gomes B, Higginson IJ. A comparison of symptom prevalence in far advanced cancer, AIDS, heart disease, chronic obstructive pulmonary disease and renal disease. J Pain Symptom Manage. 2006;31:58–69.
Radbruch L, Nauck F, Ostgathe C, Elsner F, Bausewein C, Fuchs M, et al. What are the problems in palliative care? Results from a representative survey. Support Care Cancer. 2003;11:442–51.
Greenhalgh J. The applications of PROs in clinical practice: what are they, do they work, and why? Qual Life Res. 2009;18:115–23.
Donabedian A. Explorations in quality assessment and monitoring. Ann Arbor: Health Administration Press; 1980.
Bausewein C, Simon ST, Benalia H, Downing J, Mwangi-Powell FN, Daveson BA, et al. Implementing patient reported outcome measures (PROMs) in palliative care – users’ cry for help. Health Qual Life Out. 2011;9:27.
Daveson B, Simon S, Benalia H, Downing J, Higginson IJ, Harding R, et al. Are we heading in the same direction? European and African doctors’ and nurses’ views and experiences regarding outcome measurement in palliative care. Palliat Med. 2012;26:242–9.
Simon ST, Higginson IJ, Harding R, Daveson BA, Gysels M, Deliens L, et al. Enhancing patient-reported outcome measurement in research and practice of palliative and end-of-life care. Support Care Cancer. 2012;20:1573–8.
Bausewein C, Daveson BA, Benalia A, Simon ST, Higginson IJ. Outcome measurement in palliative care. The Essentials; 2011. PRISMA (Reflecting the Positive Diversities of European Priorities for Research and Measurement in End-of-Life Care).
Bausewein C, Daveson BA, Currow DC, Downing J, Deliens L, Radbruch L, et al. EAPC White paper on Outcome Measurement in Palliative Care: improving practice, attaining outcomes and delivering quality services. Recommendations from the European Association for Palliative care (EAPC) Task Force on outcome measurement. Palliat Med June 11, 2015 [Published online before print].
International Society for Quality of Life Research; Aaronson NK, Choucair AK, Elliott TE, Greenhalgh J, Halyard MY, Hess R, et al. User’s guide to implementing patient-reported outcomes assessment in clinical practice, Version: November 11, 2011. http://www.isoqol.org/research/isoqol-publications. Accessed June 2013.
Valderas JM, Alonso J, Guyatt GH. Measuring patient-reported outcomes: moving from clinical trials into clinical practice. Med J Aust. 2008;189:93–4.
Greenhalgh J, Long AF, Flynn R. The use of patient reported outcome measures in routine clinical practice: lack of impact or lack of theory? Soc Sci Med. 2005;60:833–43.
Cocks K, King MT, Velikova G, Fayers PM, Brown JM. Quality, interpretation and presentation of European Organisation for Research and Treatment of Cancer quality of life questionnaire core 30 data in randomised controlled trials. Eur J Cancer. 2008;44:1793–8.
Efficace F, Osoba D, Gotay C, Sprangers M, Coens C, Bottomley A. Has the quality of health-related quality of life reporting in cancer clinical trials improved over time? Towards bridging the gap with clinical decision making. Ann Oncol. 2007;18:775–81.
Espallargues M, Valderas JM, Alonso J. Provision of feedback on perceived health status to health care professionals: a systematic review of its impact. Med Care. 2000;38:175–86.
Greenhalgh J, Meadows K. The effectiveness of the use of patient‐based measures of health in routine practice in improving the process and outcomes of patient care: a literature review. J Eval Clin Pract. 1999;5:401–16.
Marshall S, Haywood K, Fitzpatrick R. Impact of patient‐reported outcome measures on routine practice: a structured review. J Eval Clin Pract. 2006;12:559–68.
Higginson IJ, Simon ST, Benalia H, Downing J, Daveson BA, Harding R, et al. Republished: which questions of two commonly used multidimensional palliative care patient reported outcome measures are most useful? Results from the European and African PRISMA survey. Postgrad Med J. 2012;88:451–7.
Hearn J, Higginson IJ. Development and validation of a core outcome measure for palliative care: the palliative care outcome scale. Palliative Care Core Audit Project Advisory Group. Qual Health Care. 1999;8:219–27.
Harding R, Selman L, Agupio G, Dinat N, Downing J, Gwyther L, et al. Validation of a core outcome measure for palliative care in Africa: the APCA African Palliative Outcome Scale. Health Qual Life Outcomes. 2010;8:10.
Bausewein C, Fegg M, Radbruch L, Nauck F, von Mackensen S, Borasio GD, et al. Validation and clinical application of the German version of the palliative care outcome scale. J Pain Symptom Manage. 2005;30:51–62.
Bausewein C, Le Grice C, Simon S, Higginson I. PRISMA. The use of two common palliative outcome measures in clinical care and research: A systematic review of POS and STAS. Palliat Med. 2011;25:304–13.
Palliative care Outcome Scale. A resource for palliative care. www.pos-pal.org. Accessed on August 21, 2015.
Disler RT, Green A, Luckett T, Newton PJ, Inglis S, Currow DC, et al. Experience of advanced chronic obstructive pulmonary disease: metasynthesis of qualitative research. J Pain Symptom Manage. 2014;48:1182–99.
Weingaertner V, Scheve C, Gerdes V, Schwarz-Eywill M, Prenzel R, Bausewein C, et al. Breathlessness, functional status, distress, and palliative care needs over time in patients with advanced chronic obstructive pulmonary disease or lung cancer: a cohort study. J Pain Symptom Manage. 2014;48:569–81. e1.
Malik FA, Gysels M, Higginson IJ. Living with breathlessness: a survey of caregivers of breathless patients with lung cancer or heart failure. Palliat Med. 2013;27:647–56.
Goodman CM. The Delphi technique: a critique. J Adv Nurs. 1987;12:729–34.
Dalkey NC, Brown BB, Cochran S. The Delphi method: An experimental study of group opinion. Santa Monica: Rand Corporation; 1969.
Elwyn G, O’Connor A, Stacey D, Volk R, Edwards A, Coulter A, et al. Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. BMJ. 2006;333:417.
Campbell SM, Cantrill JA, Roberts D. Prescribing indicators for UK general practice: Delphi consultation study. BMJ. 2000;321:425–8.
Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR, Lazaro P, et al. The RAND/UCLA appropriateness method user’s manual. Santa Monica: RAND; 2001.
Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336:924–6.
Parker SM, Clayton JM, Hancock K, Walder S, Butow PN, Carrick S, et al. A systematic review of prognostic/end-of-life communication with adults in the advanced stages of a life-limiting illness: patient/caregiver preferences for the content, style, and timing of information. J Pain Symptom Manage. 2007;34:81–93.
Andershed B. Relatives in end-of-life care--part 1: a systematic review of the literature the five last years, January 1999-February 2004. J Clin Nurs. 2006;15:1158–69.
Candy B, Jones L, Drake R, Leurent B, King M. Interventions for supporting informal caregivers of patients in the terminal phase of a disease. Cochrane Database Syst Rev. 2011;6:CD007617.
Davidson JE, Powers K, Hedayat KM, Tieszen M, Kon AA, Shepard E, et al. Clinical practice guidelines for support of the family in the patient-centered intensive care unit: American College of Critical Care Medicine Task Force 2004–2005. Crit Care Med. 2007;35:605–22.
Hudson P, Remedios C, Zordan R, Thomas K, Clifton D, Crewdson M, et al. Guidelines for the psychosocial and bereavement support of family caregivers of palliative care patients. J Palliat Med. 2012;15:696–702.
Hudson PL, Remedios C, Thomas K. A systematic review of psychosocial interventions for family carers of palliative care patients. BMC Palliat Care. 2010;9:17.
Scheunemann LP, McDevitt M, Carson SS, Hanson LC. Randomized, controlled trials of interventions to improve communication in intensive care: a systematic review. Chest. 2011;139:543–54.
Candy B, Jones L, Varagunam M, Speck P, Tookman A, King M. Spiritual and religious interventions for well-being of adults in the terminal phase of disease. Cochrane Database Syst Rev. 2012;5:CD007544.
Hardy SE. Methylphenidate for the treatment of depressive symptoms, including fatigue and apathy, in medically ill older adults and terminally ill adults. Am J Geriatr Pharmacother. 2009;7:34–59.
Ly KL, Chidgey J, Addington-Hall J, Hotopf M. Depression in palliative care: a systematic review. Part 2. Treatment. Palliat Med. 2002;16:279–84.
Ostlund U, Brown H, Johnston B. Dignity conserving care at end-of-life: a narrative review. Eur J Oncol Nurs. 2012;16:353–67.
Rayner L, Price A, Evans A, Valsraj K, Hotopf M, Higginson IJ. Antidepressants for the treatment of depression in palliative care: systematic review and meta-analysis. Palliat Med. 2011;25:36–51.
Rayner L, Higginson I, Price A, Hotopf M. The management of depression in palliative care: European clinical guidelines. London: Department of Palliative Care, Policy & Rehabilitation, European Palliative Care Research Collaborative; 2010.
Ujeyl M, Muller-Oerlinghausen B. Antidepressants for treatment of depression in palliative patients: a systematic literature review. Schmerz. 2012;26:523–36.
Bausewein C, Booth S, Gysels M, Higginson I. Non-pharmacological interventions for breathlessness in advanced stages of malignant and non-malignant diseases. Cochrane Database Syst Rev. 2008;2:CD005623.
Gallagher R, Roberts D. A systematic review of oxygen and airflow effect on relief of dyspnea at rest in patients with advanced disease of any cause. J Pain Palliat Care Pharmacother. 2004;18:3–15.
Jennings AL, Davies AN, Higgins JPT, Anzures-Cabrera J, Broadley Karen E. Opioids for the palliation of breathlessness in advanced disease and terminal illness. Cochrane Database of Systematic Reviews 2012:CD002066.
Pan CX, Morrison RS, Ness J, Fugh-Berman A, Leipzig RM. Complementary and alternative medicine in the management of pain, dyspnea, and nausea and vomiting near the end of life. A systematic review. J Pain Symptom Manage. 2000;20:374–87.
Booth S, Wade R, Johnson M, Kite S, Swannick M, Anderson H. The use of oxygen in the palliation of breathlessness. A report of the expert working group of the Scientific Committee of the Association of Palliative Medicine. Respir Med. 2004;98:66–77.
Simon ST, Higginson IJ, Booth S, Harding R, Bausewein C. Benzodiazepines for the relief of breathlessness in advanced malignant and non-malignant diseases in adults. Cochrane Database Syst Rev. 2010;1:CD007354.
Wiseman R, Rowett D, Allcroft P, Abernethy A, Currow D. Chronic refractory dyspnoea: Evidence based management. Aust Fam Physician. 2013;42:137–40.
Mahler DA, Selecky PA, Harrod CG, Benditt JO, Carrieri-Kohlman V, Curtis JR, et al. American College of Chest Physicians consensus statement on the management of dyspnea in patients with advanced lung or heart disease. Chest. 2010;137:674–91.
Bradt J, Dileo C. Music therapy for end-of-life care. Cochrane Database Syst Rev 2010:CD007169.
Dy S, Wilson RF, Fawole OA, Lau BD, Martinez KA, Vollenweider D, et al. Improving health care and palliative care for advanced and serious illness: closing the quality gap - revisiting the state of the science. Rockville: Agency for Healthcare Research and Quality. Evidence Report/Technology Assessment; 2012.
El-Jawahri A, Greer JA, Temel JS. Does palliative care improve outcomes for patients with incurable illness? A review of the evidence. J Support Oncol. 2011;9:87–94.
Lorenz KA, Lynn J, Dy SM, Shugarman LR, Wilkinson A, Mularski RA, et al. Evidence for improving palliative care at the end of life: a systematic review. Ann Intern Med. 2008;148:147–59.
Qaseem A, Snow V, Shekelle P, Casey Jr DE, Cross Jr JT, Owens DK, et al. Evidence-based interventions to improve the palliative care of pain, dyspnea, and depression at the end of life: a clinical practice guideline from the American College of Physicians. Ann Intern Med. 2008;148:141–6.
Gomes B, Calanzani N, Curiale V, McCrone P, Higginson IJ. Effectiveness and cost-effectiveness of home palliative care services for adults with advanced illness and their caregivers. Cochrane Database Syst Rev. 2013;6:CD007760.
Currow DC, Hegarty M. Residential aged-care facility palliative care guidelines: improving care. Int J Palliat Nurs. 2006;12:231–3.
Plonk Jr WM, Arnold RM. Terminal care: the last weeks of life. J Palliat Med. 2005;8:1042–54.
Rayner L, Price A, Hotopf M, Higginson IJ. The development of evidence-based European guidelines on the management of depression in palliative cancer care. Eur J Cancer. 2011;47:702–12.
Hudson P, Remedios C, Zordan R, Thomas K, Clifton K, Crewdson M, et al. Clinical practice guidelines for the psychosocial and bereavement support of family caregivers of palliative care patients. Melbourne: Centre for Palliative Care, St. Vincent’s Hospital Melbourne; 2010.
Guidelines for a Palliative Approach in Residential Aged Care, Australian Government, national health and medical Research Council. 2006. Palliative Care Australia, Deakin West.
McCusker et al. Health care guideline: Palliative Care. Fourth Edition. Bloomington, MN: Institute for Clinical Systems Improvement; 2011.
Working group for the Clinical Practice Guidelines of Palliative Care. Clinical Practice Guideline for Palliative Care. Full version. No 2006/08. Madrid: National Plan for the NHS of the MSC. Health Technologies Assessment Agency of the Basque Country. Clinical practice guidelines in the Spanish NHS; 2008.
National Consensus Project for Quality Palliative Care. Clinical practice guidelines for quality palliative care. 3rd ed. Pittsburgh: National Consensus Project for Quality Palliative Care; 2013.
Palliative Care Guidelines. Breaking bad news. Palliative care adult network guidelines http://book.pallcare.info/index.php?tid=109. Accessed on March 25, 2015.
Palliative Care Guidelines. Depression. Palliative care adult network guidelines http://book.pallcare.info/index.php?tid=49. Accessed on March 25, 2015.
Palliative Care Guidelines: Depression. Depression in Palliative Care. NHS Lothian. Re-issue date: August 2010. Review date: August 2013.
Palliative Care Guidelines. Dyspnoea. Palliative care adult network guidelines http://book.pallcare.info/index.php?tid=41. Accessed on March 25, 2015.
Palliative Care Guidelines. Breathlessness. Breathlessness in Palliative Care. NHS Lothian. Re-issue date: August 2010. Review date: August 2013.
Palliative Care Guidelines. Pocket Edition 2010. Third edition/Version 2. NHS Lothian. Re-issue date: August 2010. Review date: August 2013.
Periyakoil VJ. Panic disorder at the end-of-life. Fast Facts and Concepts. Milwaukee: End of Life/Palliative Education Resource Center, Medical College of Wisconson; 2005;145.
Downing J, Powell T, Bausewein C, Higginson IJ. Guidelines for using the palliative care outcome scale (POS). London: King’s College London; 2012.
Collaboration PCO. Three years of progress (2010 to 2013). Wollongong: Australian Health Services Research Institute, University of Wollongong; 2013.
Palliative Care Outcomes Collaboration. Helicopter Report: Trends in patient outcomes – January 2014. Wollongong: Australian Health Services Research Institute, University of Wollongong; 2014.
Duncan E, Murray J. The barriers and facilitators to routine outcome measurement by allied health professionals in practice: a systematic review. BMC Health Serv Res. 2012;12:96.
Parker PA, Baile WF, de Moor C, Lenzi R, Kudelka AP, Cohen L. Breaking bad news about cancer: patients’ preferences for communication. J Clin Oncol. 2001;19:2049–56.
Bensing JM, Deveugele M, Moretti F, Fletcher I, van Vliet L, Van Bogaert M, et al. How to make the medical consultation more successful from a patient’s perspective? Tips for doctors and patients from lay people in the United Kingdom, Italy, Belgium and the Netherlands. Patient Edu Couns. 2011;84:287–93.
Thorne S, Hislop TG, Kim-Sing C, Oglov V, Oliffe JL, Stajduhar KI. Changing communication needs and preferences across the cancer care trajectory: insights from the patient perspective. Supportive care in cancer. Support Care Cancer. 2014;22:1009–15.
Pincock S. Poor communication lies at heart of NHS complaints, says ombudsman. BMJ. 2004;328:10.
Tamblyn R, Abrahamowicz M, Dauphinee D, Wenghofer E, Jacques A, Klass D, et al. Physician scores on a national clinical skills examination as predictors of complaints to medical regulatory authorities. JAMA. 2007;298:993–1001.
Morita T, Akechi T, Ikenaga M, Kizawa Y, Kohara H, Mukaiyama T, et al. Communication about the ending of anticancer treatment and transition to palliative care. Ann Oncol. 2004;15:1551–7.
Higginson IJ, Bausewein C, Reilly CC, Gao W, Gysels M, Dzingina M, et al. An integrated palliative and respiratory care service for patients with advanced disease and refractory breathlessness: a randomised controlled trial. Lancet Respir Med. 2014;2:979–87.
Bruera E, Kuehn N, Miller MJ, Selmser P, Macmillan K. The Edmonton Symptom Assessment System (ESAS): a simple method for the assessment of palliative care patients. J Palliat Care. 1991;7:6–9.
Cohen SR, Mount BM, Strobel MG, Bui F. The McGill Quality of Life Questionnaire: a measure of quality of life appropriate for people with advanced disease. A preliminary study of validity and acceptability. Palliat Med. 1995;9:207–19.
Osoba D, Zee B, Pater J, Warr D, Kaizer L, Latreille J. Psychometric properties and responsiveness of the EORTC quality of Life Questionnaire (QLQ-C30) in patients with breast, ovarian and lung cancer. Quality Life Res. 1994;3:353–64.
Borreani C, Bianchi E, Pietrolongo E, Rossi I, Cilia S, Giuntoli M, et al. Unmet needs of people with severe multiple sclerosis and their carers: qualitative findings for a home-based intervention. PLoS One. 2014;9:e109679.
Ford E, Catt S, Chalmers A, Fallowfield L. Systematic review of supportive care needs in patients with primary malignant brain tumors. Neuro Oncol. 2012;14:392–404.
Etkind SN, Daveson BA, Kwok W, Witt J, Bausewein C, Higginson IJ, et al. Capture, transfer, and feedback of patient-centered outcomes data in palliative care populations: does it make a difference? A systematic review. J Pain Symptom Manage. 2015;49:611–24.
Bausewein C, Booth S, Gysels M, Kühnbach R, Haberland B, Higginson IJ. Individual breathlessness trajectories do not match summary trajectories in advanced cancer and chronic obstructive pulmonary disease: results from a longitudinal study. Palliat Med. 2010;24:777–86.
Rayner L, Price A, Hotopf M, Higginson IJ. Expert opinion on detecting and treating depression in palliative care: A Delphi study. BMC Palliat Care. 2011;10:10.
Galbraith S, Fagan P, Perkins P, Lynch A, Booth S. Does the use of a handheld fan improve chronic dyspnea? A randomized, controlled, crossover trial. J Pain Symptom Manage. 2010;39:831–8.
Bausewein C, Booth S, Gysels M, Kuhnbach R, Higginson IJ. Effectiveness of a hand-held fan for breathlessness: a randomised phase II trial. BMC Palliat Care. 2010;9:22.
Higginson IJ, Evans CJ, Grande G, Preston N, Morgan M, McCrone P, et al. Evaluating complex interventions in end of life care: the MORECare Statement on good practice generated by a synthesis of transparent expert consultations and systematic reviews. BMC Med. 2013;11:111.
We would like to thank all experts who participated in the Delphi studies and/or who commented upon the implementation guidance.
European Intersectorial and Multidisciplinary Palliative Care Research Training (EURO IMPACT) is funded by the European Union Seventh Framework Programme (FP7/2007-2013, under grant agreement number ). EURO IMPACT aims to develop a multidisciplinary, multi-professional and inter-sectorial educational and research training framework for palliative care research in Europe. EURO IMPACT is coordinated by Prof. Luc Deliens and Prof. Lieve Van den Block of the End-of-Life Care Research Group, Ghent University and Vrije Universiteit Brussel, Brussels, Belgium. Other partners are VU University Medical Center, EMGO Institute for Health and Care Research, Amsterdam, the Netherlands; King’s College London, Cicely Saunders Institute, London, Cicely Saunders International, London, and International Observatory on End-of-Life Care, Lancaster University, Lancaster, United Kingdom; Norwegian University of Science and Technology, and EAPC Research Network, Trondheim, Norway; Regional Palliative Care Network, IRCCS AOU San Martino–IST, Genoa, and Cancer Research and Prevention Institute, Florence, Italy; EUGMS European Union Geriatric Medicine Society, Geneva, Switzerland; Springer Science and Business Media, Houten, the Netherlands.
UK National Institute for Health Research (NIHR) Senior investigator Award.
The NIHR Collaboration for Leadership in Applied Health Research and Care (CLAHRC) South London.
The funder had no role in the design of the study; collection, analysis and interpretation of data; in writing of the manuscript; or in the decision to submit the manuscript for publication.
The authors declare that they have no competing interests.
All authors participated in the design of the study, helped draft the manuscript and read and approved the final version of the manuscript for submission. LV carried out the Delphi study and literature searches. LV and IH drafted the recommendations and the implementation guidance, with help from RH, CB, SP.
About this article
- Clinical decision support tools
- Delphi studies
- Palliative care
- Palliative care outcome scale (POS)
- Patient reported outcome measures