Study design

The Outcome Measures in Rheumatology (OMERACT), Core Outcome Measures in Effectiveness Trials (COMET), and Core Outcome Set-STAndards for Reporting (COS-STAR) guidelines were used for developing and reporting this COS [21, 29, 30]. The scope of the COS was ’Medication review among patients aged 65 years and older with polypharmacy (≥5 daily medications) and multi-morbidity (≥2 chronic morbidities)’. The project was registered on the COMET database (http://www.comet-initiative.org/studies/details/806?result=true). Details on the study protocol have been published elsewhere [31]. The COS-STAR checklist is detailed in Additional file 1: Table S1.

The study was performed in four countries (Belgium, Ireland, Switzerland, and the Netherlands). The medical centers included tertiary academic medical centers with a wide range of surgical and internal medicine specialties, including geriatric medicine (Belgium, Switzerland, and the Netherlands), and a secondary medical center specialized in geriatrics (Ireland). A steering committee was set up with researchers from the four participating medical centers.

Ethical approval

Ethical approval was obtained in Belgium and Ireland. In the Netherlands and Switzerland, official ethical approval was not required, as the ethics committees confirmed that the relevant legislation was not applicable.

List of potential outcomes for inclusion in Delphi survey

As a first step, a systematic review identified all outcomes previously used or planned to be used for evaluating the effect of medication review among older patients. It was achieved through an update of a recent systematic review on medication review published by Lehnbom et al. [9] combined with a systematic search in randomized clinical trials registries and on the Cochrane Database. All data extractions on outcomes were performed by two independent reviewers. Any disagreement was resolved by discussion and consensus. Details have been published elsewhere [32].

In parallel, a qualitative study identified unknown and relevant outcomes that should be included in the Delphi survey. We held semi-structured one-to-one interviews with patients and family caregivers. Patients aged ≥ 65 taking at least five different daily medications were recruited from the geriatric outpatient clinic, the acute geriatric ward, and other medical and surgical wards in two Belgian teaching hospitals (purposive sampling). Family caregivers assisting such patients were eligible for inclusion. The interviews took the form of a discussion about the patient’s medications, the perception of risks and benefits of his/her medications, the concept of medication review, and what he/she would expect from a medication review. A topic guide was developed, pilot-tested, and used by both interviewers (AS and OD). Interviews were recorded and then transcribed verbatim. Audio recordings of the interviews were analysed using NVivo10®. Two independent researchers (a physician and a psychologist) used an interpretative approach to identify outcomes and outcome definitions based on participants’ descriptions. The analysis focused on identifying a list of outcomes that are important to older people. Analysts proceeded with coding, labelling, and indexing of data to facilitate the process of identifying relevant outcomes and outcome domains. Because it was difficult for patients to respond readily to the notion of outcome, analysts concentrated on going beyond a simple cataloguing of outcomes to form a deeper understanding of what participants wanted and expected from a medication review. They each worked individually, then compared their findings and reached consensus on a final report. Both analysts were experienced in qualitative research and blind to the systematic review findings. In a later stage, the main researcher (JBB), who performed the systematic review and led the development of the COS, reviewed the analysis reports and read the transcripts. The three researchers met to discuss findings, and any disagreement was resolved by discussion and consensus. The results, including consensus achieved by the two researchers and unsolved disagreements, were further discussed with healthcare professionals and other researchers to validate a final list of eligible outcomes for the Delphi survey.

Next, the results from the systematic review and the qualitative study were merged into a list of outcomes during a consultation exercise. Clinical experts and researchers identified overlap between outcomes and checked the medical and the plain language terms. People without medical knowledge further improved the plain language terms. The outcomes were classified into domains and areas according to the OMERACT classification [29]. The plain language terms and explanations were tested for understandability with two older patients (aged 86 and 92) and thereby improved. This provided a final list of outcomes and definitions.

Outcomes were selected regardless of feasibility issues during the systematic review, qualitative study, and consultation exercise. The purpose was to identify all possibly relevant outcomes, particularly for patients. If a feasibility issue was stressed by participants during the Delphi survey, this issue was discussed during consensus meetings until consensus was achieved.

Delphi questionnaire survey

Three groups of stakeholders were drawn (164 individuals). Participants in each group were recruited in the four European countries by the medical centers, and additional external participants were recruited worldwide for Group 3. Group 1 consisted of family caregivers, patients aged 65 to 80, and patients older than 80. Group 2 consisted of healthcare professionals: general practitioners (GPs), community and hospital pharmacists, geriatricians, specialists in internal medicine, and nurses. Group 3 (the ’expert’ group) included medication review researchers (recruited based on their publication’s profile, assessed via Scopus®) and researchers in other relevant areas (e.g. sociologists of aging), representatives of scientific organizations, and policymakers. We aimed for proportions of 35%, 35%, and 30% of participants from Groups 1, 2, and 3, respectively. Participants in Group 1 were identified from researchers’ professional networks, local associations for older people, and personal networks. In order to ensure enough variability among patients, we purposively selected participants who varied in terms of age, gender, and practice setting. Participants in Group 2 were recruited among healthcare professionals from the academic hospital, other secondary non-academic hospitals, ambulatory care centers, and nursing homes connected or not to the local coordinating center.

The online questionnaire was developed by a company (WorldAPP®) specializing in online surveys (see http://app.keysurvey.fr/f/1038815/5ded/).¹ We expected completion of the online survey to be difficult for some older patients. All centers therefore proposed to older patients or family caregivers to complete the questionnaire with the help of a local interviewer at home or during a consultation. The other stakeholders answered it online.

The Delphi process consisted of three rounds of questionnaires. Participants who did not participate in Round 1 were not invited for Round 2, and those who did not participate in Round 2 were not invited for Round 3 [33]. In Rounds 1 and 2, participants were asked to score each outcome on a scale from 1 (not important) to 9 (of critical importance). Outcomes considered as ’very important’ (rating 7 to 9) by ≥ 75% of participants in at least one stakeholder group were presented in the subsequent Delphi round. Additional outcomes suggested by Round 1 participants were presented in Round 2, if relevant. In Round 2, responses from Round 1 were summarized and were presented to the participants with their own results and the mean results of each stakeholder group. In Round 3, all participants were asked whether the outcomes should be systematically measured in all studies of medication review in the elderly (YES/NO). An outcome was eligible for inclusion in the COS if ≥ 75% of participants in at least one stakeholder group rated it YES in Round 3.

Protocol adaptations

During the study, two deviations from the original protocol were needed. In the first, outcomes that did not meet the consensus rating for inclusion (i.e. ≥ 75% rating of 7 to 9 in at least one stakeholder group) were removed at each stage. The original protocol rule was to remove outcomes that met a consensus rating for exclusion (i.e. ≥ 75% rating of 1 to 3 in all groups). This change was motivated by high ratings by participants, comments of participants who felt the questionnaire was too long, and those of local interviewers who reported that most older participants felt tired at the end of the first questionnaire (Round 1). In the second deviation, the rating scale was replaced by a direct YES/NO question in Round 3 to further limit the number of outcomes in the final COS. Both deviations were discussed with participants during consensus meetings and with external experts and were approved by the steering committee.

Consensus meetings

After Round 1 and Round 2, feedback was retrieved by local interviewers for older patients and by comments made in the online questionnaire by some other patients, family caregivers, healthcare professionals, and the experts. Most comments made by the experts were very clear and precise. All comments were summarized and discussed during two consensus meetings to agree on the list of outcomes for the next round: (1) with healthcare professionals from the Belgian center and (2) with the steering committee. Final decisions were taken by the steering committee.

After Round 3, three consensus meetings took place, including a face-to-face meeting with patients and caregivers in Belgium, a conference call with healthcare professionals from the Belgian center, and a conference call with medication review experts and the steering committee members. Due to language issues, patients and family caregivers from other centers were not invited. The results of each round were presented, with the consensus results from Round 3 analyses used as the starting point for discussion. The goal was to comprehensively address points for discussion and to agree on the final COS. Interaction between participants from the three groups was indirect: the feedback and opinions of each were transmitted to the others by the coordinators (JBB and AS). A final consensus meeting was held by conference call for the steering committee members and primary local investigators of the OPERAM project.

List of eligible outcomes

The systematic review identified 57 eligible outcomes from 47 published studies and 32 RCT protocols (see Additional file 2: Additional references). Fifteen interviews were conducted with patients and family caregivers. The interpretative approach identified 19 outcomes, seven of which had not been identified in the systematic review. A final list of 64 (57 + 7) eligible outcomes was obtained by merging the two lists. These outcomes were grouped into eight health domains according to the recommendations of the OMERACT Filter 2.0 (Additional file 3: Table S2).

Delphi survey

Of the 164 participants invited, 150 completed Round 1, including 55 patients or family caregivers, 55 healthcare professionals, and 40 experts. The participant characteristics are detailed in Additional file 4: Table S3. Nearly half the patients were older than 80, and more than one third of the healthcare professionals were GPs. The four centers recruited 126 participants, divided evenly between Belgium (n = 33, 26%), Ireland (n = 30, 24%), the Netherlands (n = 32, 25%), and Switzerland (n = 31, 25%). The 24 remaining participants were specialized medication review researchers from Australia (n = 3), Europe (n = 13), and North America (n = 8).

The flow of participants and outcomes is presented in Fig. 1. Of the 150 who completed Round 1, 136 completed Round 2, and 129 completed Round 3. The overall attrition rate was 14%. After Round 1, 31 outcomes did not meet the consensus rating for inclusion in any of the three groups, including the outcome ’death’. Participants proposed 32 other outcomes. Most overlapped with outcomes proposed during Round 1. One outcome expanded the concept of treatment burden, which was split into two distinct outcomes: ’drug regimen complexity’ and ’number of drugs taken daily’. The consensus meeting decided to remove 30 outcomes; ’death’ was kept for Round 2 at the request of two stakeholder groups, and the ’drug regimen complexity’ outcome was added. Thirty-five outcomes were presented to the participants in Round 2.

International consensus

Of the 11 patients or family caregivers who participated in Round 3 in Belgium, six agreed to participate in a face-to-face consensus meeting. All were women: three aged 65 to 80 and three older than 80. Of the 14 healthcare professionals who participated in Round 3 in Belgium, six (three GPs, one hospital physician, and two pharmacists) agreed to participate in a conference call consensus meeting. Due to limited availability, no researcher attended the consensus meeting. However, three specialized medication review researchers agreed to participate in a one-hour conference call to discuss the results.

The comments and consensus obtained during the meetings and conference calls were unambiguous for 11 of the 17 outcomes reviewed. Four were definitively included and seven removed. The six remaining outcomes were discussed before consensus and a final decision were reached (Table 1). For example, 89% of participants in the patient group had selected ’pain relief’ for inclusion in the COS, whereas 77% of participants in the expert group had rejected it. After discussion, a consensus was reached on including this outcome in the final COS. A summary of the discussions for these six outcomes is given in Additional file 5: Table S4.

Final core outcome set

Summary

This study aimed to identify outcomes of great relevance to clinical trials of medication review in multi-morbid older patients with polypharmacy from different stakeholders’ perspectives. For this purpose, the study used methodological guidance of initiatives like COMET and OMERACT [21, 29] and a Delphi survey of 129 participants from four European countries along with experts from three continents. The opinions of old and very old patients, healthcare professionals, and experts were acknowledged equally, a feature lacking in most COS research. Consensus was achieved on seven outcomes to include in the COS, which can be recommended for future use in clinical trials.

Discussion of the included and excluded outcomes

Implications for future research

Effectiveness of medication review in older patients has most often been evaluated through outcomes related to healthcare use and mortality, such as all-cause hospitalizations, length of hospital stay, emergency department visits, and all-cause deaths [11, 15, 16, 20]. These outcomes were not considered essential by most stakeholders in our Delphi survey, including older patients. This result highlights the importance of developing a COS that avoids reductive conclusions based on outcomes selected by researchers only. Most outcomes included in our COS have been underreported in previous RCTs of medication review. We contend, therefore, that efforts should be made in the future to provide data on these outcomes, to determine more comprehensively the effectiveness of medication review in older patients. We did not ask participants to consider the feasibility of measuring the outcomes during the Delphi survey. Future research should determine which measurement tool is suitable for each outcome.

Strengths and limitations

This is the first Delphi study to explore international, multi-stakeholder, and multidisciplinary consensus on core outcomes to be reported in clinical trials of medication review in older patients with multi-morbidity and polypharmacy. The strengths of this study include the following: (1) use of methods following the guidance of initiatives like COMET and OMERACT [21, 29]; (2) the large number of older and very old patients involved; (3) interviews with very old patients to increase participation; (4) a large expert panel of stakeholders representing various disciplines and countries; (5) giving participants the opportunity to comment on each choice; (6) low attrition rate; (7) rigorous reporting of methods and results according to COS-STAR guidelines [30].

Our study has some limitations. Departing from the protocol, we had to adapt the methods which were used to obtain consensus during the study. These changes were discussed and validated with the steering committee, the OPERAM research team, and two external experts in COS development. This was in line with the existing guidelines for COS development, which do not recommend one specific method but state that several methods can be used and mixed if necessary to achieve consensus according to the specific needs of the study [21, 29]. Moreover, Delphi surveys are flexible, with various recognized ways of measuring consensus [33]. Our choices were made in order to combine scientific rigor with pragmatism and relevance. Finally, patients and caregivers included in the qualitative study and in the consensus meeting were only from Belgium. The health-seeking culture and attitudes towards polypharmacy may vary across countries, and some comments or consensus achieved by Belgian participants may be specific to their culture or the Belgian health organization. However, the Delphi study included numerous patients and healthcare professionals from the four countries involved in the study, in addition to experts from several countries, which implies a real international perspective in the development of this COS.