The Outcome Measures in Rheumatology (OMERACT), Core Outcome Measures in Effectiveness Trials (COMET), and Core Outcome Set-STAndards for Reporting (COS-STAR) guidelines were used for developing and reporting this COS [21, 29, 30]. The scope of the COS was ’Medication review among patients aged 65 years and older with polypharmacy (≥5 daily medications) and multi-morbidity (≥2 chronic morbidities)’. The project was registered on the COMET database (http://www.comet-initiative.org/studies/details/806?result=true). Details on the study protocol have been published elsewhere . The COS-STAR checklist is detailed in Additional file 1: Table S1.
The study was performed in four countries (Belgium, Ireland, Switzerland, and the Netherlands). The medical centers included tertiary academic medical centers with a wide range of surgical and internal medicine specialties, including geriatric medicine (Belgium, Switzerland, and the Netherlands), and a secondary medical center specialized in geriatrics (Ireland). A steering committee was set up with researchers from the four participating medical centers.
Ethical approval was obtained in Belgium and Ireland. In the Netherlands and Switzerland, official ethical approval was not required, as the ethics committees confirmed that the relevant legislation was not applicable.
List of potential outcomes for inclusion in Delphi survey
As a first step, a systematic review identified all outcomes previously used or planned to be used for evaluating the effect of medication review among older patients. It was achieved through an update of a recent systematic review on medication review published by Lehnbom et al.  combined with a systematic search in randomized clinical trials registries and on the Cochrane Database. All data extractions on outcomes were performed by two independent reviewers. Any disagreement was resolved by discussion and consensus. Details have been published elsewhere .
In parallel, a qualitative study identified unknown and relevant outcomes that should be included in the Delphi survey. We held semi-structured one-to-one interviews with patients and family caregivers. Patients aged ≥ 65 taking at least five different daily medications were recruited from the geriatric outpatient clinic, the acute geriatric ward, and other medical and surgical wards in two Belgian teaching hospitals (purposive sampling). Family caregivers assisting such patients were eligible for inclusion. The interviews took the form of a discussion about the patient’s medications, the perception of risks and benefits of his/her medications, the concept of medication review, and what he/she would expect from a medication review. A topic guide was developed, pilot-tested, and used by both interviewers (AS and OD). Interviews were recorded and then transcribed verbatim. Audio recordings of the interviews were analysed using NVivo10®. Two independent researchers (a physician and a psychologist) used an interpretative approach to identify outcomes and outcome definitions based on participants’ descriptions. The analysis focused on identifying a list of outcomes that are important to older people. Analysts proceeded with coding, labelling, and indexing of data to facilitate the process of identifying relevant outcomes and outcome domains. Because it was difficult for patients to respond readily to the notion of outcome, analysts concentrated on going beyond a simple cataloguing of outcomes to form a deeper understanding of what participants wanted and expected from a medication review. They each worked individually, then compared their findings and reached consensus on a final report. Both analysts were experienced in qualitative research and blind to the systematic review findings. In a later stage, the main researcher (JBB), who performed the systematic review and led the development of the COS, reviewed the analysis reports and read the transcripts. The three researchers met to discuss findings, and any disagreement was resolved by discussion and consensus. The results, including consensus achieved by the two researchers and unsolved disagreements, were further discussed with healthcare professionals and other researchers to validate a final list of eligible outcomes for the Delphi survey.
Next, the results from the systematic review and the qualitative study were merged into a list of outcomes during a consultation exercise. Clinical experts and researchers identified overlap between outcomes and checked the medical and the plain language terms. People without medical knowledge further improved the plain language terms. The outcomes were classified into domains and areas according to the OMERACT classification . The plain language terms and explanations were tested for understandability with two older patients (aged 86 and 92) and thereby improved. This provided a final list of outcomes and definitions.
Outcomes were selected regardless of feasibility issues during the systematic review, qualitative study, and consultation exercise. The purpose was to identify all possibly relevant outcomes, particularly for patients. If a feasibility issue was stressed by participants during the Delphi survey, this issue was discussed during consensus meetings until consensus was achieved.
Delphi questionnaire survey
Three groups of stakeholders were drawn (164 individuals). Participants in each group were recruited in the four European countries by the medical centers, and additional external participants were recruited worldwide for Group 3. Group 1 consisted of family caregivers, patients aged 65 to 80, and patients older than 80. Group 2 consisted of healthcare professionals: general practitioners (GPs), community and hospital pharmacists, geriatricians, specialists in internal medicine, and nurses. Group 3 (the ’expert’ group) included medication review researchers (recruited based on their publication’s profile, assessed via Scopus®) and researchers in other relevant areas (e.g. sociologists of aging), representatives of scientific organizations, and policymakers. We aimed for proportions of 35%, 35%, and 30% of participants from Groups 1, 2, and 3, respectively. Participants in Group 1 were identified from researchers’ professional networks, local associations for older people, and personal networks. In order to ensure enough variability among patients, we purposively selected participants who varied in terms of age, gender, and practice setting. Participants in Group 2 were recruited among healthcare professionals from the academic hospital, other secondary non-academic hospitals, ambulatory care centers, and nursing homes connected or not to the local coordinating center.
The online questionnaire was developed by a company (WorldAPP®) specializing in online surveys (see http://app.keysurvey.fr/f/1038815/5ded/).Footnote 1 We expected completion of the online survey to be difficult for some older patients. All centers therefore proposed to older patients or family caregivers to complete the questionnaire with the help of a local interviewer at home or during a consultation. The other stakeholders answered it online.
The Delphi process consisted of three rounds of questionnaires. Participants who did not participate in Round 1 were not invited for Round 2, and those who did not participate in Round 2 were not invited for Round 3 . In Rounds 1 and 2, participants were asked to score each outcome on a scale from 1 (not important) to 9 (of critical importance). Outcomes considered as ’very important’ (rating 7 to 9) by ≥ 75% of participants in at least one stakeholder group were presented in the subsequent Delphi round. Additional outcomes suggested by Round 1 participants were presented in Round 2, if relevant. In Round 2, responses from Round 1 were summarized and were presented to the participants with their own results and the mean results of each stakeholder group. In Round 3, all participants were asked whether the outcomes should be systematically measured in all studies of medication review in the elderly (YES/NO). An outcome was eligible for inclusion in the COS if ≥ 75% of participants in at least one stakeholder group rated it YES in Round 3.
During the study, two deviations from the original protocol were needed. In the first, outcomes that did not meet the consensus rating for inclusion (i.e. ≥ 75% rating of 7 to 9 in at least one stakeholder group) were removed at each stage. The original protocol rule was to remove outcomes that met a consensus rating for exclusion (i.e. ≥ 75% rating of 1 to 3 in all groups). This change was motivated by high ratings by participants, comments of participants who felt the questionnaire was too long, and those of local interviewers who reported that most older participants felt tired at the end of the first questionnaire (Round 1). In the second deviation, the rating scale was replaced by a direct YES/NO question in Round 3 to further limit the number of outcomes in the final COS. Both deviations were discussed with participants during consensus meetings and with external experts and were approved by the steering committee.
After Round 1 and Round 2, feedback was retrieved by local interviewers for older patients and by comments made in the online questionnaire by some other patients, family caregivers, healthcare professionals, and the experts. Most comments made by the experts were very clear and precise. All comments were summarized and discussed during two consensus meetings to agree on the list of outcomes for the next round: (1) with healthcare professionals from the Belgian center and (2) with the steering committee. Final decisions were taken by the steering committee.
After Round 3, three consensus meetings took place, including a face-to-face meeting with patients and caregivers in Belgium, a conference call with healthcare professionals from the Belgian center, and a conference call with medication review experts and the steering committee members. Due to language issues, patients and family caregivers from other centers were not invited. The results of each round were presented, with the consensus results from Round 3 analyses used as the starting point for discussion. The goal was to comprehensively address points for discussion and to agree on the final COS. Interaction between participants from the three groups was indirect: the feedback and opinions of each were transmitted to the others by the coordinators (JBB and AS). A final consensus meeting was held by conference call for the steering committee members and primary local investigators of the OPERAM project.