Providing multimedia information to children and young people increases recruitment to trials: pre-planned meta-analysis of SWATs
BMC Medicine volume 21, Article number: 244 (2023)
Randomised controlled trials are often beset by problems with poor recruitment and retention. Information to support decisions on trial participation is usually provided as printed participant information sheets (PIS), which are often long, technical, and unappealing. Multimedia information (MMI), including animations and videos, may be a valuable alternative or complement to a PIS. The Trials Engagement in Children and Adolescents (TRECA) study compared MMI to PIS to investigate the effects on participant recruitment, retention, and quality of decision-making.
We undertook six SWATs (Study Within A Trial) within a series of host trials recruiting children and young people. Potential participants in the host trials were randomly allocated to receive MMI-only, PIS-only, or combined MMI + PIS. We recorded the rates of recruitment and retention (varying between 6 and 26 weeks post-randomisation) in each host trial. Potential participants approached about each host trial were asked to complete a nine-item Decision-Making Questionnaire (DMQ) to indicate their evaluation of the information and their reasons for participation/non-participation. Odds ratios were calculated and combined in a meta-analysis.
Data from 3/6 SWATs for which it was possible were combined in a meta-analysis (n = 1758). Potential participants allocated to MMI-only were more likely to be recruited to the host trial than those allocated to PIS-only (OR 1.54; 95% CI 1.05, 2.28; p = 0.03). Those allocated to combined MMI + PIS compared to PIS-only were no more likely to be recruited to the host trial (OR = 0.89; 95% CI 0.53, 1.50; p = 0.67). Providing MMI rather than PIS did not impact on DMQ scores. Once children and young people had been recruited to host trials, their trial retention rates did not differ according to intervention allocation.
Providing MMI-only increased the trial recruitment rate compared to PIS-only but did not affect DMQ scores. Combined MMI + PIS instead of PIS had no effect on recruitment or retention. MMIs are a useful tool for trial recruitment in children and young people, and they could reduce trial recruitment periods.
High-quality randomised controlled trials involving children and young people (CYP) are essential to ensure that interventions are safe and effective [1,2,3,4]. However, there is a lack of trial evidence, partly caused by poor recruitment, causing delayed completion and/or discontinuation [5, 6], and participant attrition [7, 8]. This contributes to trial costs  and research waste [10, 11]. Recruitment of CYP to trials is complicated by issues with consent and assent, judgements of ‘competence’, and different legal requirements for Clinical Trial of an Investigational Medicinal Product (CTIMP) and non-CTIMP trials [12, 13]. CYP may often be in a better position than their parent/carer to envisage what participation will mean [14, 15] and so excluding them from decision-making may lead to misunderstandings and subsequent withdrawal. Finally, when parents or carers are consenting on behalf of a child, the consent threshold tends to be higher , and parents or carers and healthcare professionals both tend to be more risk-averse (opting for standard care rather than a trial more often than they would for an adult) . In 2013, the UK Chief Medical Officer  called for researchers to ‘…work with children and young people to input to the design of clinical studies…to facilitate (their) increased participation in trials’, an approach that has been undertaken in this study.
Information plays a crucial role in decision-making about trial participation . In most cases, information is provided to potential participants as a printed participant information sheet (PIS); these have received prolonged criticism for being lengthy, technical, unappealing, and hard to navigate [20,21,22,23]. People with lower levels of literacy can find them especially difficult . The UK Health Research Authority has encouraged researchers to use shorter PIS in low-risk research and explore the use of non-print media . Providing multimedia information (MMI) through digital platforms could include animations, ‘talking-head’ videos, diagrams, photos, and written text [24, 26, 27].
In healthcare practice, MMIs have mostly been more effective than print for informing patients’ knowledge [28,29,30,31] such as about medical procedures [32,33,34,35,36,37,38,39,40]. The evidence in research recruitment is limited [26, 27], with only one study in CYP, which reported a greater understanding of trials from MMI . In adults, the evidence is mixed: studies have reported increased understanding and knowledge of the trial  and positive participant evaluations , but another study reported that MMI and PIS produced similar trial recruitment rates .
MMIs have several potential advantages over PIS, including the following:
Choice of the order in which aspects of the MMI are viewed
Increased content choice, allowing some user personalisation
Delivery by sound allows provision to those with sight impairment
Concurrent delivery in sound and vision, facilitating user engagement and understanding, and reduced cognitive load 
We suggest that positive effects of MMI on recruitment might be seen through more effective attention to, and use of the information, producing beneficial effects on knowledge, attitudes to trials, and decision confidence (Fig. 1).
The science of trial recruitment and retention is being developed through ‘Study Within A Trial’ (SWAT) methods, which are self-contained studies embedded within a host trial to evaluate alternative ways to deliver or organise trial processes [45,46,47]. The pre-planned coordination of SWATs is a recent development, first undertaken in MRC-START , and the approach has four main strengths:
Increased sample size and certainty of findings
Greater researcher control over participant sampling, outcomes, and (especially) interventions
Cost-efficiency, depending on the development time for the SWAT intervention(s)
The pre-planned approach to SWATs was undertaken in this study.
To evaluate MMI templates in a series of SWATs set within trials in CYP, to test their effects on recruitment and retention, and participant decision-making, by comparing the provision of MMIs to PIS, and the provision of MMIs in addition to PIS.
SWATs were embedded in a series of six host randomised control trials recruiting CYP. Within each SWAT, potential trial participants were randomised to receive trial recruitment information in one of three formats: MMI-only, PIS-only, or combined MMI + PIS (Fig. 2). Separate randomisation was done by each host trial, whereby their respective participants or recruiting sites (if cluster) were randomised to receive the information they had chosen to use in their trial. Details of the randomisation (ratios, clustering, stratification), and the arms included in each SWAT, can be found in Tables 1 and 2. Data from the SWATs were analysed and combined in a pre-planned two-stage meta-analysis.
Recruitment of host trials
We publicised the Trials Engagement in Children and Adolescents (TRECA) study through printed flyers and email to UK trial centres and the UK Trial Managers’ Research Network. We emphasised that the cost of developing the MMI would be borne by TRECA, and we also offered host trials modest financial support (£1500) to compensate for additional administrative work. Interested trials were selected based on the following:
Recruitment timing (within the TRECA period)
Not currently using MMI, website, or video in trial recruitment
Recruiting CYP capable of at least some involvement in consent decisions (set as age 6 years and above)
Ideally, we wanted host trials to cover a range of health conditions and intervention types and be recruiting CYP across a range of ages. Given the historical prominence of oncology trials in CYP trial research , we preferred at least one host trial to have that setting.
Finally, the sample size calculation for the meta-analysis (see below) meant that host trials needed to approach an average of at least 329 CYP about participation. It is important to note that in these recruitment SWATs, the outcome of interest is the number of participants recruited out of the number of people approached to take part in the host trial. The people to be approached were randomly allocated to different recruitment methods, and the outcome is calculated according to the proportion of people recruited to the host trial from each arm in the SWAT.
Host trials were offered SWATs with two arms (MMI-only versus PIS-only; or MMI-only versus PIS + MMI) or three arms (MMI-only versus PIS-only versus PIS + MMI).
Interventions used in the TRECA study are listed in Table 2. The PIS was the Research Ethics Committee-approved information being used in each host trial; the PIS content for each trial was not modified. The decision on who read the PIS (i.e. parent, child, both together, both separately) was made by individual participants.
The MMI for each trial was based on a template developed through co-design and extensive empirical work during the first year of the TRECA study, comprising a qualitative study with stakeholder groups ; a user testing study with CYP, parents, and carers ; and close collaboration with the TRECA Patient and Parent Involvement group (PPI) . Two templates were developed (one for children aged 6–11 years and a second for children aged 12–18 years and parents). The MMIs were developed by a website and video production company (Morph Studios Ltd.) and contained all the content from the PIS, organised into six sections within the MMI (home page, about the trial, taking part, after the trial, questions, contacts) plus five short video animations with voiceover (one that was trial-specific, summarising the trial; four that were trial-generic, explaining different aspects of trials), and a series of short ‘talking heads’ videos (e.g. the trial principal investigator or a young participant talking about aspects of the trial including the experience of participation). None of the video clips included subtitles. The number of ‘talking heads’ videos varied among the SWATs (see Table 3 for links to example MMIs). The written text of each MMI was amended for clarity through readability indices . The decision on who viewed the MMI (i.e. parent, child, both together, both separately) was made by individual participants.
The primary outcome was trial recruitment in the MMI-only versus the PIS-only arms, to test the effect of replacing printed information with MMI. Trial recruitment was assessed by the number of eligible participants who were recruited to the host trial, according to the allocated SWAT arm.
The secondary outcomes were as follows:
Trial recruitment in the MMI-only versus the combined PIS + MMI arms, to test the effect of providing MMI in addition to PIS
Trial retention in the MMI-only versus the PIS-only arms
Trial retention in the MMI-only versus the combined PIS + MMI arms
Trial retention was assessed by the number of participants in the host trial who were retained in the trial at follow-up. When a trial had multiple primary outcome time points, we selected one to maximise similarity across the host trials.
Decision-Making Questionnaire (DMQ) scores in the MMI-only versus the PIS-only arms
DMQ scores in the MMI-only versus the combined PIS + MMI arms
Decision-making (i.e. evaluation of the information plus reasons for participation/non-participation) was assessed by the mean score on the Decision-Making Questionnaire (DMQ) scale (see Supplementary Materials 4 for the three versions of the DMQ). The DMQ measured the quality of decision-making by potential participants and was developed for the study because currently available measures were intended for adults or measured decisions about treatments. When a parent/carer was involved in the decision, we also asked them to complete the scale separately. The scale was adapted to facilitate completion by younger CYP. We aimed to obtain DMQ scores both from individuals who decided to participate in the host trial and those who declined. In those who decided to take part, the CYP and/or parent/carer were asked to complete the DMQ once the host trial participation documentation was completed, or it was emailed to them. In CYP who declined participation, they were asked to complete the DMQ in the clinic or it was posted or emailed to home, as appropriate.
The DMQ versions for older CYP (intended for ages 12 and over) and parents or carers contained the same number of questions, with slight changes in question phrasing. It contained nine questions (with five fixed-response options), and there were a further three free-text questions. The younger CYP version of the DMQ (intended for ages 6–11) was of a similar format, comprising three questions with fixed responses and three ‘free-text’ questions.
Answers to fixed-response questions were allocated values of 0–4. The values for each question were summed to create an overall score out of 36 (or out of 12 for the younger CYP version), in which higher scores represented more positive evaluations. Up to three missing responses were allowed on the 9-question scale, and one was allowed on the three-question version. A total score was calculated by replacing missing values with the mean score from the completed responses given by the participant. Any questionnaires with more than three (older CYP/parent/carer version) or one (younger CYP version) missing values, were not scored.
Trial and SWAT registration
The TRECA study was registered on the ISRCTN registry (ISRCTN73136092) and the Northern Ireland Hub for Trials Methodology Research SWAT Repository (SWAT 97). The SWATs were undertaken through amendments to Research Ethics Committee approvals obtained by the host trial research teams.
Sample size calculation
The TRECA sample size was based on the meta-analysis of the recruitment data from each SWAT on the primary outcome (MMI-only versus PIS-only). We assumed 80% power at 5% type I error (alpha rate).
Assuming the baseline recruitment rate (in the PIS-only arm) was 80%, to detect an increase to 88% in the MMI arm in a single randomised controlled trial (RCT) with 1:1 randomisation between arms, n = 329 per group was needed. We multiplied this number by three to account for the 3-arm randomisation in the SWATs (n = 987). We assumed the heterogeneity in observed effect across the trials (I 2 statistic) would be 50%, having the effect of doubling the sample size, deriving an overall sample of 1974 across the six SWATs. This calculation has been updated from that included in the initial published protocol, due to an inability to reproduce the calculation. However, this does not impact on the validity of the results as the sample size of the overall project was driven by the individual SWATs that were undertaken, which could not be pre-determined, and which had their own recruitment targets.
All analyses were conducted in STATA v16, following the principles of intention-to-treat (ITT) with participants’ outcomes analysed according to their original, randomised group. A modified ITT (mITT) approach was used, whereby any participants who were subsequently found to be ineligible for the host trial, were not included in the analysis. Where it was possible for a participant to receive a SWAT allocation different to what they were randomised to (for example, randomised to MMI, but received PIS), we also undertook a per-protocol analysis; these results are not presented and can be found in the main publication . The analysis, outcomes, and significance levels were pre-specified in a statistical analysis plan (see Supplementary Material 1) before the analysis was conducted. Model assumptions were checked.
As there was a single primary outcome, we did not need to adjust for multiplicity issues arising from the PIS arm being in all comparisons.
All participant baseline data were summarised descriptively by the SWAT arm, and no formal statistical comparisons were undertaken. Continuous measures were reported as means and standard deviations (after normality was checked), and categorical data were reported as counts and percentages. Baseline data for the host trials varied due to different trial data collection; all data that were collected have been reported. Baseline data were only available for participants who were randomised into each host trial (see Supplementary Material 2).
Recruitment: MMI-only versus PIS-only
Recruitment rates were compared using logistic regression, undertaken separately for each SWAT, with SWAT allocation (MMI or PIS) included as a covariate. The results from the regression have been presented as odds ratios, with associated 95% confidence intervals and p-values.
Recruitment: MMI + PIS versus PIS-only
This analysis was undertaken in the same way as the primary analysis, including only those trials which included a combined MMI + PIS arm.
Retention was measured as the proportion of completed follow-ups, i.e. the number of participants retained at the first follow-up measurement of the host trial primary outcome variable, divided by the number who were due a follow-up. The retention rate was compared using logistic regression, with SWAT allocation and host trial allocation (for FORCE and CHAMP-UK only) included as covariates. When a host trial used stratification variables (Table 1) in the randomisation, these were included as covariates wherever possible. As for the recruitment analyses, two pairwise comparisons were used: MMI-only versus PIS-only, and MMI + PIS versus PIS-only. The results have been presented as odds ratios, with associated 95% confidence intervals, and p-values.
Quality of decision-making questionnaires
The responses to each question (including the number of missing responses) and the calculated total scores of the DMQ were summarised descriptively overall and presented by host trial, SWAT allocation and type of questionnaire (younger CYP, older CYP, or parent/carer).
As CYP and their parent/carer may have both completed a questionnaire, data from all three questionnaires were not combined, due to the lack of independence. Hence, scores for CYP (younger or older) and parents/carer questionnaires were analysed separately using linear regression, with SWAT allocation and host trial status (whether the participant was recruited) included as covariates. The mean difference has been presented with 95% confidence intervals.
The DMQ results were compared using a regression model for each SWAT, adjusted for SWAT allocation, and whether CYP consented to participate in the host trial. To assess the robustness of the method used to replace the missing values, a sensitivity analysis was conducted, where the analysis was repeated using only the questionnaires in which all nine questions were answered.
For all the above analyses, if the SWAT was cluster randomised, the analysis included cluster as a random effect.
The results from each SWAT were combined in meta-analyses. A two-stage random-effects meta-analysis was used in each case, where the results from each model were combined using an inverse-variance approach; no further adjustments were made. For each of the outcomes, any available data were combined using a meta-analysis.
Six host trials were recruited to TRECA: FORCE, CHAMP-UK, Thermic-3, BALANCE, BAMP, and UKALL-2011 (Tables 1 and 2), recruiting CYP with a variety of ages and health conditions in the UK (Table 1). Three of the SWATs generated sufficient data for logistic regression models, which were combined in a pre-planned statistical meta-analysis (FORCE, CHAMP-UK, Thermic-3). Unfortunately, three SWATs were significantly affected by adverse circumstances: the BAMP trial closed early due to funding restrictions; the UKALL-2011 SWAT took many months to approve, which limited recruitment before the host trial closed; and recruitment to the BALANCE trial was closed for several months due to the COVID-19 pandemic. Consequently, it was not possible to run logistic regression models for these three SWATs, due to being too small (2 trials) or having insufficient variation in outcomes (1 trial). Descriptive results are provided for each of the trials. However, the meta-analysis was based on the three SWATs where models could be fitted (total n = 1758).
The pathway through each SWAT is displayed in Fig. 3 and the results of the six individual SWATs are reported in Tables 4, 5, and 6. The results of the FORCE and Thermic-3 SWATs have already been reported individually [55, 56], and the results of the CHAMP-UK SWAT are reported in Supplementary Material 3.
Meta-analysis of SWAT data
The pooled results of the mITT data show that those CYP who received MMI-only information were more likely to be recruited into a trial than those who received PIS-only information: pooled odds ratio (OR) = 1.54 (95% CI: 1.05, 2.28; p = 0.03). The width of the 95% confidence interval indicates considerable uncertainty about the true effect. The statistical heterogeneity (I 2) in the meta-analysis was 0% (Fig. 4).
The meta-analysis comparing PIS-only versus the combined MMI + PIS arms included data from only two SWATs (CHAMP-UK and Thermic-3) because the FORCE SWAT did not include a combined MMI + PIS arm. There was no significant difference between the two arms: pooled OR = 0.89 (95% CI: 0.53, 1.50; p = 0.67).
Trial retention rates
Retention was recorded between 6 and 26 weeks post-randomisation, depending on the trial. The retention rates of CYP recruited to host trials in the MMI-only and PIS-only arms were similar: pooled OR = 1.29 (95% CI: 0.36, 4.65; p = 0.70). Statistical heterogeneity was 0% (Table 5).
The retention rate was higher in the MMI + PIS arm than in the PIS-only arm, but sample sizes were small and the difference was not statistically significant: pooled OR = 2.18 (95% CI: 0.48, 10.00; p = 0.31). Statistical heterogeneity was 0%.
Quality of decision-making questionnaires
The meta-analysis of the DMQs is limited to the parent/carer (P/C) version of the questionnaire because there were insufficient data for meta-analyses of the younger and older CYP versions. The results of the younger and older CYP versions are given descriptively where possible (see Table 6).
Overall, for the parent/carer version those in the MMI-only arm had similar (but slightly lower) DMQ scores to those in the PIS-only arm: pooled adjusted mean difference (AMD) = − 0.79(95% CI: − 2.80, 1.22; p = 0.44). Statistical heterogeneity was moderately high (53.6%).
Participants in the combined MMI + PIS arm had lower DMQ scores than those in the PIS-only arm: pooled AMD = − 2.07 (95% CI: − 4.13, 0.01; p = 0.05), which is of borderline statistical significance. Statistical heterogeneity was 0%.
Brief summary of findings
MMI provision, rather than standard PIS, led to higher recruitment rates of CYP to trials. There was no effect on trial retention. Providing combined MMI + PIS resulted in no benefit on trial recruitment or retention rates compared to PIS alone.
The provision of MMI rather than PIS produced no effect on DMQ scores. However, the provision of combined MMI + PIS resulted in less favourable evaluations than from PIS alone.
Strengths and limitations
The MMIs were developed through extensive empirical work, to ensure that their content, layout, and appearance would best meet the preferences and needs of CYP, parents/carers, healthcare practitioners, and researchers. Two templates were developed (one for children aged 6–11 and one for children aged 12–18 and parents), based on estimated literacy and cognitive maturity. Finer-grained age distinctions would be possible in MMIs although there would be both financial and practical implications.
The six host trials recruited CYP with a range of ages and health conditions across the UK, although we found that many otherwise eligible host trials were too small to include in TRECA. Unfortunately, we were able to include data from only three of the SWATs in the meta-analysis. However, the pre-planned meta-analysis increased the available sample size for evaluating the effects of MMIs, and allowed the comparison of similar interventions (i.e. template-driven MMIs) across several trials.
The study faced several challenges , the main effect of which was to reduce the size of the meta-analysis dataset. There was a good response to publicity about TRECA, but some potential host trials did not meet our criteria. In addition, trial recruitment postponements due to COVID-19 affected three SWATs, reducing the dataset size in two of them (by around n = 200 in total) and delaying the meta-analysis. Unfortunately, any adverse circumstance affecting host trial progress will almost always affect SWATs. On the other hand, the sample size estimate assumed a meta-analysis heterogeneity value of 50%, but in fact, a value of 0% was obtained, increasing power.
The DMQ return rates were around half that anticipated, reducing statistical precision. The questionnaire return rate was much higher in CYP recruited to host trials than in those who declined participation. It constitutes a significant limitation to the findings; potentially, it was also a source of bias in the DMQ dataset. Sending questionnaires via website or email (as happened in the CHAMP-UK trial), rather than providing printed versions, could have increased return rates, but this approach has not been evaluated . We could also have sought other data sources on recruitment. For example, we could have interviewed people who did or did not consent in host trials, or clinical and research staff responsible for trial recruitment, although this approach would have had its own challenges.
Finally, a common limitation with SWATs is the application of host trial entry criteria to patient eligibility after the SWAT random allocation of individuals has happened. The effect is to lose SWAT participants and reduce the fidelity of the SWAT randomisation; consequently, a modified intention-to-treat (mITT) analysis was required in this study. In all, 18 participants were omitted from the analysis for being ineligible for the host trial (7 CHAMP-UK, 1 UKALL, 4 BALANCE, 6 THERMIC-3).
What this study adds
This study contributes to the growing evidence base of RCT-level evidence for interventions targeting trial recruitment and retention , particularly in CYP, which is currently lacking.
The retention rates of CYP at follow-up in the trials are reassuring, indicating that increased recruitment through MMI was not achieved at the expense of understanding the host trial when being recruited.
One aim of the study was to produce an MMI template for trial recruitment, in part to ensure that all the TRECA SWATs would be testing similar multimedia information, and to allow future practitioners and researchers to use the template in trials if the MMI was shown to be effective and/or acceptable. That opportunity is now available, and access can be provided on request.
There is growing evidence on the use of MMI in research recruitment and healthcare, although few interventions have been evaluated within SWATs using random allocation. As such, this study makes a meaningful contribution.
Finally, this study shows that using MMI in research recruitment is achievable and acceptable. Our view is that a hesitancy among researchers towards digital or multimedia information stems from two concerns:
First, a view that Research Ethics Committees will be reluctant to approve non-print participant information. Notably, we did not encounter any such resistance during this study.
Second, that MMI is too difficult, expensive, or time-consuming to produce. Depending on its quality and complexity, a trial MMI currently costs £10,000–£15,000. The pooled absolute recruitment rate in this study was 4.8% higher in the MMI-only arm (68.7% versus 63.9%, see Table 4), which would equate to a relative reduction of 7% in the recruitment period in the host trials, potentially reducing study length and cost. If replicated, the use of MMIs in trials could be cost-neutral or even cost-saving. However, no formal health economic analysis was undertaken.
Implications of the study
This study demonstrated a benefit of MMI on trial recruitment in CYP, although the relevant evidence base is small and further research is needed. Furthermore, the effect of MMIs on recruitment generated an odds ratio statistic with the lower end of the confidence interval not far above 1 (with an associated probability of 0.03) and so there remains considerable uncertainty around its true effect. That is particularly the case for CYP, but it also applies to trials recruiting adults.
The combined provision of MMI + PIS resulted in no gains, and possibly less positive questionnaire evaluations. However, the small available dataset for this question greatly lessens the certainty of this finding. Given the time demands and complexity of generating participant information in two formats, this study does not suggest that using both MMI and PIS is a better option, although it would be helpful for further research to assess this point.
One SWAT could not provide participants with MMI access as planned due to poor hospital internet connectivity . Consequently, in that SWAT, participants were given the MMI URL on a laminated card, to access at home, increasing the risk of non-access. A solution could be through social media or email/text message communication between the healthcare provider and patients, although there are several barriers (ethical and practical) that may need to be overcome.
One aim was to assess the relative impacts of MMI and PIS on more deprived and/or less health-literate populations. Suboptimal or complex patient information tends to have a disproportionately negative effect on people with low literacy (including low health literacy) . Unfortunately, we were unable to examine these aspects. Demographic baseline data are generally only available for those who have consented to the host trial. Due to research ethics restrictions, data access is rarely permitted for people who have declined trial participation, placing a significant limit on assessing equity within ‘recruitment science’. There remains a need to assess whether carefully developed MMI has similar benefits on understanding and decision-confidence across population groups. Easy internet access in the population is commonplace but not universal. There is a likelihood of deepening of inequities if information is exclusively provided via MMIs, or both MMIs and PIS are provided but MMI content has greater clarity. MMIs may also have negative associations for some users: video and animations can appeal and engage, although these effects must not be achieved through superficiality or imbalance. Content in different formats may lead to duplication or ‘information redundancy’ . Finally, a recent review identified that CYP with long-term health conditions may associate digital health technologies with concerns about privacy, trust, and confidentiality .
Information plays an essential role in research consent. Valid participation decisions (whether positive or negative) must be informed, and the person taking consent has an ethical duty to be sure that the participant is making an informed decision. The role of MMI in individual consent decisions, and in allowing recruiters to fulfil their ethical duty, is an area in need of research. However, the external context is changing: people are increasingly expecting digital provision of information, particularly younger people, and digital provision may also be altering content expectations [61, 62]. These are two of the factors which may increase participant acceptance and use of MMI in trial recruitment.
This study has generated some relevant findings from the DMQs, although low return rates (particularly from those declining trial recruitment) mean that there is considerable uncertainty about MMI evaluations. For example, the findings of a SWAT undertaken within a hypothetical trial were that the TRECA-generated MMI produced higher appraisals of ‘information was easy to understand’ and ‘I had confidence in decision-making’ than did PIS provision . There is also a need for research with adults and CYP around the effects of MMI on communication between potential participants and those recruiting them; MMIs may impact recruiter-participant interaction, but this needs evaluation.
We should also know more about people’s use of MMIs to inform decisions, to ensure they are being informed about research and not just being entertained. Furthermore, it is unclear whether all the components of the MMI are required to achieve benefits. Two concerns are that:
Animations and video are relatively expensive components of MMIs, which may create a barrier in less well-funded trials.
The inclusion in MMIs of all the written PIS text plus information content within the animations and videos allows choice but may create information duplication . Analysis of the use of the various components of MMIs could indicate whether duplication and/or redundancy are apparent.
The overall effect of the MMIs in TRECA was to increase recruitment in children and young people (indicated by a meta-analysis of three SWATs with negligible statistical heterogeneity), which indicates their potential as a recruitment tool. However, given the width of the confidence intervals, and a dataset based on just three SWATs, there remains a need for more research to produce a more precise estimate of how well MMIs work and in what settings. The TRECA dataset on combined MMI + PIS was small and further research would increase the certainty on its effects. Finally, we need to explore what components of MMIs are associated with benefits, and whether MMIs are cost-effective.
Providing only multimedia information increased the trial recruitment rate compared to providing only printed information, but it did not affect people’s evaluation of the information. Among children and young people recruited to host trials, trial retention rates did not differ according to intervention allocation. Providing combined multimedia and printed information instead of printed information only had no effect on recruitment or retention. Multimedia information is a useful tool for trial recruitment in children and young people, and it could reduce trial recruitment periods.
Availability of data and materials
Upon request, and subject to review, the authors will provide the data that support the findings of this study.
Adjusted mean difference
Coronavirus disease 2019
Children and young people
Participant information sheet
Randomised controlled trial
Study Within A Trial
Trials Engagement in Children and Adolescents
United States of America
Joseph PD, Craig JC, Caldwell PH. Clinical trials in children. Br J Clin Pharmacol. 2013;79(3):357–69. https://doi.org/10.1111/bcp.12305.
Chappuy H, Doz F, Blanche S, et al. Children’s views on their involvement in clinical research. Pediatr Blood Cancer. 2008;50(5):1043–6.
Rocchi F, Tomasi P. The development of medicines for children. Part of a series on Pediatric Pharmacology, guest edited by Gianvincenzo Zuccotti, Emilio Clementi, and Massimo Molteni. Pharmacol Res. 2011;64(3):169–75. https://doi.org/10.1016/j.phrs.2011.01.016.
Klassen TP, Hartling L, Craig JC, et al. Children are not just small adults: the urgent need for high-quality trial evidence in children. PLoS Med. 2008;5(8):e172. https://doi.org/10.1371/journal.pmed.0050172.
Srivastava A, Bourgeois FT. Evaluation of publication of pediatric drug trials. JAMA Netw Open. 2021;4(4):e215829–929. https://doi.org/10.1001/jamanetworkopen.2021.5829.
Pica N, Bourgeois F. Discontinuation and nonpublication of randomized clinical trials conducted in children. Pediatrics. 2016;138(3):e20160223. https://doi.org/10.1542/peds.2016-0223.
Walters SJ, Bonacho dos Anjos Henriques-Cadby I, Bortolami O, et al. Recruitment and retention of participants in randomised controlled trials: a review of trials funded and published by the United Kingdom Health Technology Assessment Programme. BMJ Open. 2017;7(3):e015276. https://doi.org/10.1136/bmjopen-2016-015276.
Bruzzese JM, Gallagher R, McCann-Doyle S, et al. Effective methods to improve recruitment and retention in school-based substance use prevention studies. J Sch Health. 2009;79(9):400–7. https://doi.org/10.1111/j.1746-1561.2009.00427.x.
Kitterman DR, Cheng SK, Dilts DM, et al. The prevalence and economic impact of low-enrolling clinical studies at an academic medical center. Acad Med. 2011;86(11):1360–6. https://doi.org/10.1097/ACM.0b013e3182306440.
Moher D, Glasziou P, Chalmers I, et al. Increasing value and reducing waste in biomedical research: who’s listening? Lancet. 2016;387(10027):1573–86. https://doi.org/10.1016/s0140-6736(15)00307-4.
Ioannidis JP, Greenland S, Hlatky MA, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383(9912):166–75. https://doi.org/10.1016/s0140-6736(13)62227-8.
The General Medical Council. 0–18 years: guidance for all doctors. 2007. https://www.gmc-uk.org/ethical-guidance/ethical-guidance-for-doctors/0-18-years (Accessed 4 Oct 2022).
UK Medical Research Council. MRC ethics guide - medical research involving children. 2004. https://mrc.ukri.org/documents/pdf/medical-research-involving-children/ (Accessed 5 Dec 2022).
Coyne I. Research with children and young people: the issue of parental (proxy) consent. Child Soc. 2010;24(3):227–37.
Luchtenberg M, Maeckelberghe E, Locock L, et al. Young people’s experiences of participation in clinical trials: reasons for taking part. Am J Bioeth. 2015;15(11):3–13.
Caldwell PH, Butow PN, Craig JC. Parents’ attitudes to children’s participation in randomized controlled trials. J Pediatr. 2003;142(5):554–9. https://doi.org/10.1067/mpd.2003.192.
Caldwell PH, Murphy SB, Butow PN, et al. Clinical trials in children. Lancet. 2004;364(9436):803–11. https://doi.org/10.1016/s0140-6736(04)16942-0.
Health Do. Chief Medical Officer’s annual report 2012: our children deserve better: Prevention Pays. 2013.
Sheridan R, Martin-Kerry J, Hudson J, et al. Why do patients take part in research? An overview of systematic reviews of psychosocial barriers and facilitators. Trials. 2020;21(1):259. https://doi.org/10.1186/s13063-020-4197-3.
Tarnowski KJ, Allen DM, Mayhall C, et al. Readability of pediatric biomedical research informed consent forms. Pediatrics. 1990;85(1):58–62.
Ogloff JR, Otto RK. Are research participants truly informed? Readability of informed consent forms used in research. Ethics Behav. 1991;1(4):239–52. https://doi.org/10.1207/s15327019eb0104_2.
Eder ML, Yamokoski AD, Wittmann PW, et al. Improving informed consent: suggestions from parents of children with leukemia. Pediatrics. 2007;119(4):e849–59. https://doi.org/10.1542/peds.2006-2208.
Caldwell PH, Dans L, de Vries MC, et al. Standard 1: consent and recruitment. Pediatrics. 2012;129(Suppl 3):S118–23. https://doi.org/10.1542/peds.2012-0055D.
Tait AR, Voepel-Lewis T. Digital multimedia: a new approach for informed consent? JAMA. 2015;313(5):463–4. https://doi.org/10.1001/jama.2014.17122.
Health Research Authority. Applying a proportionate approach to the process of seeking consent: HRA Guidance. 2016.
Hutchison C, Cowan C, McMahon T, et al. A randomised controlled study of an audiovisual patient information intervention on informed consent and recruitment to cancer clinical trials. Br J Cancer. 2007;97(6):705–11. https://doi.org/10.1038/sj.bjc.6603943.
Shneerson C, Windle R, Cox K. Innovating information-delivery for potential clinical trials participants. What do patients want from multi-media resources? Patient Educ Couns. 2013;90(1):111–7. https://doi.org/10.1016/j.pec.2012.06.031.
Savage I, Goodyer L. Providing information on metered dose inhaler technique: is multimedia as effective as print? Fam Pract. 2003;20(5):552–7.
Hopper KD, Zajdel M, Hulse SF, et al. Interactive method of informing patients of the risks of intravenous contrast media. Radiology. 1994;192(1):67–71. https://doi.org/10.1148/radiology.192.1.8208968.
Krishna S, Francisco BD, Balas EA, et al. Internet-enabled interactive multimedia asthma education program: a randomized trial. Pediatrics. 2003;111(3):503–10.
Wilson EA, Makoul G, Bojarski EA, et al. Comparative analysis of print and multimedia health materials: a review of the literature. Patient Educ Couns. 2012;89(1):7–14. https://doi.org/10.1016/j.pec.2012.06.007.
Schenker Y, Fernandez A, Sudore R, et al. Interventions to improve patient comprehension in informed consent for medical and surgical procedures: a systematic review. Med Decis Making. 2011;31(1):151–73. https://doi.org/10.1177/0272989x10364247.
Kinnersley P, Phillips K, Savage K, et al. Interventions to promote informed consent for patients undergoing surgical and other invasive healthcare procedures. Cochrane Database Syst Rev. 2013;7:Cd009445. https://doi.org/10.1002/14651858.CD009445.pub2.
Tuong W, Larsen ER, Armstrong AW. Videos to influence: a systematic review of effectiveness of video-based education in modifying health behaviors. J Behav Med. 2014;37(2):218–33. https://doi.org/10.1007/s10865-012-9480-7.
Dahodwala M, Geransar R, Babion J, et al. The impact of the use of video-based educational interventions on patient outcomes in hospital settings: a scoping review. Patient Educ Couns. 2018;101(12):2116–24. https://doi.org/10.1016/j.pec.2018.06.018.
Dekkers T, Melles M, Groeneveld BS, et al. Web-based patient education in orthopedics: systematic review. J Med Internet Res. 2018;20(4):e143. https://doi.org/10.2196/jmir.9013.
Knox ECL, Quirk H, Glazebrook C, et al. Impact of technology-based interventions for children and young people with type 1 diabetes on key diabetes self-management behaviours and prerequisites: a systematic review. BMC Endocr Disord. 2019;19(1):7. https://doi.org/10.1186/s12902-018-0331-6.
Ciciriello S, Johnston RV, Osborne RH, et al. Multimedia educational interventions for consumers about prescribed and over-the-counter medications. Cochrane Database Syst Rev. 2013;4:Cd008416. https://doi.org/10.1002/14651858.CD008416.pub2.
Nishimura A, Carey J, Erwin PJ, et al. Improving understanding in the research informed consent process: a systematic review of 54 interventions tested in randomized control trials. BMC Med Ethics. 2013;14:28. https://doi.org/10.1186/1472-6939-14-28.
Friedlander JA, Loeben GS, Finnegan PK, et al. A novel method to enhance informed consent: a prospective and randomised trial of form-based versus electronic assisted informed consent in paediatric endoscopy. J Med Ethics. 2011;37(4):194–200. https://doi.org/10.1136/jme.2010.037622.
Afolabi MO, Bojang K, D’Alessandro U, et al. Multimedia informed consent tool for a low literacy African research population: development and pilot-testing. J Clin Res Bioethics. 2014;5(3):178. https://doi.org/10.4172/2155-9627.1000178.
McFarlane SJ, Morgan SE, Occa A, et al. An evaluation of clinical trial multimedia to support Hispanic cancer patients’ informational and decision-making needs. J Cancer Educ. 2021;36(1):110–7. https://doi.org/10.1007/s13187-019-01606-2.
Jolly K, Sidhu M, Bower P, et al. Improving recruitment to a study of telehealth management for COPD: a cluster randomised controlled ‘Study Within A Trial’ (SWAT) of a multimedia information resource. Trials. 2019;20(1):453. https://doi.org/10.1186/s13063-019-3496-z.
Hermann M. Dreidimensionale Computeranimation – neues Medium zur Unterstützung des Aufklärungsgesprächs vor Operationen Akzeptanz und Bewertung der Patienten anhand einer prospektiv randomisierten Studie – Bild versus Text. Der Chirurg. 2002;73(5):500–7.
Treweek S, Pitkethly M, Cook J, et al. Strategies to improve recruitment to randomised trials. Cochrane Database Syst Rev. 2018a;2:Mr000013. https://doi.org/10.1002/14651858.MR000013.pub6.
Treweek S, Bevan S, Bower P, et al. Trial Forge Guidance 1: what is a Study Within A Trial (SWAT)? Trials. 2018b;19(1):139. https://doi.org/10.1186/s13063-018-2535-5.
National Institute for Health Research (NIHR). Why the NIHR’s new funding stream for ‘Studies Within A Trial’ (SWATs) is potentially game-changing. 2018. https://www.nihr.ac.uk/blog/why-the-nihrs-new-funding-stream-for-studies-within-a-trial-swats-is-potentially-game-changing/11101#:~:text=innovation%20in%20research-,Why%20the%20NIHR’s%20new%20funding%20stream%20for%20'Studies%20Within%20A,SWATs)%20is%20potentially%20game%2Dchanging&text=Good%20clinical%20trials%20are%20hard%20work.&text=This%20is%20what%20makes%20NIHR's,(HTA)%20Programme%20so%20fabulous.
Madurasinghe VW, Bower P, Eldridge S, et al. Can we achieve better recruitment by providing better information? Meta-analysis of ‘Studies Within A Trial’ (SWATs) of optimised participant information sheets. BMC Med. 2021;19(1):1–8.
EMA Regulation (EC) No 1901/2006 of the European Parliament and of the Council of 12 December 2006 on medicinal products for paediatric use and amending Regulation (EEC) No 1768/92, Directive 2001/20/EC, Directive 2001/83/EC and Regulation (EC) No 726/2004 (Official Journal L 378, 27/12/2006 p.1–19). http://ec.europa.eu/health/documents/eudralex/vol-1/index_en.htm (Accessed 5 Dec 2022).
Martin-Kerry JM, Knapp P, Atkin K, et al. Supporting children and young people when making decisions about joining clinical trials: qualitative study to inform multimedia website development. BMJ Open. 2019a;9(1):e023984. https://doi.org/10.1136/bmjopen-2018-023984.
Sheridan R, Martin-Kerry J, Watt I, et al. User testing digital, multimedia information to inform children, adolescents and their parents about healthcare trials. J Child Health Care. 2018:1367493518807325. https://doi.org/10.1177/1367493518807325
Sheridan R, Preston J, Stones S, et al. Patient and public involvement in a study of multimedia clinical trial information for children, young people and families. Research for All. 2020;4(1):47–65. https://doi.org/10.18546/RFA.04.1.05.
Readability formulas. https://readabilityformulas.com/free-readability-formula-tests.php (Accessed 22 Feb 2023).
Knapp P, Martin-Kerry J, Sheridan R, et al. The effectiveness and acceptability of multimedia information when recruiting children and young people to trials: pre-planned meta-analysis of SWATs. NIHR Journals Library (in press).
Moe-Byrne T, Knapp P, Perry D, et al. Does digital, multimedia information increase recruitment and retention in a children’s wrist fracture treatment trial, and what do people think of it? A randomised controlled Study Within A Trial (SWAT). BMJ Open. 2022;12(7):e057508. https://doi.org/10.1136/bmjopen-2021-057508.
Knapp P, Heys R, Dabner L, et al. The effects of multimedia information on recruitment and retention in a children’s cardiac surgery trial: a randomised controlled SWAT (Study Within A Trial) [version 1; peer review: awaiting peer review]. F1000Research. 2022;11(340). https://doi.org/10.12688/f1000research.110083.1.
Martin-Kerry J, Parker A, Bower P, et al. SWATted away: the challenging experience of setting up a programme of SWATs in paediatric trials. Trials. 2019;20(1):141. https://doi.org/10.1186/s13063-019-3236-4.
Edwards P, Roberts I, Clarke M, et al. Increasing response rates to postal questionnaires: systematic review. BMJ. 2002;324(7347):1183. https://doi.org/10.1136/bmj.324.7347.1183.
Antal H, Bunnell HT, McCahan SM, et al. A cognitive approach for design of a multimedia informed consent video and website in pediatric research. J Biomed Inform. 2017;66:248–58.
Blower S, Swallow V, Maturana C, et al. Children and young people’s concerns and needs relating to their use of health technology to self-manage long-term conditions: a scoping review. Arch Dis Child. 2020;105(11):1093–104. https://doi.org/10.1136/archdischild-2020-319103.
Duffy B. Generations: does when you’re born shape who you are? London: Atlantic Books; 2021.
Katz R, Ogilvie S, Shaw J, et al. Gen Z explained: the art of living in a digital age. USA: University of Chicago Press; 2021.
Knapp P, Mandall N, Hulse W, et al. Evaluating the use of multimedia information when recruiting adolescents to orthodontics research: a randomised controlled trial. J Orthod. 2021;48:(4). https://doi.org/10.1177/14653125211024250.
The authors thank the CYP, their families/carers, investigators, study teams, and operational staff who contributed to the included studies. Per funder requirements, the full findings of this study will be published in Health and Social Care Delivery Research. The funder had no role in the study design; collection, analysis, and interpretation of the data; writing of the manuscript; or decision to submit this article for publication.
This study was funded by the NIHR Health and Social Care Delivery Research Programme (14/21/21). The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.
Ethics approval and consent to participate
The study protocol, protocol amendments, and informed assent/consent documents were reviewed and approved by the NHS Yorkshire and the Humber – Bradford Leeds Research Ethics Committee (17/YH/0082) and the Health Research Authority (IRAS ID 212761). The TRECA study was conducted in accordance with the principles of the Declaration of Helsinki, and all participants provided written informed consent to the host trial. No specific consent was required for the TRECA study, as approved by the Research Ethics Committee (17/YH/0082).
Consent for publication
All authors reviewed and approved the manuscript for publication.
SRS is an employee of Envision Pharma Group, owns stock options in Envision Pharma Group, is a member of Savvy Cooperative, and is a trustee of RAiISE, a charitable incorporated organisation registered in England and Wales.
No other authors have any competing interests to add.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Knapp, P., Moe-Byrne, T., Martin-Kerry, J. et al. Providing multimedia information to children and young people increases recruitment to trials: pre-planned meta-analysis of SWATs. BMC Med 21, 244 (2023). https://doi.org/10.1186/s12916-023-02936-1
- Multimedia information