User testing of the AML16 participant information sheet showed that it performed sub-optimally. Participants could not find or took a long time to find some of the information, but when information was found, it was almost always understood. Only a small minority of participants were able to find and show understanding of all aspects of the AML16 trial from the original version of the PIS. The revised version performed much better. Time taken in interviews was much shorter, illustrating the difficulty faced in finding answers by those in the original version group, and almost two-thirds of participants were able to find and show understanding of all aspects of the AML16 trial. When asked to compare the two sheets and state a preference, almost all participants preferred the revised version.
The sheet had been significantly revised during the developmental phase, by noting its performance during user-testing, and re-writing and re-designing to address the problems identified, using good practice in information design and clear writing. The contribution of user-testing as a developmental tool appears crucial - for example, in indicating continued weaknesses and in confirming when changes to the sheet have had a meaningful effect. These weaknesses could not have been identified solely through expert review.
The results confirm the pattern reported in three smaller, uncontrolled studies [22–24]. This study confirms that it is possible to improve written trial information (both in terms of its ability to inform and its appeal) by a combination of testing by lay people, along with the application of clear writing and information design. There was no difference between the two versions in the amount of information understood, but readers first have to locate information in order to understand it. This finding may indicate the greater contribution made by revision of the structure and layout of the PIS, as compared to changes in wording. However, to determine conclusively the relative effects of wording revisions and design revisions would require a different study design to that used here.
In several aspects of the study there were significant differences between what took place and what would happen to patients actually being asked to consider trial participation. The participants were members of the public asked to imagine themselves in a situation, participants were able to read the PIS at home before testing, and finally the user testing interviews took place in a quiet interview room rather than a busy hospital or clinic. While these factors may have aided participants in their reading, it is unlikely to have biased the data in favor of either version. Also in contrast to an actual trial, participants were asked to show an understanding of the AML16 trial from the PIS alone, without access to spoken information from a recruiting clinician. It is unclear whether the presence of a clinician would narrow differences between the two versions of the PIS, or whether clinicians might in fact provide clearer spoken information when being able to refer to a PIS that was more clearly written and designed. This would be worthy of further research.
As mentioned in the introduction Ancker  and others argue that a PIS should be analyzed according to how it performs, rather than by a number obtained from a readability formula, a view supported by this study. US Institutional Review Boards often require a certain formula score to be obtained before a trial sheet is approved. Formulae can indicate the difficulty of language within a document, but they offer very limited data in terms of which aspects of a document work and which do not. The user testing scores obtained for the original AML16 PIS would question whether a participant in the trial would have been able to give valid or informed consent, particularly since only one in seven people could find and understand answers to all questions.
The study examined participants' ability to find and understand written information, rather than test their willingness to participate in a trial. This study shows convincingly that a participant dependent only on written information would be more able to use it effectively, when it has been written and designed with its purpose in mind, as argued by Jefford and Moore . However the effect on actual trial participants is an important question. If using improved sheets resulted in increased trial recruitment rates, this would be both meaningful and valuable. Thus the effectiveness (and cost-effectiveness) of improving participant information materials is a question that requires an answer. A recent systematic review of interventions to increase trial recruitment reported that different forms of information had no impact . What is not clear from the review is whether enhanced information failed to impact on knowledge, or whether it did increase knowledge but failed to change behavior - an important distinction that deserves clarification.