Skip to main content

Application of real-time global media monitoring and ‘derived questions’ for enhancing communication by regulatory bodies: the case of human papillomavirus vaccines



The benefit-risk balance of vaccines is regularly debated by the public, but the utility of media monitoring for regulatory bodies is unclear. A media monitoring study was conducted at the European Medicines Agency (EMA) concerning human papillomavirus (HPV) vaccines during a European Union (EU) referral procedure assessing the potential causality of complex regional pain syndrome (CRPS) and postural orthostatic tachycardia syndrome (POTS) reported to the authorities as suspected adverse reactions.


To evaluate the utility of media monitoring in real life, prospective real-time monitoring of worldwide online news was conducted from September to December 2015 with inductive content analysis, generating ‘derived questions’. The evaluation was performed through the validation of the predictive capacity of these questions against journalists’ queries, review of the EMA’s public statement and feedback from EU regulators.


A total of 4230 news items were identified, containing personal stories, scientific and policy/process-related topics. Explicit and implicit concerns were identified, including those raised due to lack of knowledge or anticipated once more information would be published. Fifty derived questions were generated and categorised into 12 themes. The evaluation demonstrated that providing the media monitoring findings to assessors and communicators resulted in (1) confirming that public concerns regarding CRPS and POTS would be covered by the assessment; (2) meeting specific information needs proactively in the public statement; (3) predicting all queries from journalists; and (4) altering the tone of the public statement with respectful acknowledgement of the health status of patients with CRSP or POTS.


The study demonstrated the potential utility of media monitoring for regulatory bodies to support communication proactivity and preparedness, intended to support trusted safe and effective vaccine use. Derived questions seem to be a familiar and effective format for presenting media monitoring results in the scientific-regulatory environment. It is suggested that media monitoring could form part of regular surveillance for medicines of high public interest. Future work is recommended to develop efficient monitoring strategies for that purpose.

Peer Review reports


The benefit-risk balance of vaccines is frequently debated in the public domain and in particular in the media. These debates are linked on the one hand to the high expectations people have towards vaccines as one of the most successful health interventions to date and on the other hand to the phenomenon of vaccine hesitancy [1, 2]. The media debates have some aspects in common to all vaccines, but may also be driven by sentiments specific to certain vaccines [Karafillakis E, Larson H. The benefit of the doubt: a systematic literature review of perceived risks of vaccines in European populations. submitted]. Media attention may increase in cases of information on a new vaccine, an epidemic of a virus for which there is not yet a vaccine or the occurrence of a new vaccine-related concern. Regulatory bodies may be put in the spotlight, as they are in charge of vaccines licensure, safety surveillance, continuous benefit-risk assessment and, if needed, risk minimisation or other regulatory action. They also need to inform the public about the outcome of their assessments and provide advice on safe and effective use of vaccines [3].

As the communication process involves not only messaging, but also incorporating information from our surroundings and listening to others, mechanisms for effective listening need to be established. From the perspective of a regulatory body, listening should ensure that concerns expressed in the public domain are addressed in risk assessments [4], so that information, based on evidence and plausibility as well as on honesty over uncertainty, can be provided to the public and addresses their concerns. Furthermore, listening is an opportunity to collect data contributing to the body of evidence or its interpretation for risk assessment, such as data on how medicinal products are used by people. Through applying listening mechanisms, data on information needed by users of medicines enabling informed choice and safe use of medicines can be gathered too. In relation to vaccines, listening is a fundamental element of a new communication model which envisions communication as integrating with vaccine safety assessment and trust-building strategies [5].

Listening mechanisms available to regulatory bodies include directly interacting with members of the public (e.g. through working groups, public hearings, information contact points), conducting or reviewing research (e.g. surveys) and media monitoring. For the purpose of optimising communication about vaccines, media monitoring was proposed as early as the mid-1990s [6]. Based on this proposal and other vaccine communication experiences and research, media monitoring has been encouraged in the European Union (EU) vaccine pharmacovigilance guidance addressed to marketing authorisation holders and regulatory bodies [3]. The most recent media coverage study in Europe for a specific vaccine, conducted in Italy in relation to a seasonal influenza vaccine, recommended that public institutions should engage in prospective media monitoring [7]. While well-established medicines agencies conduct general media monitoring daily, the utility of medicinal product-specific media monitoring for regulatory bodies is as yet unclear.

The case of HPV vaccines

When in July 2015 a EU referral procedure was initiated for human papillomavirus (HPV) vaccines [8], the European Medicines Agency (EMA) decided, given frequent media debates about vaccines, to conduct and evaluate, for the first time, medicinal product-specific media monitoring to support preparations for communicating with the public about the assessment and outcome of the referral procedure. The assessment of HPV vaccines was considered a specifically important test case for media monitoring, as a public debate with a wide range of topics about these vaccines had already accompanied their licensure and launch in 2006. The debate then had been characterised by celebrating future reduction in mortality from HPV-related cervical cancer on the one hand, but also raised a number of concerns on the other, mainly about long-term effectiveness and benefit [914] as well as safety [9, 11, 1318]. Beyond benefit and safety, social concerns had arisen at the time too, speculating that HPV vaccination would encourage increased or unsafe sexual activity of the young [1315].

The media monitoring study was initiated in September 2015 during the EU referral procedure assessing a potential causal association between HPV vaccines and the suspected adverse reactions of complex regional pain syndrome (CRPS) and postural orthostatic tachycardia syndrome (POTS). Cases of CRPS and POTS symptoms occurring after HPV vaccination had been reported to the authorities [8]. The assessment was carried out by the Pharmacovigilance Risk Assessment Committee (PRAC), the EMA’s scientific committee responsible for monitoring and assessing safety issues for human medicines in the EU and providing recommendations for risk minimisation or other regulatory action as necessary. The PRAC is composed of members from each of the 28 EU member states, Iceland and Norway as well as additional scientific experts and healthcare professional and patient organisation representatives. Raising a safety issue with a medicine at EU level through a referral procedure results in a regulatory decision being implemented in all member states, Iceland, Norway and Liechtenstein. The PRAC recommendations on the referral for HPV vaccines were finalised and published in November 2015. The EMA has a communication department which prepares communication on all safety-related referral procedures and responds to queries from the public, including journalists.


Aim and objectives

The aim of this study was to evaluate whether prospective real-time media monitoring for specific medicinal products has the potential to enhance communication in terms of proactivity and preparedness for information provision to the public, in particular by a regulatory body.

Therefore, the following objectives were defined:

  1. 1.

    Develop a method for medicinal product-specific media monitoring in a real-life scenario.

  2. 2.

    Identify, through media monitoring, areas of concerns, information needs and expectations of the public in relation to the EU referral procedure assessment of HPV vaccines and take them into account when preparing communications.

  3. 3.

    Evaluate the utility of media monitoring to enhance communication.

It is not the objective of this article to provide information on the safety profile of HPV vaccines or to explain the outcome of the EU referral procedure on HPV vaccines and CRPS/POTS, as these are presented elsewhere [8, 19].

Study period

The study period started on 7 September and lasted until 23 December 2015, i.e. two months before and after the PRAC meeting, where the HPV vaccines assessment was scheduled for discussion and finalisation of the PRAC outcome occurred.

Search strategy

The media monitoring was conducted using Vuelio®, a media intelligence system which sources media content from a wide range of news and social media outlets worldwide in real time [20]. The following search settings were applied to all available online news stories and blog posts:

  • media types: worldwide online health, science, news and tabloid media, online websites of television channels and blogs (no restriction was set by e.g. media type, size of readership or size of country of origin, as it was considered that some relevant news would possibly be disseminated only by media with a small reach or by blogs)

  • search terms: the colloquial and technical terms as well as tradenames: “HPV vaccin*”; “papilloma W/3 vaccin*”; “cervical cancer vaccin*”; “HPV jab”; “papilloma jab”; “cervical cancer jab”; “Gardasil”; “Cervarix”; “Silgard” (the asterisk symbol * indicates that the ending of ‘vaccin’ could vary (e.g. vaccine, vaccination); ‘W/3’ (or W/2, W/4, W/5, etc.) indicates the number of words that could be inserted between two word elements)

  • languages: all official EU languages except for Irish and Maltese, i.e. 22 languages (the search terms were translated by native speakers at the EMA).

Daily media screening

Due to the application of a broad search strategy, the Vuelio system identified a large number of news stories and blog posts. These articles were first screened daily by JF for topic relevance, applying the following exclusion criteria, namely if the article:

  1. 1.

    Is about business or financial news only; or

  2. 2.

    Contains no information about HPV vaccines (but was identified by the system due to the different meanings of the multilingual search terms in other languages, in particular of the abbreviation “HPV”).

Articles in languages other than English were initially screened using Google Translate®, and for those considered critical, summary translations were provided by native speakers at the EMA.

Weekly media content analysis

Weekly charts of the global media screening output by date and by country were prepared by means of the Vuelio system, and weekly key topic summaries containing references to articles were created by JF, applying the following inclusion criteria, namely if the article:

  1. 1.

    Covers a topic about HPV vaccines with higher weekly media coverage than other topics;

  2. 2.

    Discusses an adverse event following immunisation, including personal experiences of individuals or the risk-benefit balance of HPV vaccines;

  3. 3.

    Reports about studies on HPV vaccines;

  4. 4.

    Reports about advocacy or parents’ groups active in relation to HPV vaccines; or

  5. 5.

    Mentions the EMA, the PRAC or an assessment, referring to the EU referral procedure for HPV vaccines.

Based on the key topic summaries, a content analysis of the included articles was undertaken by PB, resulting in formulation of considerations for communication preparations. Content analysis is a qualitative study design, and as such appropriate for exploring phenomena not yet well understood. It is an interpretative approach concerned with understanding the meanings that persons attach to actions, decisions, beliefs and values within their social world, as well as the mental processes persons use to make sense of the world around them [21]. The considerations for the communication preparations reflected upon concerns, information needs and expectations of the public in relation to HPV vaccines and their benefit-risk assessment by regulatory bodies as well as experiences and advice blogged by healthcare providers about the type of questions they are frequently asked by parents and how they respond. This content analysis therefore consisted of interpreting the explicit content as well as connotations and underlying assumptions implicit to the content and identifying questions raised explicitly or implicitly, including those raised due to lack of knowledge or anticipated to be raised once more information would be provided in the public domain. Lack of knowledge in this context refers to knowledge of scientific nature or about regulatory processes beyond the scope of common knowledge.

The charts, the key topic summaries and the considerations were presented in weekly media monitoring reports.

Presentation and categorisation of media content as derived questions

A cumulative review of the weekly media content analyses was performed by PB in the month prior to the scheduled finalisation of the PRAC assessment for supporting the preparations of the communication for the EU referral outcome. Identified concerns, information needs and expectations of the public were used by PB to derive questions from the media content. These were worded in an abstract manner, i.e. often differently from the questions raised explicitly in the media or were worded anew if the concern had not been presented in the media as a question at all, but its discussion implied a question or lack of knowledge which was considered as important for regulatory bodies to be prepared to answer or even fill with information proactively. The derived questions were written using language with terms familiar to colleagues in the regulatory and scientific domain involved in risk assessment or communication about the outcome of the assessment. PB grouped the derived questions into theme-based categories which had emerged from reviewing the content and organising these text data. Each category was allocated a high-level derived question and further sub-questions on specific aspects.

This approach to content analysis corresponds to an inductive method, i.e. a method by which concepts for categorising the content are not defined in advance or derived from other research but created through the content analysis itself, and where abstraction is applied through generating categories. This has been described as a conceptual and empirical challenge. An inductive approach is recommended where prior knowledge on the content or aspects relevant to the purpose of the content analysis is limited. Instead of relying on pre-defined categories, this flexible and sensitive approach allows one to go beyond a simplistic content description towards a deeper and new understanding of complex phenomena [22], here the meaning of what has been expressed explicitly and implicitly in the media. The inductive approach also allowed for creation of categories specifically relevant to a regulatory body.

Evaluation of utility

For evaluating the utility of medicinal product-specific media monitoring for regulatory bodies, the following was undertaken: (1) obtaining feedback in person from colleagues within the EU regulatory network using the media monitoring results; (2) reviewing the summary of PRAC recommendations (PRAC outcome) with a view on whether and how the derived questions were addressed; and (3) validating the predictive capacity of the derived questions by comparing the derived questions retrospectively with the queries raised by journalists towards the EMA during the study period.

The authors were not involved in the activities of the PRAC or the EMA communication department during the time of the referral procedure, had no influence on the assessment or the drafting of the outcome statement (other than through providing the results from the media monitoring) and had no knowledge of the journalist queries raised during the study period. The authors can therefore be considered ‘blinded’ for the purpose of the study.


Media monitoring

A total of 4230 articles (‘news clips’, i.e. news stories (3737) and blog posts (493)), were collected worldwide during the monitoring period (after applying the exclusion criteria). These originated from 2124 media outlets (see Additional file 1). The highest media coverage for Europe in terms of absolute numbers of articles was found in Denmark (626, based on the country allocation in the Vuelio system), whose authorities received reports for the suspected adverse reactions of concern at higher rates and initiated the EU referral procedure [8]. An analysis of intensity of the worldwide coverage for HPV vaccines by day identified six peaks with more than 100 articles in a day. The highest peak occurred on 5 November 2015 when the PRAC recommendations on the EU referral procedure were published by the EMA (Fig. 1). JF related these peaks to the triggering events identified through the media screening (Table 1) and compiled all key topics considered major over time in terms of the volume of media coverage or the relevance of the topic in relation to HPV vaccine risks, benefits, marketing authorisation or immunisation policy decisions (Table 2).

Fig. 1
figure 1

Time chart depicting volume of worldwide media coverage from 7 September to 23 December 2015 by day and identifying peaks (generated by the Vuelio® system)

Table 1 Triggering events of peaks of the worldwide media coverage on HPV vaccines from 7 September to 23 December 2015a
Table 2 Major topics discussed in the media for HPV vaccines worldwide from 7 September to 23 December 2015a

The media content analysis (after applying the inclusion criteria) by PB identified patterns over time: While many debates remained nationally contained, some topics ‘travelled’, in particular between Scandinavian countries and those countries with active parents’ groups, such as Demark, Ireland and the UK. There was also a change in focus; in particular in Denmark the debate moved from presenting personal stories to additionally including scientific and policy-related points. There were increasingly references to scientific publications on safety aspects, and overall the debate turned from questioning vaccine safety as such to doubting the trustworthiness of the data, the pharmaceutical industry as a data source and the integrity of the regulatory and other public health bodies in collecting and assessing data. Some debates claimed that the EMA and its scientific committees do not exercise separation between pre- and post-authorisation decision-making and rely on data provided by industry rather than requiring data from independent sources. This led to including in the respective weekly considerations for communication preparations that regulatory bodies should be prepared to answer in detail how they ensure the legally demanded independence of their work and manage potential conflicts of interests and how the pharmaceutical industry is inspected for their compliance with legal requirements.

Based on the cumulative review of the concerns, information needs and expectations of the public identified through the weekly media content analyses, 50 derived questions were formulated, which could be categorised into 12 themes with a high-level question each (12 questions) and sub-questions on specific aspects (38 questions) (Table 3).

Table 3 Derived questions based on a content analysis of the media coverage for HPV vaccines worldwide from 7 September to 22 October 2015 and categorised by themes

In addition to the theme categorisation and identifying patterns of flow and focus over time, the media content analysis allowed for an understanding of some of the motivations and expectations of parents. The parents who expressed their opinions in the media considered the information they had received on vaccine risks in general as insufficient. Parents who suspected that their daughters had been harmed by HPV vaccination mainly wanted to provide case information to the authorities, obtain support and treatment within the governmental health insurance as well as remedy the lack of respect their daughters feel that they experience in relation to their condition. Some also requested ending the HPV vaccination programme or wanted other parents to be provided with information about the ongoing EU referral review prior to giving an informed consent to vaccination. Giving special attention to respectfully acknowledge the health status of the patients, regardless of what the outcome of the EU referral would be, was therefore added to the considerations for preparing communication.


The users of the media monitoring results were the EMA communication department, the PRAC and the EU member states. Through the media monitoring, the media office of the EMA communication department became immediately aware of emerging issues which needed attention. The weekly media monitoring reports and the derived questions were circulated within the EMA and to the PRAC.

(1) Feedback from users of the media monitoring results

PRAC members from those EU member states leading the EU referral assessment and/or having higher national media activity or those PRAC members contributing special expertise in vaccines or communication (i.e. five PRAC members) were asked in person for feedback. They noted that the weekly media monitoring reports enhanced their communication preparedness for possible questions from the public they had not envisaged before. Members from the Scandinavian countries remarked that the reports helped them to put the media attention at national level in a broader European and global context. When taking a specific look at the weekly considerations in early October 2015, the PRAC members leading the assessment confirmed that all identified concerns and information gaps relating to CRPS and POTS as voiced by the public would be covered by the ongoing assessment. It was agreed that the broader public concerns, such as those about aluminium-containing adjuvants, had been evaluated in the context of previous assessments. This provided useful reassurance that no additional data reviews would be necessary to respond to anticipated questions from the public.

The medical writers of the EMA communication department stated that they were guided by the derived questions as to which information items from the assessment to include proactively in the summary of PRAC recommendations [23], i.e. the public statement on the PRAC outcome for website publication and dissemination to the EU regulatory network, its international partners, relevant patient and healthcare professional organisations and journalists (see (2) below). The derived questions guided these colleagues further as to which information to include in the talking points, prepared for the EMA itself as well as for the authorities in EU member states, to enable prompt provision of accurate and consistent information in response to external requests, including those from journalists (see (3) below). The talking points were also used by senior EMA colleagues to prepare for attending, upon invitation, a discussion on HPV vaccines at the Danish parliament in December 2015. The identification of the pattern of the public debate becoming increasingly focussed on scientific and policy-related points, particularly in Denmark, was considered to be especially helpful for these preparations.

(2) Review of the impact of the derived questions on the summary of PRAC recommendations

The summary of PRAC recommendations [23] showed that, in addition to reporting upon the assessment outcome in general, specific information had been included addressing all categories of derived questions relating to the assessment of the suspected CRPS/POTS reported for HPV vaccines, i.e. categories 1 to 5. The medical writers had selected 7 questions out of 21 (33%), and addressed them as follows:

  • The PRAC reviewed the published research, data from clinical trials and reports of suspected side effects from patients and healthcare professionals, as well as data supplied by Member States; (addresses derived questions 2.0. and 4.1.).

  • The PRAC took into account detailed information received from a number of patient groups that also highlighted the impact these syndromes can have on patients and families (addresses derived question 2.6.).

  • Symptoms of CRPS and POTS may overlap with other conditions, making diagnosis difficult in both the general population and vaccinated individuals. and The PRAC noted that some symptoms of CRPS and POTS may overlap with chronic fatigue syndrome (CFS, also known as myalgic encephalomyelitis or ME). Many of the reports considered in the review have features of CFS and some patients had diagnosis of both POTS and CFS. Results of a large published study that showed no link between HPV vaccine and CFS were therefore particularly relevant. (addresses derived questions 1.1. and 2.3.).

  • …available estimates suggest that in the general population around 150 girls and young women per million aged 10 to 19 years may develop CRPS each year, and at least 150 girls and young women per million may develop POTS each year. The review found no evidence that the overall rates of these syndromes in vaccinated girls were different from expected rates in these age groups, even taking into account possible underreporting. (addresses derived questions 5.0. and 3.3.).

The quotes show that detailed medical and methodological aspects such as those regarding patient impact, case ascertainment and adverse event underreporting, that are usually not included in summaries of PRAC recommendations, were included in the summary on HPV vaccines (however it is routine for the EMA to publish these aspects in the full assessment reports).

With regard to the tone of the summary of PRAC recommendations, the evaluation identified words intended to express commitment and diligence towards patients with CRPS and POTS and to acknowledge the seriousness of what they were experiencing. The summary highlighted that the scientific review was detailed, performed thoroughly and in consultation with leading experts. It further stressed that CRPS and POTS can severely affect the quality of life. This kind of wording is not the routine way of expression of a regulatory body, and a comparison confirmed that other summaries published by the EMA in 2015 were devoid of empathy.

(3) Capacity of derived questions to predict queries from journalists

The retrospective comparison of the queries from journalists to the EMA with the derived questions validated the predictive capacity and utility of media monitoring for communication preparations.

At the EMA press briefing about the PRAC outcome on 5 November 2015, four queries were raised by journalists [24], which were all predicted by the derived questions and therefore addressed in the talking points. This enabled EMA colleagues to provide well-informed responses promptly at the briefing. The queries related to themes 2, 3, 4, 5 and 12.

Before and after publication of the PRAC outcome, the EMA was contacted by journalists, as it is often the case for EU referral procedures on safety issues. While the referral procedure for HPV vaccines was ongoing, journalists often requested the timetable for finalising the assessment or asked for access to documents or interviews. A total of 16 journalist queries received during the entire study period contained actual questions, 9 before and 7 after the publication of the PRAC recommendations on 5 November 2015. These queries all corresponded with the derived questions (queries before 5 November: 1.0., 1.1., 2.0., 2.3., 3.0., 4.0, 5.0., 9.0, 9.2., 9.4., 10.0., 10.2. and 10.3., and after 5 November: 1.1., 2.0., 2.3., 3.0., 3.2., 3.3., 4.0., 7.0., 9.2., 10.0. and 12.0.). However, in some instances, the level of detail of the queries from journalists was not predicted. Clarifications on how referral procedures work in general were also frequently requested before and after publication of the PRAC outcome. The EMA responded to all queries from journalists, whether or not they were included in the talking points. The EMA’s declaration of interest policy and actual experts’ declarations had already been made accessible by the public [25]. For the comparison of the queries from journalists with the derived questions for validation purposes, the time period after the publication of the PRAC recommendations on 5 November was the relevant one, as the media monitoring results were used for preparing communication on the PRAC outcome, i.e. to inform the drafting of the statement on the PRAC recommendations and the talking points.


The article reports on the first time experience of the EMA with medicinal product-specific media monitoring and in-depth content analysis. The aim of this study was to evaluate whether such prospective real-time media monitoring has the potential to enhance communication in terms of proactivity and preparedness for information provision to the public, and the study results contribute to filling the current knowledge gap as to whether regulatory bodies, and other stakeholders in medicines, should engage in medicinal product-specific media monitoring. To the authors’ knowledge, no other study has yet been conducted within a regulatory body and evaluated the utility of product-specific media monitoring. Published studies on vaccines in the media often review media coverage retrospectively. Here, media monitoring has been conducted prospectively in real time.

As this study investigated media monitoring for the purpose of supporting communication proactivity and preparedness, it was not the objective to quantify media coverage or obtain a representative picture of the public debate. Instead, it was the objective to gain an understanding of concerns, information needs and expectations of the public, in the sense of all groups of the public, since a regulatory body serves all citizens and needs to understand which kind of information on scientific evidence and regulatory processes is missing in the public domain.

Through conducting the study, a method for media monitoring has been developed, including a new approach to analysing and presenting media coverage by translating the media monitoring findings into derived questions for regulators to consider. With this approach, concerns, information needs and expectations towards regulatory bodies that have been discussed in the public domain explicitly, implied, raised due to a lack of knowledge, or anticipated to be raised once more information has been published, have been expressed in the form of specific questions in scientific language and with reference to regulatory policies and procedures. Question formats are familiar to those in the scientific-regulatory domain and seemed to be an effective way to provide feedback from the public domain to those in charge of assessing medicines or defining messages on assessment outcomes. While media monitoring seeks to understand relevant mental models (i.e. explanations of someone's thought process about how something works in the real world [26]) prevalent in the public domain, the translation of the media monitoring findings into derived questions intends to make the findings fit into the thought processes of the scientific-regulatory domain, whilst helping to focus communication on specific pieces of information either missing, or expected, in the public domain. Some vaccine safety experts have raised questions as to whether listening and providing feedback in response to unsubstantiated concerns voiced by the public could risk what has been referred to in the literature as the ‘social amplification of risk’ [27]. While recognising this risk, genuinely listening with openness to concerns voiced by the public and responding honestly and with transparency is considered essential for building and sustaining trust [28], a fundamental principle in relation to matters of public good.

The study used a large data set of worldwide news stories and blog posts. Although the study was conducted for only one regulatory procedure, it was one of major importance. However, the study has some limitations in the utility evaluation. Despite media monitoring findings being widely disseminated via the PRAC to all EU member states and to the EMA communication department, feedback was only obtained from those most closely working on the EU referral procedure itself. However, more systematic evaluation methods included reviewing the impact of the media monitoring findings on the public PRAC outcome statement, and comparing the derived questions with the queries from journalists, which revealed their predictive capacity of 100% in terms of themes. The study did not include a survey or obtain views directly from young people, parents or healthcare professionals in the EU. It is assumed that if members of the public search for information from authorities, they usually view the websites of the authorities in their own country rather than the EMA website, given that the EMA website is mainly visited by professionals in pharmaceutical industry, regulatory authorities and the media. Further evaluation could include studying how the outcome of the EU referral procedure on HPV vaccines was reported in the news and social media, whether public concerns persisted and were discussed in the light of the assessment by PRAC, or whether journalists felt satisfied by the information published by the EMA in terms of scope, format and language. Correlations between information from regulatory bodies, media coverage and vaccination rates among the population would be interesting for establishing the impact of communication by regulatory bodies on medicines use behaviours.

In terms of resources, 165 working hours of a communication specialist (JF) and 75 working hours of a medicines safety specialist (PB) were used for the media screening and content analysis in 2015. This equals 0.34 and 0.15, respectively, full-time equivalents per month. In order to use resources for media monitoring efficiently, the experience from this study suggests that limiting the number of languages monitored to English and to languages from those countries with high relevant media coverage could be sufficient, but further work is required to clarify the most efficient process to support routine implementation. The use of exclusion terms (e.g. “budget”, “profit”) to automatically, rather than manually, exclude articles (e.g. financial news) had been discussed when setting up the search strategy, but was not implemented because of concerns around excluding articles about important policy and trust issues. The study showed that concerns about profit-driven bias and expectations for independent data gathering and assessment were indeed voiced by the public and important to be addressed by regulatory bodies through transparency of their procedures. In order to investigate if resource efficiency can be augmented by automated media screening, further work focussing on developing hierarchical or conditional search algorithms with increased specificity and without losing sensitivity is recommended. Now that a potentially workable method for media content analysis has been developed, future media monitoring for other medicinal products could make efficient use of the available method.

The study results contribute to the Accelerated Development of VAccine benefit-risk Collaboration in Europe (ADVANCE), a private-public consortium aiming to establish a reliable, valid and tested framework for providing rapidly robust data and scientific evidence on vaccine benefits and risks in Europe. As part of this framework, ADVANCE is developing recommendations for communication strategies in relation to vaccine benefit-risk monitoring [29].

The following principles and actions on communication are therefore recommended in general for any stakeholder assessing vaccines (e.g. regulatory and public health authorities, industry or academia) and for ADVANCE in particular:

  • Efficient media monitoring should be built into the process of vaccine benefit-risk monitoring, and benefit-risk assessment should ensure the provision of responses to all safety concerns, including those debated in the public domain.

  • Explanations on methods for benefit-risk monitoring and assessment should be provided in a language understandable to the public, and should be developed and ideally be tested with a view to explaining how the method works and how robust the results are.

  • Given that conflicts of interests are a major concern voiced by the public, procedures to ensure unbiased monitoring and assessment as well as, if applicable, the mechanisms of public-private partnership (PPP) governance models (as envisaged by ADVANCE as one option for a future platform for vaccine benefit-risk monitoring) need to be proactively communicated to the public.


The study demonstrated the utility of media monitoring for the EMA in the case of the EU referral procedure for HPV vaccines. The presentation of media coverage in the newly developed format of a structured set of derived questions supported scientific assessors and communicators in focussing on specific pieces of information, missing or expected in the public domain, and in enhancing communication proactivity and preparedness. More specifically, the study demonstrated that the derived questions helped the EMA communication department to proactively enrich the public summary statement of PRAC recommendations with medical and methodological aspects. The derived questions further proved ‘ready-for-use’ by the EMA communication department when drafting the talking points for communication preparedness, enabling prompt responses from the EMA at the press briefing and to written queries from journalists. The study demonstrated full predictive capacity regarding the queries from journalists, although in some instances the level of detail of the questions was not anticipated. In addition, the findings from the media monitoring altered the tone of the public statement with acknowledgement of the health status of CRSP and POTS patients, in order to meet public expectations of indicating respect towards patients.

This study seems to be the first on setting up medicinal product-specific media monitoring as a feasible process within a regulatory body in a real-life scenario and evaluating whether it is useful for improving its communication. The study suggests that efficient strategies for medicinal product-specific prospective real-time media monitoring could be part of safety surveillance for medicines of high public health impact and/or high public interest and support not only communication in terms of proactivity and preparedness, but also in relation to transparency and public participation activities. Overall, medicine safety and public trust into underlying surveillance systems could benefit from listening more systematically to the public, identifying and challenging misleading or missing information and addressing what different population groups may want and need to know for making informed choices and using medicines safely and effectively.



Accelerated Development of VAccine benefit-risk Collaboration in Europe


Chronic fatigue syndrome


Complex regional pain syndrome


European Medicines Agency


European Union


Human papillomavirus


Postural orthostatic tachycardia syndrome


Public-private partnership


Pharmacovigilance Risk Assessment Committee (of the EMA)


  1. Leach M, Fairhead J. Vaccine anxieties: global science, child health and society. London: Taylor & Francis Earthscan; 2007.

    Google Scholar 

  2. World Health Organization (WHO) SAGE working group dealing with vaccine hesitancy. Strategies for addressing vaccine hesitancy: a systematic review. Geneva: WHO; 2014.

  3. European Medicines Agency (EMA) and Heads of Medicines Agencies. Guideline on good pharmacovigilance practices — product- or population-specific considerations I: vaccines for prophylaxis against infectious diseases. London: EMA; 2013.

  4. Bahri P. Public pharmacovigilance communication: a process calling for evidence-based, objective-driven strategies. Drug Saf. 2010;33:1065–79.

    Article  PubMed  Google Scholar 

  5. Larson H, Brocard Paterson P, Erondu N. The globalization of risk and risk perception: why we need a new model of risk communication for vaccines. Drug Saf. 2012;35:1053–9.

  6. Saliou P. Crisis communication in the event of a flu pandemic. Eur J Epidemiol. 1994;10:515–7.

    Article  CAS  PubMed  Google Scholar 

  7. Odone A, Chiesa V, Ciorba V, Cella P, Pasquarella C, Signorelli C. Influenza and immunization: a quantitative study of media coverage in the season of the ‘Fluad case’. Epidemiol Prev. 2015;39 Suppl 1:139–45.

    PubMed  Google Scholar 

  8. European Medicines Agency (EMA). Human papillomavirus vaccines [referral webpage]. London: EMA; 2015-2016.

  9. Rondy M, van Lier A, van de Kassteele J, Rust L, de Melker H. Determinants for HPV vaccine uptake in the Netherlands: a multilevel study. Vaccine. 2010;28:2070–5.

    Article  PubMed  Google Scholar 

  10. Gerhardus A, Razum O. A long story made too short: surrogate variables and the communication of HPV vaccine trial results. J Epidemiol Community Health. 2010;64:377–8.

    Article  PubMed  Google Scholar 

  11. Nghi NQ, Lamontagne DS, Bingham A, Rafiq M, le Mai TP, Lien NT, Khanh NC, Hong DT, Huyen DT, Tho NT, Hien NT. Human papillomavirus vaccine introduction in Vietnam: formative research findings. Sex Health. 2010;7:262–70.

    Article  PubMed  Google Scholar 

  12. Rothman SM, Rothman DJ. Marketing HPV vaccine: implications for adolescent health and medical professionalism. J Am Med Assoc. 2009;302:781–6.

    Article  CAS  Google Scholar 

  13. Sherris J, Friedman A, Wittet S, Davies P, Steben M, Saraiya M. Chapter 25: education, training, and communication for HPV vaccines. Vaccine. 2006;24 Suppl 3:S3 210–8.

  14. Tafuri S, Martinelli D, Vece MM, Quarto M, Germinario C, Prato R. Communication skills in HPV prevention: an audit among Italian healthcare workers. Vaccine. 2010;28:5609–13.

    Article  CAS  PubMed  Google Scholar 

  15. Friedman AL, Shepeard H. Exploring the knowledge, attitudes, beliefs, and communication preferences of the general public regarding HPV: findings from CDC focus group research and implications for practice. Health Educ Behav. 2007;34:471–85.

    Article  PubMed  Google Scholar 

  16. Bingham A, Drake JK, LaMontagne DS. Sociocultural issues in the introduction of human papillomavirus vaccine in low-resource settings. Arch Pediatr Adolesc Med. 2009;163:455–61.

    Article  PubMed  Google Scholar 

  17. Brown EC, Little P, Leydon GM. Communication challenges of HPV vaccination. Fam Pract. 2010;27:224–9.

    Article  PubMed  Google Scholar 

  18. Chow SN, Soon R, Park JS, Pancharoen C, Qiao YL, Basu P, Ngan HY. Knowledge, attitudes, and communication around human papillomavirus (HPV) vaccination amongst urban Asian mothers and physicians. Vaccine. 2010;28:3809–17.

    Article  PubMed  Google Scholar 

  19. World Health Organization (WHO). Global Advisory Committee on Vaccine Safety Statement on safety of HPV vaccines. Geneva: WHO; 2015.

  20. Vuelio® [website]. Accessed 24 Jan 2017.

  21. Ritchie J, Lewis J, editors. Qualitative research practice: a guide for social science students and researchers. London: Sage; 2003.

    Google Scholar 

  22. Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs. 2007;62:107–15.

    Article  Google Scholar 

  23. European Medicines Agency (EMA). Review concludes evidence does not support that HPV vaccines cause CRPS and POTS. London: EMA; 2015.

  24. European Medicines Agency (EMA). Virtual press briefing: human papillomavirus (HPV) vaccines: recommendation from the Pharmacovigilance Risk Assessment Committee (PRAC) [YouTube video]. London: EMA; 2015.

  25. European Medicines Agency (EMA). Handling competing interests [webpage]. London: EMA, 1999-2017.

  26. Morgan MG, Fischhoff B, Bostrom A, Atman CJ. Risk communication: a mental model approach. Cambridge: Cambridge University Press; 2002.

    Google Scholar 

  27. Kasperson RE, Renn O, Slovic P, Brown HS, Emel J, Goble R, Kasperson JX, Ratick S. The social amplification of risk: a conceptual framework. Risk Anal. 1988;8:177–87.

    Article  Google Scholar 

  28. Renn O, Levine D. Credibility and trust in risk communication. In: Kasperson R, Stallen P, editors. Communicating risks to the public. Boston: Kluwer; 1991. p. 175–218.

    Chapter  Google Scholar 

  29. Innovative Medicines Initiative (IMI). Accelerated Development of Vaccine Benefit-Risk Collaboration in Europe (ADVANCE) Project [webpage]. Brussels: IMI; 2010.

  30. Cabut S. Etude rassurante sur les vaccins contre le cancer du col de l’utérus. Paris: Le Monde; 2015.

    Google Scholar 

  31. Associated Press staff. GOP lawmakers ask schools to oppose HPV vaccine mandate. Washington DC: The Washington Times; 2015.

  32. CBC News staff. HPV vaccine not as effective as abstinence, B.C. bishop tells parents. Toronto: CBC News; 2015.

  33. Findalen J, Cuculiza M. Danmark dropper omstridt HPV-vaccine. Copenhagen: Metroxpress; 2015.

  34. Ritzau A. Flere piger end ventet har HPV-bivirkninger: så mange er ramt. Søgade:; 2015.

  35. Gilkey MB, Malo TL, Shah PD, Hall ME, Brewer NT. Quality of physician communication about human papillomavirus vaccine: findings from a national survey. Cancer Epidemiol Biomarkers Prev. 2015;24:1673–9. Epub October 22 2015.

    Article  PubMed  PubMed Central  Google Scholar 

  36. International Papillomavirus Society. The International Papillomavirus Society endorses human papillomavirus vaccines [press release]. New York: PR Newswire; 2015.

  37. Weber C, Andersen S. Firma bag HPV-vaccinen underdrev omfanget af alvorlig lidelse. Copenhagen: Metroxpress (original from Radio24syv); 2015.

  38. Lu PJ, Yankey D, Jeyarajah J, O’Halloran A, Elam-Evans LD, Smith PJ, Stokley S, Singleton JA, Dunne EF. HPV vaccination coverage of male adolescents in the United States. Pediatrics. 2015;136:839–49. Epub 26 October 2016.

  39. Canadian Press. HPV vaccine Gardasil review finds no evidence of new risks: Health Canada. Toronto: CBC News; 2015.

  40. ITV plc staff. Claims that cancer vaccine caused girl’s ill health. London: ITV online UK; 2015.

  41. Heaney T. We must stop HPV vaccine. Cork: The Irish Examiner; 2015.

    Google Scholar 

  42. SaneVax. Irish government calls for investigations on Gardasil vaccine injuries. Austin: Health Impact News; 2015.

    Google Scholar 

  43. Youle E. Mother tells of daughter’s debilitating illness as Ham&High investigates HPV vaccine safety. London: Ham&High; 2015.

  44. The Liberty Beacon staff. HPV vaccines proven to be ineffective, dangerous & even deadly. USA: The Liberty Beacon; 2015.

  45. Rail G, Molino L, Lippman L. Appel urgent à un moratoire sur la vaccination contre les VPH. Montreal: Le Devoir; 2015.

    Google Scholar 

  46. Fu LY, Zook K, Spoehr-Labutta Z, Hu P, Joseph JG. Search engine ranking, quality, and content of web pages that are critical versus noncritical of human papillomavirus vaccine. J Adolesc H. 2016;58:33–9.

    Article  Google Scholar 

  47. Holst C. HPV-piger risikerer opslidende tur i arbejdsprøvnings-karrusel. Copenhagen:, Ekstra Bladet; 2015.

  48. TV3 staff. HPV Documentary [You Tube video]; Dublin:; Accessed 14 Dec 2015 at:

  49. Herald Scotland staff. Schoolgirls should stop being given HPV vaccine until investigation into side effects, campaigner says. Glasgow: Herald Scotland; 2015.

    Google Scholar 

  50. El Economista staff. Afectados por la vacuna del virus del papiloma humano piden a sanidad que haga un seguimiento de los casos detectados. Madrid, Barcelona: El Economista (SERVIMEDIA); 2015.

  51. The Irish Times staff. Mother fails to get order withdrawing cervical cancer vaccine. Dublin: The Irish Times; 2015.

    Google Scholar 

  52. REGRET. REGRET.IE launches national register appeal for injured girls. R.E.G.R.E.T. (Reactions and Effects of Gardasil Resulting in Extreme Trauma) Support Group. Ireland:; 2015.

  53. Birrell F, Hinks S. UK Association of HPV Vaccine Injured Daughters (AHVID). SaneVax, Inc (Safe, Affordable, Necessary & Effective Vaccines and Vaccination Practices); 2015

  54. Freyer FJ. Parents protest R.I. mandating HPV vaccine for teens. Boston: Boston Globe online; 2015.

    Google Scholar 

  55. Fifield A. The science doesn’t support them, but Japanese anti-vaxxers are winning on HPV. Washington DC: The Washington Post; 2015.

  56. Carley MJ. Autism without fear: hold it - Donald Trump’s a moron on vaccines? No: Washington’s a moron on vaccines! USA: Huffington Post Online (blog); 2015.

  57. World Health Organization (WHO). Uganda launches human papillomavirus vaccine. Geneva: WHO; 2015.

  58. Kimani G. Tanzania: Gov’t urged to create a cervical cancer tree Tanzania. Cape Town, Dakar, Lagos, Monrovia, Nairobi, Washington DC:; 2015.

  59. Dwyer D. Boston Medical Center awarded grant to improve HPV vaccination rates through provider education. Boston:; 2015.

  60. Panagiotou OA, Befano BL, Gonzalez P, Rodríguez AC, Herrero R, Schiller JT, Kreimer AR, Schiffman M, Hildesheim A, Wilcox AJ, Wacholder S, on behalf of the Costa Rica HPV Vaccine Trial (CVT) Group. Effect of bivalent human papillomavirus vaccination on pregnancy outcomes: long term observational follow-up in the Costa Rica HPV Vaccine Trial. Br Med J. 2015;351:h4358.

    Article  Google Scholar 

  61. Petráš M, Adámková V. Impact of quadrivalent human papillomavirus vaccine in women at increased risk of genital warts burden: population-based crosssectional survey of Czech women aged 16 to 40 years. Vaccine. 2015;33:6264–7.

  62. Sundhedsstyrelsen (Danish health authority). Kraftigt fald i forekomsten af kondylomer blandt unge kvinder og mænd. Copenhagen: Sundhedsstyrelsen; 2015.

  63. Evans S. The government has been urged to consider changing its policies over HPV vaccinations to stop the spread of mouth cancer. UK:; 2015.

  64. Centers for Disease Control and Prevention (CDC). Human papillomavirus vaccination coverage among female adolescents in managed care plans: United States, 2013. Atlanta: CDC; 2015.

  65. Pharmaceutical Business Review (PBR) staff. FDA approves expanded age indication for Merck’s Gardasil 9 in males. London: PBR Regulatory Affairs; 2015.

  66. Secher M. Overlæge og DF: ny HPV-konklusion er forudsigelig og bekymrende. Odense: TV2 Denmark; 2015.

    Google Scholar 

  67. Kenfelt M. HPV-lavinen. Copenhagen: Metroxpress Blog; 2015.

  68. Schreier D. Impfen: Verflechtungen mit der Industrie. Germany: Netzfrauen; 2015.

  69. Findalen J, Cuculiza M. Kræver HPV-vaccinen undersøgt af uafhængige. Copenhagen: Metroxpress; 2015.

  70. Chustecka Z. Chronic symptoms after HPV vaccination: Danes start study. New York: Medscape; 2015.

Download references


The authors thank Heidi Larson and Emilie Karafillakis from the London School of Hygiene and Tropical Medicine for their support in designing the methodology and providing regular advice regarding the study and this article. Further thanks are conveyed to Miriam Sturkenboom, Vincent Bauchau, Judith Perez-Gomez and Marianne van der Sande from the ADVANCE consortium, and Peter Arlett and Alessandro Spina at the European Medicines Agency (EMA) for reviewing the article manuscript.


The study was conducted for the Accelerated Development of VAccine benefit-risk Collaboration in Europe (ADVANCE). The research leading to these results has received support from the Innovative Medicines Initiative (IMI) Joint Undertaking under ADVANCE Grant Agreement no. 115557, resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and in-kind contribution of companies which are members of the European Federation of Pharmaceutical Industries and Associations (EFPIA). All authors have completed the International Committee of Medical Journal Editors (ICMJE) uniform disclosure form and declare that no support has been received from any organisation for the submitted work other than from the European Union’s Seventh Framework Programme, and that there are no other financial or non-financial relationships with any organisation that could appear to have influenced the submitted work.

Availability of data and materials

All data and internal working documents generated and used during this study, i.e. media screening outputs as well as weekly and cumulative reviews, are held at the EMA and are available from the corresponding author upon reasonable individual access to document request according to EMA policies.

Authors’ contributions

PB had the initial idea for the study, led on its overall design, performed the weekly and cumulative media content analysis and the utility evaluation and was lead author of the manuscript. JF led on the development of the search strategy, performed the daily media screening, prepared the weekly key topic summaries, aggregated the data for the cumulative theme and peak reviews and contributed to the manuscript accordingly. DM advised on the methods for data analysis as well as on the interpretation and the presentation of the results, and contributed accordingly to the drafting of the final manuscript. XK advised on the evaluation strategy, contributed to the writing of the manuscript and ensured the appropriate conduct of the study under the terms of ADVANCE. The lead author PB affirms that the manuscript is an honest, accurate, and transparent account of the research being reported and that no important aspects have been omitted. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Not applicable.


The views expressed in this article are the personal views of the authors and may not be understood or quoted as being made on behalf of or reflecting the position of the European Medicines Agency or one of its committees or working parties.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and Affiliations



Corresponding author

Correspondence to Priya Bahri.

Additional file

Additional file 1:

Media outlet sources of articles collected through applying the described search strategy. (DOCX 69 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bahri, P., Fogd, J., Morales, D. et al. Application of real-time global media monitoring and ‘derived questions’ for enhancing communication by regulatory bodies: the case of human papillomavirus vaccines. BMC Med 15, 91 (2017).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: