A bibliometric analysis of childhood immunization research productivity in Africa since the onset of the Expanded Program on Immunization in 1974
BMC Medicine volume 11, Article number: 66 (2013)
The implementation of strategic immunization plans whose development is informed by available locally-relevant research evidence should improve immunization coverage and prevent disease, disability and death in Africa. In general, health research helps to answer questions, generate the evidence required to guide policy and identify new tools. However, factors that influence the publication of immunization research in Africa are not known. We, therefore, undertook this study to fill this research gap by providing insights into factors associated with childhood immunization research productivity on the continent. We postulated that research productivity influences immunization coverage.
We conducted a bibliometric analysis of childhood immunization research output from Africa, using research articles indexed in PubMed as a surrogate for total research productivity. We used zero-truncated negative binomial regression models to explore the factors associated with research productivity.
We identified 1,641 articles on childhood immunization indexed in PubMed between 1974 and 2010 with authors from Africa, which represent only 8.9% of the global output. Five countries (South Africa, Nigeria, The Gambia, Egypt and Kenya) contributed 48% of the articles. After controlling for population and gross domestic product, The Gambia, Guinea-Bissau and Sao Tome and Principe were the most productive countries. In univariable analyses, the country's gross domestic product, total health expenditure, private health expenditure, and research and development expenditure had a significant positive association with increased research productivity. Immunization coverage, adult literacy rate, human development index and physician density had no significant association. In the multivarable model, only private health expenditure maintained significant statistical association with the number of immunization articles.
Immunization research productivity in Africa is highly skewed, with private health expenditure having a significant positive association. However, the current contribution of authors from Africa to global childhood immunization research output is minimal. The lack of association between research productivity and immunization coverage may be an indication of lack of interactive communication between health decision-makers, program managers and researchers; to ensure that immunization policies and plans are always informed by the best available evidence.
The routine childhood immunization program was launched by the World Health Organization (WHO) in 1974, following the successful program for the eradication of small pox . The program, known as the Expanded Program on Immunization (or EPI), consists of regularly scheduled services that reach each new cohort of children less than one year of age with vaccines at health facilities, scheduled outreach sites, or (in special circumstances) from door to door. The program is composed of a series of inter-related components, including service delivery, vaccine supply and quality, logistics, advocacy and communication, surveillance, financing, management and capacity building. During the past four decades national EPI programs have developed or adapted and implemented a broad range of strategies and activities aimed at bringing services closer to the targeted community, increasing demand for immunization services, reaching previously unreached children, and improving immunization data quality [2–6]. Through these efforts, the mean proportion of the annual birth cohort that received a full series of three doses of the diphtheria, tetanus and pertussis vaccine (DTP3) reached 77% in sub-Saharan Africa in 2010 [7, 8]. Ideally, the development or adaptation and implementation of these interventions should be informed by the best available local evidence [9–11]. The increase in childhood immunization coverage in Africa over the last four decades would, therefore, be expected to have been accompanied by similar growth in childhood immunization research from the continent.
Research publications have an important role in the scientific process providing a key linkage between knowledge generation, uptake and use [12–14]. For a long time, publications and their citation (that is, bibliometrics) have been the method of choice for quantitative assessments of academic research at international, national, institutional and individual levels [15–18]. Bibliometric analysis is also a feasible tool to comprehensively recognize the research advances in the past and future research trends in a specific field. In the context of the African continent to date, factors related to variation in immunization research productivity have not been examined, although bibliometric studies with data on Africa exist in other disciplines [16–30]. Therefore, this bibliometric study aims to fill some of the gaps in existing research by providing insights into the history and growth of childhood immunization research in Africa. We examined whether national immunization coverage or other country-level factors are associated with childhood immunization research productivity in Africa.
We used childhood immunization research articles indexed in the PubMed database as a surrogate for total childhood immunization research productivity. We searched the database in November 2011 in order to obtain the childhood immunization research volume of each African country from 1 January 1974 to 31 December 2010. Articles originating from each country, published between 1974 and 2010 were generated by selecting the advanced-search option and then selecting the "publication date" field. Next, the "affiliation" field was searched for each country. The names of the countries were imputed in their different possible forms, for example, Cameroon and Cameroun for Cameroon. Some names of countries are also names of parts of other countries, for example, Niger is the name of a place in Nigeria. To avoid errors arising from this, appropriate commands were used (that is, (Niger (AD) NOT Nigeria)). We then combined this with childhood immunization search terms: ("Immunization"(Mesh) OR "Vaccination"(Mesh) OR "Immunization, Secondary"(Mesh) OR "Immunization Programs"(Mesh) OR "Immunization Schedule"(Mesh) OR "Immunization, Passive"(Mesh) OR "Mass Vaccination"(Mesh)) AND ("Infant, Newborn"(Mesh) OR "Infant"(Mesh) OR "Child, Preschool" (Mesh)).
The 2010 data on DTP3 coverage, adult literacy rate, gross domestic product (GDP), public expenditure on health (as a percentage of GDP), human development index, research and development expenditure, physicians (per 100,000 population), total expenditure on health and out-of-pocket health expenditure (that is, private expenditure on health) were obtained from the reports published by the WHO , United Nations Development Program and the World Bank .
We calculated the ratios of the number of publications from countries to their population, GDP and health expenditure in order to allow weighted comparisons. We used Pearson's correlation analysis to examine the association between research productivity and GDP, health expenditure, and research and development expenditure. Factors associated with variation in childhood immunization research productivity were explored using univariable and multivariable regression models for count outcomes.
We used the Bayesian Information Criterion (BIC) to compare different count regression models [32–36]. The BIC assesses the overall fit of a model and allows the comparison of both nested and non-nested models. It is based on a Bayesian comparison of models [32–36]. In case of two or more count regression models, under the assumption of no prior preference for one model over the other, BIC identifies the model that is more likely to have generated the observed data [32–36].
The formula for the BIC statistic reported by Stata , (which estat ic labels as BIC used by Stata), is:
where N is the number of observations used in estimation or the number of independent terms in the likelihood and k is the model degrees of freedom, number of parameter estimated including the constant calculated as the rank of variance-covariance matrix of the parameters. Given that the models fit on the same data, the model with the smallest value of the BIC is considered to be the best [32–36]. The most parsimonious model can be identified as the one with the lowest BIC, which in this case was the zero-truncated negative binomial regression (see Additional file 1). Univariable zero-truncated negative binomial regression analyses were used to investigate the bivariate relationship between each country-level factor listed above and total research productivity. Multivariable zero-truncated negative binomial regression analyses were carried out to determine which country-level factors were independently associated with total research productivity. Only variables with a value of P <0.2 in univariable analyses were included in the multivariable model. Results were presented as incidence rate ratios (IRR) with 95% confidence intervals (CIs) and percentage change. For zero-truncated negative binomial regression, country-level indicators were log transformed to linearize these associations. All tests were two-sided and statistical significance was defined at the 5% alpha level. Data were processed and analyzed with Stata 12 software (Stata Corp., College Station, TX, USA).
A total of 1,641 articles on childhood immunization indexed by PubMed between 1974 and 2010 are described in this study. The summary statistics for all country-level factors included in this study are shown in Table 1. The percentage of children that received DTP3 based on WHO/UNICEF 2010 estimates ranged from as low as 33% in Equatorial Guinea to 99% in Cape Verde, Eritrea, Mauritius, Morocco and Seychelles. The median adult literacy was 66% (range 26.2% to 93.0%). The median number of physicians per 100,000 population was 31 (range 2 to 243).
The number of childhood immunization articles indexed in PubMed from each country is shown in Table 2. As shown in the table, Africa's publication output trends show that its contribution to global childhood immunization publications has been low during the period 1974 to 2010. The percentage share of global childhood immunization research output increased from 6.6% in 1974 to 1980 to 9.6% in 2001 to 2010. The median number of articles was 16 (range 1 to 346). Figure 1 shows the number of articles broken down by quartiles. Three countries (South Africa, Nigeria and The Gambia) are in the highest quartile with more than 100 articles. Four countries belong to the second quartile (50 to 99 articles) and 27 to the third quartile (10 to 49 articles). Twenty countries with less than 10 articles belong to the lowest quartile.
Table 3 shows the top-ranking countries in terms of relative contribution of each country to the total number of articles. Authors from South Africa produced the highest number of articles (n = 346, 21%), followed by Nigeria (n = 154, 9%), and The Gambia (n = 104, 6%). In absolute terms, authors from the top five countries combined produced almost half (48%) of the total indexed articles. As shown in the table, The Gambia, Guinea-Bissau and Sao Tome and Principe had the highest number of publications after controlling for the country's population and GDP. When controlled for total expenditure on health, the top three countries were South Africa, Nigeria and Kenya. The trend in total production of childhood immunization articles in each geographical sub-region of Africa is displayed in Figure 2. West Africa was the most productive sub-region during the period studied. Apart from Central Africa and Southern Africa, which experienced a drop between 2001 and 2010, there was a continuous increase in the production of research articles from all African sub-regions during the period 1974 to 2010. The total number of articles from West Africa, for example, increased from 44 in 1974 to 1980 to 248 in 2001 to 2010.
Figure 3 shows the results of the correlation analyses. There was a strong positive and statistically significant correlation between the country's GDP (r = 0.541, P = 0.0001, Figure 3A), research and development expenditure (r = 0.548, P = 0.0001, Figure 3B), and total number of published articles on childhood immunization. Similarly, there was a moderately positive and statistically significant correlation between private expenditure on health and total number of articles (r = 0.361, P = 0.009, Figure 3C). However, there was no statistically significant correlation between total expenditure on health and immunization research productivity (r = 0.187, P = 0.189, Figure 3D).
Country-level factors associated with total childhood immunization research productivity are shown in Table 4. In the univariable model, a country's GDP, total expenditure on health, private expenditure on health, and research and development expenditure were statistically, significantly associated with increased childhood immunization research productivity. In the univariable analyses, immunization coverage, adult literacy rate, human development index and physician density had no significant statistical association with the number of immunization articles. As shown in the table, only private expenditure on health remained statistically significant in the multivariable model when all factors were controlled for statistically. Each unit increase in log dollar amount in private expenditure on health increased the total research productivity by 264% (IRR 3.64; 95% CI 1.46 to 9.07).
We found that childhood immunization research productivity in Africa is highly skewed. South Africa, Nigeria, The Gambia, Egypt and Kenya jointly account for almost half of the articles on childhood immunization indexed in PubMed between 1974 and 2010. There was a significant increase in the number of publications from all African sub-regions between 1974 and 2010. However, Africa's contribution to global childhood immunization publications has been minimal during the period studied. The Gambia, Guinea-Bissau, Sao Tome and Principe, Zimbabwe and Malawi had better records when the total research productivity was adjusted for gross domestic product. When controlled for total expenditure on health, South Africa, Nigeria, Kenya, Egypt and The Gambia were the most productive. Multiple medical schools and research institutions in South Africa, Nigeria and Egypt may account for the large number of publications from these three countries. Similarly, the presence of the British Medical Research Council, the Kenyan Medical Research Institute and a Danish research group (Bandim Health Project) may be the drivers of publications from The Gambia, Kenya and Guinea-Bissau, respectively.
In order to set our study in the context of other existing bibliometric studies, we searched PubMed, combining the terms "bibliometric" and "Africa". The search revealed 15 bibliometric studies with data on Africa [16–30]. These were either studies that had a global reach but reported data on Africa [19–29] or studies that focused on one or more African countries [16–18, 30]. None of these was a bibliometric study of childhood immunization research.
Rahman and Fukui studied factors related to worldwide variation in biomedical research productivity, and found that gross national product per capita and research and development expenditure were significant determinants of biomedical research productivity [37, 38]. We did not find these factors to be significant predictors of immunization research productivity in Africa. The most significant predictor of immunization research productivity in Africa was found to be out-of-pocket health expenditure. The latter refers to the sum of money spent on health by private entities, such as households, commercial or mutual health insurance, non-profit institutions serving households, and resident corporations with a health services delivery or financing function. Out-of-pocket health expenditure also includes gratuities and in-kind payments to health practitioners and suppliers of goods and services whose primary intent is to contribute to the restoration or enhancement of the health status of individuals or population groups. In addition, we confirm the findings of other authors that the contribution of authors from Africa to the global biomedical research literature is minimal [18–20, 22, 23]. This meager biomedical research literature from the continent appears to be dominated by non-communicable disease research .
Locally-relevant health research is needed to ensure the effectiveness, efficiency and equity of immunization policies in Africa [14, 39]. In general, health research helps to answer questions, to generate the evidence required to guide policy and to identify new tools. A descriptive analysis of study types, quality and outcomes was beyond the scope of our bibliometric analysis. However, in a related study, Machingaidze and colleagues conducted a detailed descriptive analysis of 881 childhood immunization research publications from Africa between 1970 and 2010 . The studies were classified as clinical (n = 442, 50.2%) or operational (n = 439) research. Among clinical research studies, 41% were phase 1 to 4 controlled trials, 23% were burden of disease or epidemiology and 36% were other clinical studies. Among studies classified as operational research, 76% were on program management, 19% on immunization policy issues and 5% related to vaccine financing . There is clearly a need for increased immunization research productivity from Africa, especially locally-relevant operational research. During the new decade of vaccines [41, 42], African countries should prioritize research capacity development in vaccinology. In general, Africa requires strong leadership and political commitment, going forward, in the development of research capacity on the continent.
PubMed has been widely used for bibliometric analyses, but it is important to note that the database is dominated by English-language journals, therefore, possibly contributing to selection bias due to language barriers. By using the author addresses listed in the by-lines of research articles, one can only identify countries and organizations where the authors were employed when the research was done or where the article was written, or both. These limitations notwithstanding, we believe that this study is a good reflection of research productivity in the field of childhood immunization in Africa.
There is an enormous range of locally-relevant research that could be undertaken to support routine childhood immunization programs in Africa [12, 14]. This would inform the development of strategic immunization plans whose implementation will raise immunization coverage and prevent disease, disability and death on the continent [12, 14]. This bibliometric study examined almost four decades of childhood immunization research production by authors from Africa. The results of the study show that the five most productive countries, in terms of absolute number of publications indexed by PubMed from 1974 to 2010, are South Africa, Nigeria, The Gambia, Egypt and Kenya. Based on the best possible estimate, the most significant determinant of immunization research productivity in Africa is private health expenditure. The lack of association between research productivity and immunization coverage may be an indication of the lack of interactive communication between health decision-makers, programme managers and researchers, to ensure that immunization policies and plans are always informed by the best available evidence.
Bayesian Information Criterion
third dose of diphtheria: tetanus and pertussis vaccine
Expanded Program on Immunization
gross domestic product
incidence rate ratios
World Health Organization
Okwo-Bele JM, Cherian T: The expanded programme on immunization: a lasting legacy of smallpox eradication. Vaccine. 2011, 29 (Suppl 4): D74-79.
Bilous J, Eggers R, Gasse F, Jarrett S, Lydon P, Magan A, Okwo-Bele JM, Salama P, Vandelaer J, Villeneuve P, Wolfson L: A new global immunisation vision and strategy. Lancet. 2006, 367: 1464-1466. 10.1016/S0140-6736(06)68625-X.
Ryman T, Macauley R, Nshimirimana D, Taylor P, Shimp L, Wilkins K: Reaching every district (RED) approach to strengthen routine immunization services: evaluation in the African region, 2005. J Public Health (Oxf). 2010, 32: 18-25. 10.1093/pubmed/fdp048.
Wiysonge CS, Nomo E, Mawo JN, Ticha JM: Accelerated measles control in sub-Saharan Africa. Lancet. 2006, 367: 394-395. 10.1016/S0140-6736(06)68133-6.
Clements CJ, Nshimirimanda D, Gasasira A: Using immunization delivery strategies to accelerate progress in Africa towards achieving the Millennium Development Goals. Vaccine. 2008, 26: 1926-1933. 10.1016/j.vaccine.2008.02.032.
Bosch-Capblanch X, Ronveaux O, Doyle V, Remedios V, Bchir A: Accuracy and quality of immunization information systems in forty-one low income countries. Trop Med Int Health. 2009, 14: 2-10. 10.1111/j.1365-3156.2008.02181.x.
WHO: WHO vaccine-preventable diseases: monitoring system - 2010 global summary. 2010, Geneva, Switzerland: World Health Organization
Brown DW, Burton A, Gacic-Dobo M, Karimov RI, Vandelaer J, Okwo-Bele JM: A mid-term assessment of progress towards the immunization coverage goal of the Global Immunization Vision and Strategy (GIVS). BMC Public Health. 2011, 11: 806-10.1186/1471-2458-11-806.
Wiysonge CS, Uthman OA, Ndumbe PM, Hussey GD: Individual and contextual factors associated with low childhood immunisation coverage in sub-Saharan Africa: a multilevel analysis. PLoS One. 2012, 7: e37905-10.1371/journal.pone.0037905.
Lewin S, Hill S, Abdullahi LH, de Castro Freire SB, Bosch-Capblanch X, Glenton C, Hussey GD, Jones CM, Kaufman J, Lin V, Mahomed H, Rhoda L, Robinson P, Waggie Z, Willis N, Wiysonge CS: 'Communicate to vaccinate' (COMMVAC). Building evidence for improving communication about childhood vaccinations in low- and middle-income countries: protocol for a programme of research. Implement Sci. 2011, 6: 125-10.1186/1748-5908-6-125.
Lewin S, Lavis JN, Oxman AD, Bastias G, Chopra M, Ciapponi A, Flottorp S, Marti SG, Pantoja T, Rada G, Souza N, Treweek S, Wiysonge CS, Haines A: Supporting the delivery of cost-effective interventions in primary health-care systems in low-income and middle-income countries: an overview of systematic reviews. Lancet. 2008, 372: 928-939. 10.1016/S0140-6736(08)61403-8.
Clements CJ, Watkins M, de Quadros C, Biellik R, Hadler J, McFarland D, Steinglass R, Luman E, Hennessey K, Dietz V: Researching routine immunization-do we know what we don't know?. Vaccine. 2011, 29: 8477-8482. 10.1016/j.vaccine.2011.08.048.
Wiysonge CS, Volmink J: Strengthening research capacity. Lancet. 2002, 359: 713.
Wiysonge CS, Lavis JN, Volmink J: Make the money work for health in sub-Saharan Africa. Lancet. 2009, 373: 1174.
Akl EA, Meerpohl JJ, Raad D, Piaggio G, Mattioni M, Paggi MG, Gurtner A, Mattarocci S, Tahir R, Muti P, Schünemann HJ: Effects of assessing the productivity of faculty in academic medical centres: a systematic review. CMAJ. 2012, 184: E602-612.
Uthman OA: HIV/AIDS in Nigeria: a bibliometric analysis. BMC Infect Dis. 2008, 8: 19-10.1186/1471-2334-8-19.
Wonkam A, Kenfack MA, Muna WF, Ouwe-Missi-Oukem-Boyer O: Ethics of human genetic studies in sub-saharan Africa: the case of Cameroon through a bibliometric analysis. Dev World Bioeth. 2011, 11: 120-127. 10.1111/j.1471-8847.2011.00305.x.
Uthman OA, Uthman MB: Geography of Africa biomedical publications: an analysis of 1996-2005 PubMed papers. Int J Health Geogr. 2007, 6: 46-10.1186/1476-072X-6-46.
Adam T, Ahmad S, Bigdeli M, Ghaffar A, Rottingen JA: Trends in health policy and systems research over the past decade: still too little capacity in low-income countries. PLoS One. 2011, 6: e27263-10.1371/journal.pone.0027263.
Borse NN, Hyder AA: Call for more research on injury from the developing world: results of a bibliometric analysis. Indian J Med Res. 2009, 129: 321-326.
Boulos MN: On geography and medical journalology: a study of the geographical distribution of articles published in a leading medical informatics journal between 1999 and 2004. Int J Health Geogr. 2005, 4: 7-10.1186/1476-072X-4-7.
Falagas ME, Papastamataki PA, Bliziotis IA: A bibliometric analysis of research productivity in parasitology by different world regions during a 9-year period (1995-2003). BMC Infect Dis. 2006, 6: 56-10.1186/1471-2334-6-56.
Michalopoulos A, Falagas ME: A bibliometric analysis of global research production in respiratory medicine. Chest. 2005, 128: 3993-3998. 10.1378/chest.128.6.3993.
Falagas ME, Karavasiou AI, Bliziotis IA: A bibliometric analysis of global trends of research productivity in tropical medicine. Acta Trop. 2006, 99: 155-159. 10.1016/j.actatropica.2006.07.011.
Hofman K, Ryce A, Prudhomme W, Kotzin S: Reporting of non-communicable disease research in low- and middle-income countries: a pilot bibliometric analysis. J Med Libr Assoc. 2006, 94: 415-420.
Lewison G, Grant J, Jansen P: International gastroenterology research: subject areas, impact, and funding. Gut. 2001, 49: 295-302. 10.1136/gut.49.2.295.
Ohba N: [Bibliometric analysis of the current international ophthalmic publications]. Nihon Ganka Gakkai Zasshi. 2005, 109: 115-125.
Ovhed I, van Royen P, Hakansson A: What is the future of primary care research? Probably fairly bright, if we may believe the historical development. Scand J Prim Health Care. 2005, 23: 248-253. 10.1080/02813430500316692.
Vergidis PI, Karavasiou AI, Paraschakis K, Bliziotis IA, Falagas ME: Bibliometric analysis of global trends for research productivity in microbiology. Eur J Clin Microbiol Infect Dis. 2005, 24: 342-346. 10.1007/s10096-005-1306-x.
Uthman OA: Pattern and determinants of HIV research productivity in sub-Saharan Africa: bibliometric analysis of 1981 to 2009 PubMed papers. BMC Infect Dis. 2010, 10: 47-10.1186/1471-2334-10-47.
The World Bank Data. [http://data.worldbank.org/]
Kass R, Raftery AE: Bayes Factors. J Am Statist Assoc. 1995, 90: 773-795. 10.1080/01621459.1995.10476572.
Neath A, Cavanaugh J: Regression and time series model selection using variants of the Schwarz information criterion. Commun Statist. 1997, 26: 559-580.
Raftery AE: Bayesian model selection in social research. Sociological Methodology. Edited by: Marsden PV. 1995, Oxford: Blackwell, 25: 111-163.
Schwarz G: Estimating the dimension of a model. Ann Statist. 1978, 6: 461-464. 10.1214/aos/1176344136.
StataCorp: BIC note -- Calculating and interpreting BIC. Stata 12 Base Reference Manual. 2011, College Station, TX: Stata Press
Rahman M, Fukui T: Factors related to biomedical research productivity in Asian countries. J Epidemiol. 2001, 11: 199-202. 10.2188/jea.11.199.
Rahman M, Fukui T: Biomedical research productivity: factors across the countries. Int J Technol Assess Health Care. 2003, 19: 249-252.
Wiysonge CS, Ngcobo NJ, Jeena PM, Madhi SA, Schoub BD, Hawkridge A, Shey MS, Hussey GD: Advances in childhood immunisation in South Africa: where to now? Programme managers' views and evidence from systematic reviews. BMC Public Health. 2012, 12: 578-10.1186/1471-2458-12-578.
Machingaidze S, Hussey GD, Wiysonge CS: Trends in the quantity and quality of childhood immunisation research output from Africa 1970-2010: findings from a systematic review. Proceedings of the First International African Vaccinology Conference; 8-11 November 2012. 2012, Cape Town, South Africa: University of Cape Town
Moxon ER, Das P, Greenwood B, Heymann DL, Horton R, Levine OS, Plotkin S, Nossal G: A call to action for the new decade of vaccines. Lancet. 2011, 378: 298-302. 10.1016/S0140-6736(11)60766-6.
Levine OS, Bloom DE, Cherian T, de Quadros C, Sow S, Wecker J, Duclos P, Greenwood B: The future of immunisation policy, implementation, and financing. Lancet. 2011, 378: 439-448. 10.1016/S0140-6736(11)60406-6.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1741-7015/11/66/prepub
The authors declare that they have no competing interests.
All authors participated in discussions about the data source and planning of the analyses, and critically revised successive versions of the paper. CSW conceived the study, and CSW and OAU did the analyses and prepared the first draft of the paper. All authors have seen and approved the final version of the paper.
Electronic supplementary material
Additional file 1: Comparison of models for goodness-of-fit. In this additional file we describe the tests we conducted to investigate which regression model provides the best fit for the empirical publication data on childhood immunization research volume from Africa, including probability distributions and Bayesian Information Criterion. (DOC 2 MB)
About this article
Cite this article
Wiysonge, C.S., Uthman, O.A., Ndumbe, P.M. et al. A bibliometric analysis of childhood immunization research productivity in Africa since the onset of the Expanded Program on Immunization in 1974. BMC Med 11, 66 (2013). https://doi.org/10.1186/1741-7015-11-66