WHO-led consensus statement on vaccine delivery costing: process, methods, and findings

Background Differences in definitions and methodological approaches have hindered comparison and synthesis of economic evaluation results across multiple health domains, including immunization. At the request of the World Health Organization’s (WHO) Immunization and Vaccines-related Implementation Research Advisory Committee (IVIR-AC), WHO convened an ad hoc Vaccine Delivery Costing Working Group, comprising experts from eight organizations working in immunization costing, to address a lack of standardization and gaps in definitions and methodological guidance. The aim of the Working Group was to develop a consensus statement harmonizing terminology and principles and to formulate recommendations for vaccine delivery costing for decision making. This paper discusses the process, findings of the review, and recommendations in the Consensus Statement. Methods The Working Group conducted several interviews, teleconferences, and one in-person meeting to identify groups working in vaccine delivery costing as well as existing guidance documents and costing tools, focusing on those for low- and middle-income country settings. They then reviewed the costing aims, perspectives, terms, methods, and principles in these documents. Consensus statement principles were drafted to align with the Global Health Cost Consortium costing guide as an agreed normative reference, and consensus definitions were drafted to reflect the predominant view across the documents reviewed. Results The Working Group identified four major workstreams on vaccine delivery costing as well as nine guidance documents and eleven costing tools for immunization costing. They found that some terms and principles were commonly defined while others were specific to individual workstreams. Based on these findings and extensive consultation, recommendations to harmonize differences in terminology and principles were made. Conclusions Use of standardized principles and definitions outlined in the Consensus Statement within the immunization delivery costing community of practice can facilitate interpretation of economic evidence by global, regional, and national decision makers. Improving methodological alignment and clarity in program costing of health services such as immunization is important to support evidence-based policies and optimal resource allocation. On the other hand, this review and Consensus Statement development process revealed the limitations of our ability to harmonize given that study designs will vary depending upon the policy question that is being addressed and the country context. Supplementary Information The online version contains supplementary material available at 10.1186/s12916-022-02278-4.


Background
Immunization has been shown to provide a high return on investment across low-and middle-income countries [1]. Nevertheless, disparities in immunization access persist within and between countries. With the launch of the new Immunization Agenda 2030 [2], many low-and middle-income countries (LICs and MICs) are considering introducing new vaccines or vaccine-related technologies, life-course immunization programs, and improving the effectiveness and efficiency of their immunization programs. To determine the feasibility of doing so, estimation of vaccine procurement and delivery costs is of considerable interest to policymakers, program managers, researchers, and other stakeholders concerned with improving immunization programs. In particular, results from delivery cost studies can help countries in decisionmaking and planning on introducing new infant and lifecourse vaccines and technologies, preparation of budgets and financing for rollout of vaccines, and evaluation of alternative service delivery approaches.
Recent reviews of immunization delivery cost literature identified a lack of standardization in methods and reporting, making cross-study comparison and synthesis difficult [3]. These discrepancies limit the interpretability and utility of immunization cost study evidence for immunization program decision-making. In light of these challenges, the World Health Organization (WHO) Immunization and Vaccines-related Implementation Research Advisory Committee (IVIR-AC) recommended at their March 2018 meeting that the WHO Guidance on Vaccine Delivery Costing be updated [4]. An ad hoc Working Group comprising vaccine delivery costing experts 1 was therefore convened by the WHO secretariat to review guidance documents and tools on vaccine delivery costing, focused on low-and middle-income country settings. This initial review found that several groups were already developing methodological guidance to address the disparate definitions and approaches in the field, which partly address the original IVIR-AC's request. In March 2019, IVIR-AC modified its request to instead review guidance documents and costing tools, assess their similarities and differences, and identify gaps in guidance [5]. In addition, the Working Group recommended that a Consensus Statement be developed to harmonize the differences in costing terminology and principles for groups working in vaccine delivery costing.
For the purpose of this paper, vaccine delivery costing is defined as "costs associated with delivering immunizations to target populations, exclusive of vaccine procurement costs" [6].
This paper describes the history and process involved to develop the Consensus Statement on Vaccine Delivery Costing, the methods used and the findings of the review of guidance documents and costing tools, and terms and principles as well as recommendations agreed upon by the Working Group.

Process of developing the consensus statement
The consultation process of coming to agreement on a Consensus Statement included setting up a time-limited Working Group of staff of organizations working in vaccine delivery costing (who are also the authors of this paper), conducting a review of guidance documents, costing tools, and other documents and the costing terms, methods, and principles used in these, agreeing upon the costing terminologies and principles, making recommendations to harmonize their differences, writing the text of the Consensus Statement and Annexes (Additional file 1). Figure 1 shows a timeline of the meetings and activities that led to the development of the Consensus Statement.
In March 2018, the IVIR-AC initiated the process and requested that WHO update its guidance for conducting vaccine delivery costing in LICs and MICs so that methods used in costing tools and guidance documents could be standardized among WHO and other organizations.
As a follow-up, the WHO secretariat set up a Working Group of Experts comprising staff of organizations conducting research and policy advice on vaccine delivery costing in LICs and MICs to ensure that no parallel efforts were taking place. The initial Working Group comprised of technical experts from the Bill & Melinda Gates Foundation (BMGF), International Vaccine Institute (IVI), Levin & Morgan LLC, UNICEF, and WHO, and two members of IVIR-AC. The group noted that there are several ongoing workstreams conducting cost studies and developing guidance documents and/or costing tools, with different purposes and approaches to costing. In addition, some of these workstreams had developed guidance documents specific to their approach, which were already in the public domain. Thus, a review of these would be required to determine if an additional vaccine delivery costing guidance would be necessary. The Working Group also suggested that a presentation be made at the IVIR-AC meeting in March 2019 to present the findings on the different workstreams to determine the next steps.
In March 2019, the WHO team and the BMGF-funded ThinkWell project (Immunization Costing Action Network [ICAN]) presented to IVIR-AC on findings from the discussion with the Working Group [2]. 2 IVIR-AC recommended that an in-person workshop meeting be held with other groups working on vaccine delivery costing so that a consensus could be reached on the best way to standardize costing terms and principles.
In July 2019, WHO and the BMGF convened a meeting with eleven experts from different organizations and institutions in immunization economics during an International Health Economics Association (iHEA) meeting in Basel, Switzerland. The Working Group was expanded to include technical experts from other organizations involved in vaccine delivery costing such as Harvard (Expanded Programme on Immunization Costing [EPIC] studies) and the United States Centers for Disease Control and Prevention (CDC). Based on the subject matter knowledge and professional experience of the Working Group members, the different purposes of the workstreams were discussed and a matrix of costing tools listing out the characteristics of each was created. The Working Group agreed that it would be useful to develop a Consensus Statement that presents the different purposes of each workstream, a review of existing vaccine delivery costing guidance documents and tools, and agreed-upon costing terms and principles.
As a follow-up from the meeting in Basel, from August 2019 to March 2020, an analysis of guidance documents and tools was conducted for each of the four workstreams identified by the Working Group. The Group identified similarities and differences in costing methods, terms and principles among the approaches and in guidance documents, and gaps where further guidance was needed. Based on these findings, the WHO team developed a proposed draft Consensus Statement report with recommendations for costing terms and principles that could be adhered to for future vaccine delivery costing work and accompanying annexes that summarized the findings from the review on costing terms, costing principles, and methods for vaccine delivery costing. After extensive consultation within the Working Group and several rounds of written revisions to reach consensus on the statement, the findings and recommendations were presented to IVIR-AC in September 2020. The IVIR-AC commended the process to create the Consensus Statement (Additional file 1) [7].

Review of vaccine delivery costing guidance documents and tools
The first step was to conduct a landscape analysis of the organizations involved in vaccine delivery costing and their workstreams, and the available guidance documents and tools on vaccine delivery costs. This landscape analysis was conducted through discussions between the Working Group members during teleconferences and an in-person meeting as well as internet searches of websites of organizations working in the field (e.g., ICAN, Immunization Economics) between August 2019 and March 2020. It was not a systematic literature review and did not aim to include general health service costing tools and guidance documents beyond those with known use for costing immunization in LICs and MICs. However, the analysis built on the recent systematic review and reporting guidance for immunization costing studies conducted by some working group member organizations [3].
The second step was to compare the characteristics of the guidance documents and tools for immunization costing identified in terms of (1) how costing terms were defined in the guidance documents and costing tools; (2) whether data collection, sampling, and analysis were described in the guidance documents; and (3) whether costing principles were specified in guidance documents.
To review the costing terms in the guidance documents, the definitions were extracted from the source documents and entered into a table so that similarities and differences could be compared qualitatively and recommendations could be made for harmonized definitions for key terms. The costing principles and the guidance text, including on data collection, sampling, and analysis, were also compared and entered into a table by workstreams to assess the similarities, differences, and gaps. To do so, the costing principles in the guidance documents were compared to the ones in the checklist in the Global Health Cost Consortium (GHCC) [8] that has become a normative reference standard for global health costing work. These principles are similar to those found in the CHEERS checklist [9]. Recommendations were then made for harmonized principles in the Consensus Statement.

Existing immunization delivery costing workstreams
The Working Group identified four major current workstreams on vaccine delivery costing in LICs and MICs. These include the following: (1) retrospective routine immunization (multiple vaccines) cross-sectional costing, (2) retrospective single-vaccine costing, (3) new vaccine introduction cost projection, and (4) national immunization program cost projection (Fig. 2). Although the workstreams had involvement from particular organizations at the time of the review, they are defined by their different objectives and corresponding methodologies and constitute a typology of immunization delivery costing work to which other organizations and practitioners beyond those listed contribute.
The first workstream is focused on estimating retrospective (i.e., already incurred) routine immunization cross-sectional costs of service delivery units at a single point in time for multiple vaccines delivered through the routine immunization program. These analyses focus on estimating routine immunization costs incurred at the facility, district, and higher administrative levels in the health system. Such analyses typically estimate unit costs (cost per dose, cost per person, or cost per fully immunized person [FIP]). Some examples of this work include the EPIC studies [10] and other work by institutes, such as the Harvard T.H. Chan School of Public Health, Wits University, Curatio Foundation, PAHO, Think-Well, UNICEF, Johns Hopkins University, and PATH. The objectives of research within this workstream are to develop benchmarks for costs to be used in future studies, to analyze variation in unit costs, and to compare the findings with data from other costing studies [4].
The second workstream is to estimate retrospective costs for a specific vaccine or campaign, typically using incremental costing. That is, it usually aims to measure the value of additional resources employed to introduce a new vaccine or conduct a vaccination campaign. This is often done through data collection at a single point in time (post-campaign or post-introduction) with reference to documents and recall by key informants to estimate which resource use was specifically incremental. Examples of such studies include those conducted by groups such as EPIC, ThinkWell, CDC, and IVI. This workstream includes retrospective cost studies of vaccine implementation using vaccine-specific costing tools (e.g., These studies yield results that will assist countries with comparing budgeted amounts to actual implementation resource use, budgeting for future immunization activities, and conducting cost-effectiveness analyses that compare the incremental resource use for a specific vaccine introduction or campaign with its incremental health impacts. The third workstream is focused on estimating new vaccine introduction costs through projection of the value of resources or ingredients (e.g., time, equipment, training, and vaccines) needed for vaccine introduction, typically using incremental costing for a specific period. Data for these analyses are obtained through interviews with program managers and facility visits to obtain current information on personnel time, supplies, equipment, and other resources as well as retrospective cost data from other vaccine introduction. These analyses are often conducted using costing tools, including some of the same tools used for retrospective single-vaccine costing (e.g., C4P, CHOLTOOL, MVICT, SIICT, and TCVCT). These studies produce cost estimates that will assist countries with planning and decision-making on new vaccines during the introduction period.
The fourth workstream is projection of immunization program costs. Some costing tools used to produce these estimates include the comprehensive multi-year plan (cMYP), 2nd Year of Life (2YL), and OneHealth tool where the activities of a national program and related cost is entered for a baseline year and then the future years are projected. These analyses are an integral part of strategic planning for budgeting and resource mobilization over a specific period of time such as 5 years. Whereas work under the first three workstreams may produce estimates of financial, economic, or undepreciated financial costs, projections under the fourth workstream are intended to estimate undepreciated financial costs (i.e., undiscounted monetary outlays).
In practice, projects may combine elements of multiple workstreams (e.g., retrospective single vaccine costing in one country may be used to help inform estimates of new vaccine introduction costs for a different vaccine or delivery strategy). Table 1 shows the nine guidance documents on vaccine delivery costing identified by the Working Group. Some of these provide guidance for more than one type of costing. Three are for estimation of retrospective routine immunization cross-sectional costs, five are for estimation of retrospective single-vaccine costs, five are for projection of new vaccine introduction costs, and one for projection of immunization program costs.  Table A2a)) since these do not include specific vaccine delivery costing methodologies.

Guidelines
Publication The list of costing tools for vaccine delivery identified by the Working Group is shown in Additional file 1: Table A2b. Table 2 shows a comparison of costing term definitions among the various guidance documents. It shows that among the different guidance documents, definitions are generally similar but have differences in wording, e.g., vaccine delivery cost, economic cost, start-up/ introduction cost, and prospective cost. Also, some terms (retrospective costing, cost projections, bottom-up and top-down costing) are only defined in the Global Health Costing Consortium reference case. Note that some guidance documents have been grouped together since they were developed by the same teams; i.e., (i) EPIC documents and (ii) WHO vaccine-specific costing tool user manuals. Figure 3 shows the percentage of guidance documents with definitions of individual costing terms. As can be seen, most documents had definitions of financial cost, economic cost, capital cost, recurrent cost, incremental cost, and vaccine delivery cost, and about half of these defined start-up/introduction cost. Fewer than half of the guidance documents had definitions of perspective, micro-costing (ingredients costing), full costing, retrospective costing, or cost projection.
Several gaps were noted from the review. Most guidance documents did not go into detail about some methodological decision points in costing, such as how the choice of perspective will affect which costs are included as financial costs, which may limit the comparability of such costs across studies. For example, if a payer or provider perspective is used, the organizations included in the study definition as "payers" or "providers" will determine whose monetary outlays are considered as financial costs. If a donor (e.g., Gavi) provides funding to a UNICEF country office for social mobilization for a new vaccine introduction, expenditures of those funds will be included as financial costs only if the study perspective is defined as including UNICEF (e.g., a provider perspective defined as all partners "providing" the new vaccine introduction activities, or a health sector perspective including all health sector partners); however, if the study is conducted from a perspective that does not include UNICEF (e.g., a provider perspective defined as only the government "providing" the new vaccine introduction activities, or a government perspective), these resources from UNICEF would not be counted as financial costs but only as economic costs as an in-kind contribution from UNICEF.
Also, most guidance documents did not address whether to include economic costs of existing capital such as equipment or building space, or how to make or assess assumptions for slackness (i.e., available unused capacity) of existing capital goods. Also, vaccine delivery costing definitions differ on whether actual vaccine product costs should be included or not. If not, which specific aspects of the vaccine product costs should be excluded (e.g., vaccine only, diluent, syringes, safety boxes, freight, and insurance). For financial costs, the guidance review suggested whether to include existing personnel costs will depend on whether the costing is incremental or full. Figure 4 shows the percentage of documents that recommended key costing principles (details in Additional file 1: Table A3). As can be seen, most guidance documents recommended principles on stating objectives, defining units, describing time horizon, methods and data sources, and annualizing capital costs, while less than half recommend specifying the perspective, scope, sampling, data collection timing, discount rates, shadow prices, exploring variation, analyzing uncertainty, and methods of communicating results.
In Table 3, the recommendations of guidance documents on data collection and analysis are disaggregated by workstream. While guidance is given on some aspects in all documents, in other cases, no guidance is provided. Specifically, guidance is given on data collection for all of the workstreams with the exception of projection of new vaccine introduction costs.

Recommended costing terminology and principles
After reviewing the definitions of costing terms, the following definitions of costing terms are recommended:

Vaccine delivery costs
Costs associated with delivering immunization programs to target populations, exclusive of vaccine costs. 2. Vaccine cost At a minimum includes the cost of the vaccine and diluent (if applicable); the analysis should include accounting for wastage rates; the analyst should specify whether this also includes injection supplies (syringes), international shipment, insurance, and customs/duties.  Goods or items used in the delivery of a service or intervention that last less than a year, e.g., personnel salaries (SIICT guide, pg.

21) [17] (Not included in all tool manuals)
Costs of resources consumed within one year (CMYP guide, pg. 19) [16]  One-time costs for items that have a useful life of over one year (pg. B-23) [8] NA Items that last longer than one year and are therefore incurred only every few years rather than annually (pg. 3) [14] Goods that last for longer than one year, such as equipment (SIICT guide, pg. 21) [17] An input that has a useful life of more than one year (cMYP guide, pg. 19) [16] Incremental cost Make assumptions about what particular resources were affected by the intervention, and only measure those resources (How to cost immunization, pg. 8) [12] Cost of adding a new or a batch of services or intervention over and above an existing program (pg. 59) [8] Additional costs associated with introducing new vaccines or making changes in delivery (pg. 32) [6] Only looks at the cost of an addition, e.g., a new vaccine, to existing services (pg. 2) [14] Additional resources required to add an intervention to an existing immunization program (CHOLTOOL guide, pg.  use of existing resources and any donated goods or services from any source. Capital costs are annualized and discounted. 5. Undepreciated financial cost Financial costs without depreciation of capital costs (note: such costs have been termed "initial investment" in some costing tools and referred to as fiscal costs in previous analyses.) 6. Recurrent cost Value of resources that last less than one year. Start-up activity costs may include recurrent costs. 7. Capital cost Value of resources lasting more than one year such as equipment, buildings, and trainings. Start-up activity costs may include capital costs. 8. Incremental cost Cost of adding a new service/intervention or a package of services/interventions over and above an existing program; inclusion of existing resources will depend on assumptions made about excess capacity (i.e., whether resources are underemployed; if there are no slack resources (e.g., all personnel time is fully allocated before the addition of the new service/intervention), then their use for the new service or intervention incurs an opportunity cost that should be included-either by measurement or assumption). 9. Full cost Baseline cost as well as the additional/incremental cost of the new intervention, including vaccine cost.

Cost projection
Estimation of future costs of both recurrent and capital inputs. 11. Prospective data collection Direct observation of resource use during intervention implementation; i.e., data are collected concurrently with intervention implementation. 12. Retrospective data collection Data collection after resource use is completed. 13. Start-up cost Cost of initial one-time programmatic activities. Examples may include initial micro-planning, initial training activities, and initial sensitization/social mobilization/ information, education, and communication (IEC); does not include routine or repeated programmatic activities such as refresher training or annual microplanning. Start-up activities may include both recurrent and capital costs; they are defined by the nonrepeating nature of the activity, not the type of input. 14. Micro-costing Focuses on granular accounting of input prices and quantities; disaggregates costs of particular output into specific goods and services consumed. 15. Bottom-up costing Measures input quantities at the client (e.g., per vaccination administered) or activity level. 16. Top-down costing Divides overall program cost or expenditures, often including those at administrative levels above service delivery level, by number of outputs to calculate unit cost. 17. Perspective The point of view considered for costs (and benefits, if included) in a costing study, by whom the costs were incurred. Payers are the disbursing agents for a good or service, and may differ from the original source of funding. A provider perspective includes costs incurred by health service providers (can be limited to the government), a payer perspective includes costs to the payer(s), such as government or an external partner, while the societal perspective includes all costs incurred by providers as well as clients. 18. Shared cost Shared resources that are not used only for immunization, but also for other productive activities. The recommended costing principles include the following.

Definitions of terms used in studies of vaccine
delivery costing should conform closely to the recommended definitions in this Consensus Statement. 2. The study scope in terms of its purpose, audience, target population, time horizon, and service/output should be clearly stated. It should also state whether data collection will be prospective or retrospective, and whether the analysis will be retrospective or a cost projection. 3. The perspective of the cost estimation should be stated and justified. 4. Types of costs to be generated should be clearly defined in terms of start-up/introduction or nonstart-up/introduction (sometimes called operating costs), recurrent and capital, undepreciated financial, financial or economic, and incremental or full.
Capital costs should be appropriately annualized and depreciated for financial and economic costs and the discount rate justified. 5. The scope of the inputs to be estimated should be defined, justified, and if needed referenced. For example, do the costs include national and subnational costs or only facility-level service delivery costs? Are non-immunization costs included? 6. The "units" in the unit costs for strategies, services, and interventions should be defined, e.g., cost per dose administered. 7. If incremental costing is conducted, any assumptions made regarding existing health system capacity should be described (see GHCC reference case, pg. 64). 8. The selection of the data sources, including any adjustments to price data (e.g., inflation or currency conversion) should be described and referenced. 9. The methods for estimating the quantity of inputs should be described-whether top-down or bottom-up, methods of allocation, use of shadow prices and the opportunity cost of time, and methods for excluding research and evaluation costs. 10. Costs should be mapped and reported as either inputs or activities: i. Resource inputs include, for example, personnel time, vaccines, injection and safety supplies, vehicles, fuel, per diem and travel allowances, cold chain equipment, stationery, laboratory equipment, and buildings; ii. Program activities include, for example, vaccine procurement, service delivery, training,  to have a range of estimates [16] micro-planning, social mobilization, and advocacy and communication, monitoring and evaluation, surveillance, adverse event following immunization monitoring, and supervision.
11. Some boundaries around costs included in the analysis may be employed to keep the costing scope feasible and will depend on the purpose of the costing study, with the rationale for any exclusions provided; use discretion about including one-time costs that are unique or unlikely to be replicated or transferable across settings (for example, new vaccine launches with the President). Clarify definition and threshold for including or excluding small costs that have expected small contribution (e.g., <$25) to total costs in aggregate across all sampled units, such as the use of existing office supplies by health facility staff. 12. The sampling strategy employed should aim for internal and external validity of the data 3 . Sampling strategy should be stated, described, and justified, depending on the workstream and costing objectives. Sampling of different service delivery units is desirable as it provides a more representative picture of costs and highlights cost variation and cost drivers for a strategy or vaccine. 13. Variation in the cost of the intervention by site/ organization, sub-population, or by other drivers of heterogeneity should be explored and reported for retrospective analyses when possible. 14. The uncertainty around the cost estimates should be appropriately characterized when feasible, (e.g., sensitivity analyses; ranges of results for different input parameter scenarios for cost projections; mean and standard deviation for non-representative samples with multiple units; and confidence intervals or credible intervals for retrospective analyses). 15. Inclusion and exclusion criteria: "stopping rules" 4 should be defined, explaining which costs are included and the respective rationale. 16. Cost estimates should be communicated clearly and transparently to enable decision-makers to interpret and use the results relevant to the original policy and/or programmatic question.

Discussion
The lack of standardization in terminology, implementation, and principles for vaccine delivery costing has resulted in difficulties in making comparisons among studies, reducing the potential for synthesis of economic evidence across studies for immunization program policy, planning, budgeting, and implementation. As noted earlier, governments need to know the cost of vaccine delivery in order to make decisions on introducing new infant and life-course vaccines, budgeting, and for making improvements in service delivery. The review indicates that existing guidance documents differ somewhat in the inclusion and definitions, of costing terms and costing principles that are recommended, reflecting in part differences in the aims and scope of the costing study.
The review of guidance documents and tools on vaccine delivery costing and iterative discussions among the Working Group members revealed considerable agreement among the different groups working in vaccine delivery costing. Most of the documents made the distinction between economic and financial costs as well as recurrent and capital costs. However, fewer went into detail about the perspective to choose, definition of some costing terms such as start-up costs, micro-costing, and bottom-up/top-down costing, and in some cases, recommended approaches for data collection and analyses. The review also identified gaps in guidance for some analyses, e.g., such as how perspective affects financial costs calculation.
The review revealed that different workstreams focus on distinct aspects of immunization costing with different purposes. These require different types of data collection and analyses. For example, retrospective costing of vaccination focuses on estimating actual resource use, benchmarking of costs, and investigation of variation at the facility and other levels. Cost projections, on the other hand, focus on estimation of (typically incremental) costs to assist in decision-making, preparation of budgets, and evaluating different approaches to a new technology, vaccine, or service delivery strategy.
The process to achieve a consensus statement of vaccine delivery costing methods was facilitated by having extensive consultations with different organizations conducting this work. It also was facilitated by conducting reviews of the guidance documents and costing tools so that similarities, differences, and gaps could be identified. Other strengths of the process include broad and ongoing engagement of experts across various workstreams, including members of the Immunization Economics Community of Practice [18], as well as dedicated support for facilitation, review, and write-up.
The process to develop a consensus statement provides lessons for developing agreement among other organizations and researchers on types of research methods and tools in other study areas. It requires the potential to bring together organizations working on similar research and then having the time and resources to develop consensus. In addition, it is useful to have some teleconferences and in-person (or virtual meetings with break-out sessions) meetings to have sufficient time to come to consensus.
One limitation of the exercise was that a systematic review was not conducted and some guidelines and costing tools may have been missed. More engagement of country-level practitioners and data and analysis experts outside of those directly involved in the workstreams, as well as a systematic literature search for any methodology documents beyond those known to the workstream participants, would have strengthened the process.
The work on immunization costing is extensive but some gaps were identified. The guidance documents, mostly user manuals for costing tools and the 2002 WHO guidance on introducing new vaccines for cost projections of new vaccines, are not sufficiently detailed regarding data collection and analyses. That is, these do not include instructions on methods of data collection and sampling and analysis methods, when required. Researchers that have piloted the costing tools have also noted that the manuals need to provide more instructions on perspective (see [19], for example). For example, there is a need for more guidance on how to treat perspective when there is more than one source of financing of vaccines, how to handle slack, etc. As a result, it would be useful to add to current user manuals or develop a new guidance document for cost projections for both single vaccines, multiple vaccines, and immunization programs.

Conclusions
This review and Consensus Statement development process revealed the limitations of our ability to harmonize given that study designs will vary depending upon the policy question that is being addressed and the country context. The Working Group hopes that the consensus statement will contribute to the development of costing guidelines and tools for new vaccines (single or multiple) and immunization programs that are better aligned in terms of definitions, methods, and reporting.
Additional file 1. Consensus Statement on Vaccine Delivery Costs which includes a review of vaccine delivery cost guidance documents and costing tools as well as a consensus statement on the terminology and methodological principles to be used for vaccine delivery costing. It includes two figures and 6 tables. Figure S1. Major current workstreams in vaccine delivery costing identified by working group. Figure A1 -Timeline for developing a Consensus Statement on Vaccine Delivery Costs. Table A2a. List of guidelines by publication year, target interventions, and purposes. Table A2b. List of costing tools for vaccine delivery or immunization program. Table A3. Definitions of costing terms in guidance documents. Table A4. Comparison of costing principles among guidance. Table A5. Characteristics of costing workstreams. Table 6A. Data sources, sampling and characterization of uncertainty, and terminology by workstreams.