- Research article
- Open Access
- Open Peer Review
The potential to expand antiretroviral therapy by improving health facility efficiency: evidence from Kenya, Uganda, and Zambia
BMC Medicinevolume 14, Article number: 108 (2016)
Since 2000, international funding for HIV has supported scaling up antiretroviral therapy (ART) in sub-Saharan Africa. However, such funding has stagnated for years, threatening the sustainability and reach of ART programs amid efforts to achieve universal treatment. Improving health system efficiencies, particularly at the facility level, is an increasingly critical avenue for extending limited resources for ART; nevertheless, the potential impact of increased facility efficiency on ART capacity remains largely unknown. Through the present study, we sought to quantify facility-level technical efficiency across countries, assess potential determinants of efficiency, and predict the potential for additional ART expansion.
Using nationally-representative facility datasets from Kenya, Uganda and Zambia, and measures adjusting for structural quality, we estimated facility-level technical efficiency using an ensemble approach that combined restricted versions of Data Envelopment Analysis and Stochastic Distance Function. We then conducted a series of bivariate and multivariate regression analyses to evaluate possible determinants of higher or lower technical efficiency. Finally, we predicted the potential for ART expansion across efficiency improvement scenarios, estimating how many additional ART visits could be accommodated if facilities with low efficiency thresholds reached those levels of efficiency.
In each country, national averages of efficiency fell below 50 % and facility-level efficiency markedly varied. Among facilities providing ART, average efficiency scores spanned from 50 % (95 % uncertainty interval (UI), 48–62 %) in Uganda to 59 % (95 % UI, 53–67 %) in Zambia. Of the facility determinants analyzed, few were consistently associated with higher or lower technical efficiency scores, suggesting that other factors may be more strongly related to facility-level efficiency. Based on observed facility resources and an efficiency improvement scenario where all facilities providing ART reached 80 % efficiency, we predicted a 33 % potential increase in ART visits in Kenya, 62 % in Uganda, and 33 % in Zambia. Given observed resources in facilities offering ART, we estimated that 459,000 new ART patients could be seen if facilities in these countries reached 80 % efficiency, equating to a 40 % increase in new patients.
Health facilities in Kenya, Uganda, and Zambia could notably expand ART services if the efficiency with which they operate increased. Improving how facility resources are used, and not simply increasing their quantity, has the potential to substantially elevate the impact of global health investments and reduce treatment gaps for people living with HIV.
Over 29 million people were living with HIV in 2013, but due to an unprecedented global response, HIV burden has markedly declined and an estimated 19.1 million life-years have been saved by interventions such as antiretroviral therapy (ART) . This success was fueled by a rapid escalation of HIV-specific development assistance for health (DAH), rising from $1.4 billion in 2000 to $10.8 billion in 2015 [2, 3]. Yet, HIV funding has plateaued since 2010 and the paradigm for ART is shifting toward long-term care, diverging from past emergency response models of care . In September 2015, the World Health Organization (WHO) revised its ART guidelines , stipulating that everyone living with HIV should initiate ART irrespective of disease progression. This update sets universal HIV treatment as an equity-promoting goal, but also establishes nearly 21 million people eligible for ART who have yet to receive care [1, 5]. To reach these patients and sustain current ART services without a guarantee of additional funding, increasing health system efficiency has emerged as a vital pursuit for low- and middle-income countries (LMICs) [6, 7].
Scaling up health services involves a complex interplay of policy levers, as planners must balance improving access and equitable provision of care within operational and financial constraints. A focus on efficiency has become particularly attractive for expanding ART in LMICs, as increasing efficiency represents a feasible avenue for elevating service production without a proportional rise in expenditures [8, 9]. Otherwise, expanding care hinges upon more government spending, increased DAH, or heightened out-of-pocket payments for patients – and recent projections point to relatively minimal increases in government health spending by LMICs through 2040 . Technical efficiency refers to the relationship between a health facility’s inputs and outputs , with efficient facilities defined as those using their inputs to generate the largest quantity of outputs. From an output-oriented perspective, an inefficient health facility is one where its resources are not fully maximized, leaving usable beds empty or providers seeing few patients each day. Such inefficiencies, when aggregated up through health systems, can represent millions of missed opportunities to provide care and health dollars lost. WHO estimates that up to 40 % of health spending is wasted by system inefficiencies in LMICs , suggesting that substantial cost-savings and service expansion could occur if improving efficiency was prioritized.
Over the last 15 years, several studies have assessed the technical efficiency of health facilities in sub-Saharan Africa [13–31], quantifying opportunities for efficiency gains across levels of care. However, past work has largely focused on one type of facility or region, which may not reflect a country’s broader health system capacity. Facilities with incomplete input and output records are often excluded, heightening the risk for biased estimates of efficiency; for instance, facilities with more complete data may benefit from higher-quality resources and facility processes . To date, few studies specifically evaluate the technical efficiency of facilities that offer ART. Two recent studies measured the efficiency of facilities providing ART in Rwanda and Zambia [27, 28], but each had samples with fewer than 35 facilities. Finally, such efficiency analyses for LMICs infrequently account for the quality of outputs facilities generate . This is an area of particular concern for policymakers and providers alike, as increasing healthcare production in the absence of essential medical supplies and infrastructure will neither improve patient outcomes nor overall program sustainability.
For our analysis, we used new efficiency measurement methods specifically adapted for LMICs  to assess the technical efficiency of health facilities in Kenya, Uganda, and Zambia. Facility data originated from the Access, Bottlenecks, Costs, and Equity (ABCE) project, a multi-country study where data were collected from stratified random samples of country facility rosters . These datasets capture inputs and outputs, including ART volumes, from hundreds of facilities, and explicitly link these data to information on stocks of medical supplies and various structural characteristics. Such linkages allowed us to construct output-specific indicators of structural quality, which served as proxy for service quality . Based on efficiency estimates from facilities providing ART, we predicted the potential for ART service expansion across efficiency improvement scenarios. Recognizing current evidence gaps on efficiency in LMICs, we sought to quantify the capacity for providing more ART through gains in efficiency.
We used nationally-representative facility data from Kenya, Uganda, and Zambia [35–37]. Facility data collection occurred from September 2011 to April 2012 for Zambia, and April to November 2012 for Kenya and Uganda. Each country dataset included publicly- and privately-owned facilities across levels of care, and provided retrospective 5-year panel data for a subset of indicators (e.g., staff, patient volumes, and services provided) and cross-sectional data on facility characteristics, equipment availability, and pharmaceutical stocks. Retrospective panel data reflected fiscal years, which largely covered 2006 to 2010 for Zambia and 2007 to 2011 for Kenya and Uganda. In instances where data were incomplete for a subset of indicators, we used Amelia II software to create 50 imputed datasets. ABCE data collection and data processing are detailed elsewhere .
In-depth descriptions of facility sampling approaches have been previously published for Kenya, Uganda, and Zambia [39–41], and additional detail country-specific sampling strategies can be found in the Additional file 1: Appendix S1. In sum, a two-step stratified random sampling process occurred to construct nationally representative samples of health facilities for each country. The first step entailed creating a sampling frame from which subnational geographic units (districts for Uganda and Zambia, and counties for Kenya) would be drawn. For Uganda, one rural and one urban district were randomly drawn from 10 regional boundaries commonly used by household surveys in the country. For Kenya and Zambia, counties and districts were grouped by geographic performance indicators derived from previous surveys, and one subnational unit was randomly drawn from each category. The second sampling step involved sampling facilities from selected districts or counties across a range of facility types identified for each country, in accordance with their health systems. Ministry of Health facility inventories from 2011 in Kenya and Uganda, and 2010 in Zambia, served as the facility sampling frame source. For each country, this two-step sampling process resulted in 18 to 22 districts or counties selected through the district or county sampling frame, and between approximately 180 and 270 facilities selected through the facility sampling frame. For each country, a predetermined number of facilities were randomly selected from each facility type category within selected districts or counties.
The final ABCE datasets for these countries included 625 facilities and 2973 facility-years. We excluded national hospitals, specialty facilities, dental clinics, and pharmacies due to their substantive differences in services offered and production processes. To identify specialized facilities for exclusion, we used keyword searches of survey administrator comments, as well as manual examinations for facilities in the top and bottom 5 % of the efficiency spectrum. We also excluded facilities for which all 5 years of panel data were missing. For our analysis, we used a total of 395 facilities and 1900 facility-years.
We categorized facilities, inputs, and outputs into consistently-defined groups to facilitate comparisons across levels of care and countries. Facilities were grouped into categories, or platforms, based on their number of beds: 0 (no inpatient services), 1 to 15, 16 to 50, and more than 50 beds. For inputs, we used number of full-time equivalent (FTE) facility staff disaggregated by doctors, nurses, other medical personnel, and non-medical personnel as measures of labor input, and number of beds as a measure of capital. For outputs, we used the number of outpatient visits, ART visits, antenatal care visits, births, and inpatient bed-days. ART visits were comprised of the sum of pre-ART and ART visits. Due to survey differences in Zambia, we estimated ART visits using the number of ART patients seen at facilities and multiplying these values by the average number of patient visits extracted from clinical charts.
Structural quality adjustment
To account for potential variations in service quality, we created structural quality-adjustment scores for each output. These country- and output-specific scores were calculated by determining whether each medical supply or technology was available and functional in a given facility, and then summing the total of these binary variables. The inclusion of a given output indicator was informed by clinical guidelines, physician recommendations, and whether the indicator was captured through the ABCE facility survey administered in each country. For each facility, we divided their output-specific quality sum by the highest quality value found within the country and then multiplied their annual outputs by the facility’s output-specific quality scores. Since quality scores could not be computed over time, we applied the measure retrospectively over the panel data. The facility survey used in Zambia slightly differed from the one administered in Kenya and Uganda, so the indicators included for each of Zambia’s outputs somewhat varied. Additional file 2: Appendix S2 provides the full list of indicators used for each output by country.
Our study was conducted in three steps, namely (1) estimating facility efficiency scores; (2) assessing facility determinants of efficiency; and (3) predicting the potential for ART expansion. We also conducted a number of sensitivity analyses, which are detailed in Additional file 3: Appendix S3.
Estimating facility efficiency
We used a measurement approach recently designed for quantifying technical efficiency in lower-resource settings, referred to as the ensemble method (ENS). A description of the ENS model and its merits are detailed elsewhere , but in sum, it combines results from restricted versions of Data Envelopment Analysis (rDEA) and Stochastic Distance Function (rSDF) . DEA computes the ratio of weighted outputs to weighted inputs , and then assigns efficiency scores to each facility relative to a frontier set by facilities with the highest ratio of outputs to inputs. The rDEA model involved placing weight restrictions for both inputs and outputs, which offered a solution to a primary drawback – arbitrary weights – of traditional DEA . SDF is commonly used to estimate technical efficiency for production processes with multiple outputs (e.g., outpatients, inpatients, ART patients) . rSDF, which used a Cobb-Douglas multiple-output production function, restricted variance to be greater than zero and thus allowed interpretable measurement of error. As demonstrated in an extensive simulation study , combining results from rDEA and rSDF (the ENS model) provided the most robust estimates of technical efficiency, particularly when the underlying production function of facilities are uncertain. Further, the ENS model helped to offset some of DEA’s largest pitfalls: its tendency to underestimate technical efficiency in contexts with a non-linear multiple-output production function and its frequent overestimation of efficiency when the distribution of efficiency is uniform. For analyzing technical efficiency of facilities in lower-resource settings, the ENS model performance was viewed as the most preferred estimation strategy .
For all facility-years of data, we used the ENS model to estimate efficiency scores, which were based on the median of 50 imputed datasets. In subsequent analyses, we used efficiency scores from the most recent facility-year, viewing this as a stronger measure of current capacity for service expansion than multi-year averages of efficiency.
To produce platform- and country-level averages of efficiency, we applied weights to each facility efficiency score, as derived from ABCE sampling frames.
Assessing facility determinants of efficiency
For each country, we logit-transformed facility efficiency scores and multivariate regressions by pooling facilities across platforms and countries, and accounted for numerous facility covariates: urbanicity; ownership; electrical connectivity; hosting of administrative meetings; hosting of personnel training; the natural logarithm of reported catchment population; fraction of FTEs absent on the day of survey; and fraction of FTEs staffed by doctors, nurses, and volunteers or externally funded personnel. We also conducted bivariate regressions stratified by platform for each country.
Predicting the potential for ART expansion
We estimated the potential for increased ART visits across efficiency improvement scenarios, such that all facilities with ART and efficiency scores below a given threshold increased their efficiency to reach that threshold. For instance, in the 50 % efficiency improvement scenario, we considered a world where facilities with ART and efficiency scores below 50 % reached 50 % efficiency. We assessed these scenarios by 10 percentage point increments, from all facilities with ART reaching at least 10 % efficiency to 100 % (all facilities are fully efficient). We then computed the potential number of additional ART visits and corresponding percentage increases in ART for each country.
For this analysis, potential for expanded ART was defined by the radial, or proportional, expansion of all facility outputs. This parameter precluded options of opening new ART clinics within facilities or output transformation (e.g., increasing ART volumes by holding inpatient services constant), both of which could additionally scale up ART services. However, these approaches require an influx or shifting of resources, and would not capture the potential for maximizing inputs at facilities that already provide ART. Instead, we sought to quantify how much ART outputs could be increased, given observed facility resources and service offerings, through gains in efficiency.
To propagate uncertainty for our estimates of facility efficiency, we used bootstrap resampling, where we randomly selected one of the 50 imputed datasets, i, and sampled with replacement to create a bootstrap sample, b, with the same dimensions as i. Since not all facility years in i were represented in b, we ran rDEA on i with weight restrictions and a frontier defined by b. Similarly, for restricted rSDF efficiency estimation, we calculated rSDF parameters in b and applied them to i for bootstrap rSDF efficiency scores . Any efficiency score estimates exceeding 100 % were assigned a value of 100 %.
For each bootstrap resampling, we calculated ENS efficiency estimates and scale-up of ART visits. In total, we used 1000 boot-strapped samples to calculate 95 % uncertainty intervals (UIs) for all estimates of technical efficiency and further expansion for ART.
All analyses were conducted in R 3.2.2 and its Benchmarking 0.26 package.
Role of funding
This work was supported by two grants funded by the Bill & Melinda Gates Foundation: ‘Assessing the determinants of cost-effectiveness of ART and HIV prevention programs in Kenya, Uganda and select states in India’ and the ‘Disease Control Priorities Network’. The funder was not involved in data collection, analysis, or review of the final results, nor did the funder have a role in the decision to submit this manuscript for publication.
Across countries, facility inputs and outputs varied markedly (Table 1). Facility averages for staff and patient volumes generally increased alongside facility size, but within platforms, sizeable differences existed. For instance, in Zambia, the second-largest and largest facilities averaged about 7100 and 31,300 ART visits, respectively, but each platform type also featured one facility with over 80,000 ART visits.
In calculating averages of technical efficiency across platforms and countries, two main findings emerged (Fig. 1): (1) average efficiency scores were relatively low and (2) each country demonstrated massive within-platform heterogeneity. Across all facilities, Uganda had the highest average efficiency score (40 %; 95 % UI, 33–47 %), followed by Zambia (39 %; 95 % UI, 37–49 %) and Kenya (34 %; 95 % UI, 30–42 %). We found that 64 % of facilities recorded efficiency scores of 50 % or lower, and 95 % of facilities fell below 80 % efficiency. Among facilities which provided ART, average efficiency scores were somewhat higher: 50 % (95 % UI, 48–62 %) in Uganda, 59 % (95 % UI, 53–67 %) in Zambia, and 51 % (95 % UI, 48–58 %) in Kenya. For facilities with ART, 45 % had efficiency scores of 50 % or lower and 94 % performed below 80 % efficiency. For each country and platform, there were a number of facilities with efficiency scores lower than 20 % and at least one facility scoring 85 % or higher. Table 2 provides detailed comparisons of efficiency by country and platform.
Based on our multivariate analysis of efficiency and facility indicators (Tables 3 and 4), we found that most indicators were not significantly related to facility efficiency or did not show consistently significant associations with facility efficiency scores across countries or platforms. When stratified by platform, for instance, having a high proportion of FTEs staffed by doctors had a significant, positive association with higher levels of efficiency among facilities with zero beds, whereas the opposite was true – a significant, negative association with efficiency – among facilities with 1 to 15 beds. The log of reported catchment populations was significantly related to higher efficiency for two facility sizes (zero beds and 1 to 15 beds), while this relationship was attenuated for larger facilities. Among facilities with 1 to 15 beds and 16 to 50 beds, we found a significant, positive relationship between public ownership and higher efficiency scores. On the other hand, for facilities with 16 to 50 beds, having a high fraction of FTEs staffed by volunteers or externally funded personnel had a significant, negative relationship with facility-level efficiency. When facilities were pooled across platforms and stratified by country, a mixture of results emerged. Across countries, facilities with 1 to 15 beds had a significant, positive association with efficiency, whereas a less consistent relationship was found for facilities with 16 to 50 beds. Three facility characteristics had significant, positive associations with efficiency scores in Uganda: public ownership, log of the facility catchment population, and fraction of FTEs staffed by doctors. However, across the other countries, these indicators were neither significant nor consistently related to efficiency. In Zambia, the fraction of FTEs staffed by volunteer or externally funded personnel had a significant, negative relationship with efficiency. Regression results from our bivariate analyses were far less stable and conclusive, as detailed in Additional file 4: Appendix S4.
Figure 2 illustrates the potential for ART expansion across efficiency improvement scenarios. For the scenario where all facilities with ART reached at least 50 % efficiency, we estimated a 3 % (95 % UI, 1–9 %) increase in annual ART visits in Kenya, 12 % (95 % UI, 1–23 %) in Uganda, and 9 % (95 % UI, 2–21 %) in Zambia. If all facilities with ART and efficiency scores less than 80 % reached 80 % efficiency, we estimated a 33 % (95 % UI, 19–48 %) rise in Kenya, 62 % (95 % UI, 21–78 %) in Uganda, and 33 % (95 % UI, 18–65 %) in Zambia. The latter scenario, all facilities with ART reaching 80 % efficiency, would equate to an additional 1.56 million ART visits (95 % UI, 0.871–2.25 million) in Kenya, 1.28 million (95 % UI, 0.444–1.63 million) in Uganda, and 1.77 million (95 % UI, 0.884–4.43 million) in Zambia. Table 5 details efficiency improvement scenarios and corresponding estimates of ART expansion by country. If all facilities providing ART reached at least 80 % efficiency for these countries, we estimated 459,000 new ART patients could be seen, in accordance with national recommendations on ART visit frequency during the first year of treatment [44–46].
Our study showed that health facilities in Kenya, Uganda, and Zambia had relatively low technical efficiency, with each country averaging efficiency scores below 50 %. Further examination revealed massive heterogeneity, with facilities registering efficiency scores ranging from near 0 % to 95 % across levels of care. Most facility-based indicators were not significantly or consistently correlated with higher efficiency, suggesting that other characteristics, such as management practices, may be more closely linked to increased efficiency. In considering facility potential for HIV service expansion, we assessed the potential impact of efficiency gains, given observed resources at facilities providing ART, across improvement scenarios. For instance, if all facilities with ART and efficiency scores below 80 % reached 80 % efficiency, we predicted that ART visits could increase by 33 % in Kenya, 62 % in Uganda, and 33 % in Zambia. These results quantify the capacity for improved resource use and further ART scale-up in sub-Saharan Africa.
WHO recently updated its ART guidelines, recommending that everyone living with HIV initiate ART . This move toward universal HIV treatment aims to significantly improve patient outcomes and curb transmission, yet, with stagnated HIV funding and millions more now eligible for ART [2, 3, 5], policymakers face tough decisions about how to pay for this influx of patients. We found that facilities in Kenya, Uganda, and Zambia could accommodate more ART initiates and continue care for established patients if their efficiency improved. In fact, if all facilities with ART reached at least 80 % efficiency in these three countries, we estimated 459,000 new ART patients could be seen at these facilities. This would represent a 40 % rise in new ART patients and progress toward reducing treatment gaps in Kenya, Uganda, and Zambia, where an estimated 2.6 million people were living with HIV and lacked ART in 2013 .
During the early to mid-2000s, global health initiatives sought to quickly fund and roll-out HIV services in sub-Saharan Africa . These efforts, supported by a rapid escalation of DAH [2, 3], were critical to slowing the rampant spread of HIV and scaling up life-saving treatment. However, the eight-fold increase of DAH during this era has leveled off [2, 3], and HIV treatment now more closely resembles that of chronic conditions, requiring coordinated care within health systems to support both new and established ART patients. Improving how programs produce HIV services and strengthening their efficiency should now be front-and-center, particularly as the focus of both global and national policymakers shifts to maximizing HIV investments and building sustainable systems [4, 6].
Our analysis of potential determinants of efficiency could not definitively pinpoint a set of facility drivers of heightened technical efficiency across countries and platforms. Reported facility catchment population, an indicator of the population size a facility serves, was one of the only determinants that had a significant, positive relationship with higher levels of efficiency among smaller facilities. This result is not necessarily surprising, as smaller facilities (those with 15 or fewer beds) with bigger catchment populations may be located in areas where few, if no other, facilities exist to support the populations residing within them. Further, these facilities may have to accommodate more patients without experiencing a corresponding increase in medical staff, which could lead to higher efficiency scores. In many countries, including Kenya and Uganda, staffing levels are established by the Ministry of Health or other centralized health agencies [47, 48], and thus are not easily amenable to efficiency improvement, particularly at the facility level. Facility ownership, specifically public or government ownership, was another indicator for which we observed significant, positive associations with efficiency among facilities with 1 to 15 beds and 16 to 50 beds. Kenya, Uganda, and Zambia have all abolished or reduced user fees for patients seeking care at public facilities, especially for primary care and ART services [49–54]; such efforts to minimize patient medical expenses may have contributed to heightened utilization at public facilities and thus higher levels of service production. Again, however, this indicator – facility ownership – does not lend itself to facility-level initiatives to address inefficiencies.
We subsequently view more operational components of health service production as strong candidate areas for intervention at the facility level. The Institute for Healthcare Improvement emphasizes strengthening internal processes around the primary customer (patients) . For instance, tailoring HIV services around ART patient stability, such as lengthening the time between ART visits for stable patients, could free facility resources for expanding HIV services or intensifying care for less stable patients . Routinely collecting and reviewing data on facility operations may also help to ensure that resources optimally align with patient demand . These data then can be used to more effectively deploy resources where they are needed, and identify opportunities to improve health service access . A recent UNAIDS report documents the potential impact of ART efficiency gains at the facility level , which include using HIV “hotspot” mapping to inform facility resource allocation in Zimbabwe and integrating HIV services with other health programs in Sudan. Efforts to expand overall health system access and utilization have been a primary emphasis of past efficiency studies in sub-Saharan Africa, stressing that heightened patient demand will in turn increase facility efficiency [16, 19]. We fully agree that improving health system use has numerous benefits; nevertheless, we do not view increasing demand and facility efficiency as mutually exclusive policy decisions. Instead, our findings highlight the largely untapped potential for improving within-facility processes to extend the reach of limited resources.
Relatedly, increasing system efficiency needs to be considered alongside goals for service equity and quality . Reaching maximum efficiency may not be every facility’s goal, particularly those focused on serving hard-to-reach populations or providing highly-specialized medical services. In our study, we excluded a number of these more specialized facilities, as we did not want to assess their production levels alongside less comparable facilities. Further, the view of greater technical efficiency and equity as inherently divergent health system goals is fading, particularly as plateaued HIV-specific DAH and mounting funding gaps strengthen calls for achieving ‘more with less’ against AIDS [7, 31, 59]. In fact, given current financial projections, UNAIDS stresses the need for efficiency gains to effectively reach high-risk individual and populations who currently lack ART . It is also critical to ensure that facilities are not maximizing efficiency at the expense of service quality. An important avenue for improving both efficiency and quality is linking ART patient outcomes, such as viral load measures and program retention, with measures of efficiency. Future work should seek to explicitly link patient outcomes with the full range of facility inputs and outputs of interest.
This work should be viewed within the context of the limitations encountered, which were largely due to data issues and methodological challenges. First, facility data quality varied across countries and between facilities. We conducted extensive data cleaning procedures and used multiple imputation to limit this bias , but it is possible that inputs and outputs were not completely exhaustive or comparable. Second, for our analysis of efficiency determinants, we sought to include a full range of potential drivers of heightened efficiency, but some key factors, such as personnel absenteeism, may not have been adequately captured. Third, determining which specialized facilities to exclude from our analysis was largely informed by comments from survey administrators, and thus may not reflect all conceivable outliers. Fourth, our efforts to account for service quality were limited to structural facility indicators, as our datasets lacked linkages between services provided and desired patient-level outcomes such as viral load suppression. Our facility datasets also included few indicators on facility process or medical staff quality, such as competency tests, which further limited our quality adjustments primarily to structural components. In addition, this proxy quality measure was based on observations of facility supplies and equipment at the time of survey and was then applied retrospectively to our panel data. This analytic necessity may have resulted in skewed estimates of efficiency during earlier facility years, an issue that contributed to our decision to limit reported results to most recent facility-years. Fifth, we were unable to account for case mix at the facility level, an issue that may have resulted in underestimating efficiency for facilities catering to severely ill patients. We considered constructing a proxy indicator for facility case mix based on patient clinical information such as CD4 count at ART initiation, particularly since a subset of facilities in our analysis had linked, de-identified clinical chart data for ART patients [35–37]. However, due to substantive sampling and chart data limitations (e.g., in Kenya and Uganda, approximately 20 % of ART patients lacked CD4 count records at initiation in 2012 ), we determined that its application for the present study risked introducing notable bias. Future analyses should explore avenues for synthesizing different sources of data, including alternative facility data types or geospatial estimates, with facility assessments of efficiency.
In terms of methodological limitations, our novel approach, an ensemble model of rDEA and rSDF , has not been extensively tested on routine health system data. The model was developed in a simulation environment to improve on past measurement techniques and challenges posed by model choices . By combining results from rDEA and rSDF, we aimed to harness each method’s strengths and offset their deficiencies [11, 33]. Second, our efficiency estimates reflected overall facility production, which may not correspond with a facility’s ART clinic efficiency levels . We could not explicitly assign outputs to ART clinics within facilities, but future work should consider how efficiency may differ within facility sub-clinics. Finally, we assumed a demand for scaling up ART alongside other health services (radial expansion) and selected facility inputs, which may not necessarily reflect a country’s policy options for expanding ART or improving efficiency. For instance, program managers may target some facilities for increasing ART volumes while keeping inpatient services constant, or policymakers may introduce task-shifting initiatives to further elevate program efficiency. It is possible that these approaches, if implemented, could support greater ART expansion than what we estimated; nonetheless, our model’s parameters can be adjusted to account for production preferences.
More countries have committed to achieving universal ART and global guidelines now stipulate treatment for all people living with HIV, irrespective of their disease progression. At a time when international HIV funding has stagnated and health systems are preparing to accommodate millions of newly-eligible ART patients, stretching each health dollar is vital. In applying novel methods developed to measure technical efficiency in LMICs, we found that the majority of health facilities providing ART in Kenya, Uganda, and Zambia could considerably expand ART services by increasing efficiency. Our findings emphasize the importance of how facility resources are used, rather than their sheer quantity, and how maximizing their use could notably extend the reach of life-saving treatment to all populations affected by HIV.
ABCE, Access, Bottlenecks, Costs, and Equity; ART, antiretroviral therapy; DAH, development assistance for health; ENS, ensemble method; FTE, full-time equivalent; LMICs, low- and middle-income countries; rDEA, restricted Data Envelopment Analysis; rSDF, restricted Stochastic Distance Function; UI, uncertainty interval; WHO, World Health Organization
Murray CJL, Ortblad KF, Guinovart C, Lim SS, Wolock TM, Roberts DA, et al. Global, regional, and national incidence and mortality for HIV, tuberculosis, and malaria during 1990–2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet. 2014;384:1005–70.
Dieleman JL, Schneider MT, Haakenstad A, Singh L, Sadat N, Birger M, et al. Development assistance for health: past trends, associations, and the future of international financial flows for health. Lancet. 2016;387(10037):2536–44.
Schneider MT, Birger M, Haakenstad A, Singh L, Hamavid H, Chapin A, et al. Tracking development assistance for HIV/AIDS: the international response to a global epidemic. AIDS. 2016;30:1475–9.
Atun R, Bataringaya J. Building a durable response to HIV/AIDS: implications for health systems. J Acquir Immune Defic Syndr. 2011;57 Suppl 2:S91–5.
World Health Organization. Guideline on when to start antiretroviral therapy and on pre-exposure prophylaxis for HIV. Geneva: WHO; 2015.
Joint United Nations Programme on HIV/AIDS (UNAIDS). On the Fast-Track to end AIDS by 2030: Focus on location and population. Geneva: UNAIDS; 2030. p. 2015.
Dutta A, Barker C, Kallarakal A. The HIV treatment gap: estimates of the financial resources needed versus available for scale-up of antiretroviral therapy in 97 countries from 2015 to 2020. PLoS Med. 2015;12:e1001907.
Murray CL. Maximizing antiretroviral therapy in developing countries: the dual challenge of efficiency and quality. JAMA. 2015;313:359–60.
Yip W, Hafez R. Reforms for improving the efficiency of health systems: lessons from 10 country cases. Geneva: WHO; 2015.
Dieleman JL, Templin T, Sadat N, Reidy P, Chapin A, Foreman K, et al. National spending on health by source for 184 countries between 2013 and 2040. Lancet. 2016;387(10037):2521–35.
Hollingsworth B, Peacock S. Efficiency measurement in health and health care. London: Routledge; 2008.
World Health Organization. World Health Report 2010. Health Systems Financing: the Path to Universal Coverage. Geneva: WHO; 2010.
Kirigia JM, Lambo E, Sambo L. Are public hospitals in Kwazulu-Natal Province of South Africa technically efficient? Afr J Health Sci. 2000;7:25–32.
Kirigia JM, Sambo LG, Scheel H. Technical efficiency of public clinics in Kwazulu-Natal Province of South Africa. East Afr Med J. 2001;78:S1–S13.
Kirigia JM, Emrouznejad A, Sambo LG. Measurement of technical efficiency of public hospitals in Kenya: using data envelopment analysis. J Med Syst. 2002;26:39–45.
Kirigia JM, Emrouznejad A, Sambo LG, Munguti N, Liambila W. Using data envelopment analysis to measure the technical efficiency of public health centers in Kenya. J Med Syst. 2004;28:155–66.
Osei D, d’Almeida S, George MO, Kirigia JM, Mensah AO, Kainyu LH. Technical efficiency of public district hospitals and health centres in Ghana: a pilot study. Cost Eff Resour Alloc. 2005;3:9.
Zere E, Mbeeli T, Shangula K, Mandlhate C, Mutirua K, Tjivambi B, et al. Technical efficiency of district hospitals: evidence from Namibia using data envelopment analysis. Cost Eff Resour Alloc. 2006;4:5.
Masiye F. Investigating health system performance: an application of data envelopment analysis to Zambian hospitals. BMC Health Serv Res. 2007;7:58.
Kirigia JM, Emrouznejad A, Cassoma B, Asbu EZ, Barry S. A performance assessment method for hospitals: the case of municipal hospitals in Angola. J Med Syst. 2008;32:509–19.
Akazili J, Adjuik M, Jehu-Appiah C, Zere E. Using data envelopment analysis to measure the extent of technical efficiency of public health centres in Ghana. BMC Int Health Hum Rights. 2008;8:11.
Tlotlego N, Nonvignon J, Sambo LG, Asbu EZ, Kirigia JM. Assessment of productivity of hospitals in Botswana: a DEA application. Int Arch Med. 2010;3:27.
Sebastian MS, Lemma H. Efficiency of the health extension programme in Tigray. Ethiopia: a data envelopment analysis. BMC Int Health Hum Rights. 2010;10:16.
Marschall P, Flessa S. Efficiency of primary care in rural Burkina Faso. A two-stage DEA analysis. Health Econ Rev. 2011;1:5.
Kirigia JM, Sambo LG, Renner A, Alemu W, Seasa S, Bah Y. Technical efficiency of primary health units in Kailahun and Kenema districts of Sierra Leone. Int Arch Med. 2011;4:15.
Kirigia JM, Asbu EZ. Technical and scale efficiency of public community hospitals in Eritrea: an exploratory study. Health Econ Rev. 2013;3:6.
Masiye F, Mphuka C, Emrouznejad A. Estimating the Efficiency of Healthcare Facilities Providing HIV/AIDS Treatment in Zambia: a Data Envelopment Approach. In: Emrouznejad A, Cabanda E, editors. Managing Service Productivity: Using Frontier Efficiency Methodologies and Multicriteria Decision Making for Improving Service Performance. Berlin: Springer; 2014.
Zeng W, Rwiyereka AK, Amico PR, Ávila-Figueroa C, Shepard DS. Efficiency of HIV/AIDS health centers and effect of community-based health insurance and performance-based financing on HIV/AIDS service delivery in Rwanda. Am J Trop Med Hyg. 2014;90:740–6.
Jehu-Appiah C, Sekidde S, Adjuik M, Akazili J, Almeida SD, Nyonator F, et al. Ownership and technical efficiency of hospitals: evidence from Ghana using data envelopment analysis. Cost Eff Resour Alloc. 2014;12:1.
Alhassan RK, Nketiah-Amponsah E, Akazili J, Spieker N, Arhinful DK, Rinke de Wit TF. Efficiency of private and public primary health facilities accredited by the National Health Insurance Authority in Ghana. Cost Eff Resour Alloc. 2015;13:23.
Obure CD, Jacobs R, Guinness L, Mayhew S, Vassall A. Does integration of HIV and sexual and reproductive health services improve technical efficiency in Kenya and Swaziland? An application of a two-stage semi parametric approach incorporating quality measures. Soc Sci Med. 2016;151:147–56.
Ledikwe JH, Grignon J, Lebelonyane R, Ludick S, Matshediso E, Sento BW, et al. Improving the quality of health information: a qualitative assessment of data management and reporting systems in Botswana. Health Res Policy Syst. 2014;12:7.
Di Giorgio L, Flaxman AD, Moses MW, Fullman N, Hanlon M, Conner RO, et al. Efficiency of health care production in low-resource settings: a Monte-Carlo simulation to compare the performance of data envelopment analysis, stochastic distance functions, and an ensemble model. PLoS One. 2016;11:e0147261.
Institute for Health Metrics and Evaluation (IHME). Access, Bottlenecks, Costs, and Equity (ABCE) Project | GHDx. http://ghdx.healthdata.org/series/access-bottlenecks-costs-and-equity-abce-project. Accessed 25 Mar 2016.
Action Africa Help International, Institute for Health Metrics and Evaluation, Kenya Ministry of Medical Services, Kenya Ministry of Public Health and Sanitation. Access, Bottlenecks, Costs, and Equity (ABCE) project in Kenya, 2012. Seattle, WA: IHME; 2015
Infectious Diseases Research Collaboration, Institute for Health Metrics and Evaluation, Makerere University, Uganda Ministry of Health. Access, Bottlenecks, Costs, and Equity (ABCE) project in Uganda, 2012. Seattle, WA: IHME; 2015.
University of Zambia, Institute for Health Metrics and Evaluation, Zambia Ministry of Health, Central Statistical Office, Churches Health Association of Zambia, Clinton Health Access Initiative. Access, Bottlenecks, Costs, and Equity (ABCE) project in Zambia, 2011–2012. Seattle, WA: IHME; 2015.
Institute for Health Metrics and Evaluation. Access, Bottlenecks, Costs, and Equity: ABCE Project Cross-Country Protocol. Seattle, WA: IHME; 2015.
Institute for Health Metrics and Evaluation. Health Service Provision in Kenya: Assessing Facility Capacity, Costs of Care, and Patient Perspectives. Seattle, WA: IHME; 2014.
Institute for Health Metrics and Evaluation. Health Service Provision in Uganda: Assessing Facility Capacity, Costs of Care, and Patient Perspectives. Seattle, WA: IHME; 2014.
Institute for Health Metrics and Evaluation. Health Service Provision in Zambia: Assessing Facility Capacity, Costs of Care, and Patient Perspectives. Seattle, WA: IHME; 2014.
Cooper WW, Seiford LM, Zhu J. Data Envelopment Analysis: History, Models, and Interpretations. In: Cooper WW, Seiford LM, Zhu J, editors. Handbook on Data Envelopment Analysis. New York: Springer; 2011. p. 1–39.
Portela MCAS, Thanassoulis E. Zero weights and non-zero slacks: different solutions to the same problem. Ann Oper Res. 2006;145:129–47.
Coelli T. An introduction to efficiency and productivity analysis. Berlin: Springer; 2005.
Horrace WC, Richards SO. Bootstrapping Efficiency Probabilities in Parametric Stochastic Frontier Models. Syracuse, NY: Syracuse University; 2004.
Piot P, Coll Seck AM. International response to the HIV/AIDS epidemic: planning for success. Bull World Health Organ. 2001;79:1106–12.
Uganda Ministry of Health. Health Sector Strategic Plan II 2005/06-2009/10. Kampala: MOH; 2005.
Kenya Ministry of Medical Services, Kenya Ministry of Public Health and Sanitation. Accelerating Attainment of Health Goals: the First Kenya Health Sector Strategic & Investment Plan (KHSSP), 2012–2018. Nairobi: MOMS, MOPHS; 2012.
Kenya Ministry of Health. Reversing the Trends: the Second National Health Sector Strategic Plan of Kenya (NHSSP II), 2005–2010. Nairobi: MOH; 2005.
Kenya National AIDS Control Council. Kenya National AIDS Strategic Plan: 2009/10–2012/13: Delivering on Universal Access to Services. Nairobi: NACC; 2009.
Orem JN, Mugisha F, Kirunga C, Macq J, Criel B. Abolition of user fees: the Uganda paradox. Health Policy Plan. 2011;26:ii41–51.
Uganda AIDS Commission. Country Response: National AIDS Policy and Framework. Kampala: UAC; 2003.
Masiye F, Chitah BM, McIntyre D. From targeted exemptions to user fee abolition in health care: experience from rural Zambia. Soc Sci Med. 2010;71:743–50.
UNAIDS. Zambia Country Report: Monitoring the Declaration of Commitment on HIV and AIDS and Universal Access. Geneva: UNAIDS; 2012.
Institute for Healthcare Improvement. Going Lean in Health Care. IHI Innovation Series white paper. Cambridge, MA: IHI; 2005. http://www.ihi.org/resources/Pages/IHIWhitePapers/GoingLeaninHealthCare.aspx. Accessed 25 Mar 2016.
The Global Fund to Fight AIDS, Tuberculosis and Malaria. A Toolkit for Health Facilities: Differentiated Care for HIV and Tuberculosis. Geneva: The Global Fund; 2015.
McKinsey & Company. Strengthening sub-Saharan Africa’s health systems: A practical approach. McKinsey & Company; 2010. http://www.mckinsey.com/insights/health_systems_and_services/strengthening_sub-saharan_africas_health_systems_a_practical_approach. Accessed 25 Mar 2016.
Cleary S. Equity and efficiency in scaling up access to HIV-related interventions in resource-limited settings. Curr Opin HIV AIDS. 2010;5:210–4.
Reidpath DD, Olafsdottir AE, Pokhrel S, Allotey P. The fallacy of the equity-efficiency trade off: rethinking the efficient health system. BMC Public Health. 2012;12:S3.
Dansereau E, Gakidou E, Ng M, Achan J, Burstein R, DeCenso B, et al. Trends and determinants of antiretroviral therapy patient monitoring practices in Kenya and Uganda. PLoS One. 2015;10:e0135653.
We appreciate the project support and funding provided by the Bill & Melinda Gates Foundation. In particular, we thank Damian Walker for his support and inputs. We are also grateful to the numerous individuals who contributed to initial project development, including Mickey Chopra of the World Bank and Rudy Knippenberg of UNICEF, and those who provided valuable review and feedback on specific results, including Stefano Bertozzi, Mead Over, and the UNAIDS/World Bank HIV Economics Reference Group. We are incredibly grateful to the Ministries of Health in Kenya, Uganda, and Zambia, whose overall support, willingness to facilitate data access, and provision of crucial content knowledge made this study possible. We are deeply appreciative of all ABCE field teams, who led data collection in Kenya, Uganda, and Zambia. In particular, we thank Ann Thuo and Dinah Nioroge of Action Africa-Help International, and Florence Alanyo, Gertrude Abbo, Grace Akalo, James Okello, Paul Bazongere, and Stella Namuwaya of the Infectious Diseases Research Collaboration. We especially thank all of the health facilities and their staff, who generously gave of their time and facilitated the sharing of facility data. We thank all ABCE team members at the Institute for Health Metrics and Evaluation (IHME), especially Santosh Kumar, now of Sam Houston State University, for their contributions throughout the project’s duration. We also thank all former IHME research fellows who contributed to ABCE data collection, verification, and processing: Katherine Andrews, Benjamin Brooks, Benjamin Campbell, Emily Carnahan, Laura Dwyer-Lindgren, Rashmi Jasrasaria, Nicole Johns, Katherine Lofgren, Leslie Mallinger, and Katrina Ortblad. We thank Abby Chapin for project management, and John Everett Mumford and Erika Eldrenkramp for their contributions to efficiency literature review and analyses. We also thank Timothy Wollock and Austin Carter for their assistance in providing ART data from the Global Burden of Disease study.
This research was carried out as part of a larger project, the ‘Disease Control Priorities Network (DCPN)’, and a supplementary grant focused on ART programs, ‘Assessing the determinants of cost-effectiveness of ART and HIV prevention programs in Kenya, Uganda and select states in India’, both funded by the Bill & Melinda Gates Foundation.
Availability of data and materials
All facility data used in the present study are publicly available and can be downloaded through the Global Health Data Exchange (GHDx): http://ghdx.healthdata.org/series/access-bottlenecks-costs-and-equity-abce-project. Corresponding code also can be accessed through the GHDx: http://ihmeuw.org/ABCE_efficiency.
LDG, MWM, and AW contributed to all parts of the analysis, produced the tables and figures, and wrote the initial draft of the manuscript. NF led manuscript writing, and contributed to the interpretation of results and figure production. ROC and MH contributed to initial methods development and analysis. TA, JA, AG, GI, CK, MM, FM, CM, PN, TAO, and EAO led country-level facility data collection and interpretation of results. KAB, RB, ED, BD, KD, EG, AH, AJL, SHM, and DAR contributed to project development and management, survey piloting, data collection and data verification, and analyses underlying key inputs into the present study. HCD contributed to structural quality adjustment analyses and interpretation of results. CJLM and ADF conceptualized the project, contributed to methods development, and provided guidance on data analysis and manuscript writing. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Ethics approval and consent to participate
Ethical approval for this study was obtained from the University of Washington Human Subjects Division, Kenya Medical Research Institute Ethical Review Committee, Makerere University School of Medicine Research and Ethics Committee, and the University of Zambia Biomedical Research Ethics Committee.
Access, Bottlenecks, Costs, and Equity (ABCE) facility sampling strategy by country. This supplementary file details the sampling strategies adapted for Kenya (Figures A and B), Uganda (Figures C and D), and Zambia (Figures E and F) as part of the multi-country ABCE project. (DOCX 1409 kb)
Output-specific facility indicators for quality-adjustment scores. This supplementary file provides detailed information about output-specific quality indicators used to construct structural quality-adjustment scores in Kenya, Uganda, and Zambia (Table A). (DOCX 19 kb)
Sensitivity analyses. This supplementary file describes the sensitivity analyses used to test model performance using ART patients versus ART patient visits (Figure G and Table B), technical efficiency scores that were adjusted versus unadjusted for structural quality (Figure H and Table C), and variable approaches to estimating technical efficiency (Figure I and Table D). (DOCX 140 kb)
Bivariate and pooled analyses of facility determinants of efficiency. Results from bivariate regressions by country (Tables E to G) and pooled across both countries and platforms (Tables H and I) are included in this supplementary file. (DOCX 40 kb)