Analysing the role of complexity in explaining the fortunes of technology programmes: empirical application of the NASSS framework

Introduction

Technological innovation is viewed by policymakers as a driver of both health and wealth [1]. Technology is often depicted as ’empowering’ for both patients and staff, and has been associated with improved efficiency, quality and safety of care [2–5]. In reality, however, technology start-ups may fail to attract investment [6]; patients may or may not be able or willing to use new technologies [7]; professionals may resist them [8–10]; new technologies may clash with legacy systems and with established routines [11, 12]; a technology may be implemented but fail to deliver the anticipated benefits [13]; and small-scale demonstration projects may fail to scale up locally, spread distantly or be sustained over time [14, 15].

In a recently published systematic review, we synthesised evidence on individual, team, organisational and system influences on the success of technology-supported innovation programmes in health and social care [16]. We drew in particular on published technology implementation frameworks and key theoretical work on diffusion of innovations [17, 18], technological entrepreneurship [6, 19], the patient experience of chronic illness [20], clinician resistance to technologies [21], the social processes of ‘normalising’ technologies in organisations [22–25], business and financial planning [14], organisational resilience and sustainability [26–28], and theoretical studies on complex adaptive systems [29, 30].

Our synthesis of this diverse literature occurred in parallel with testing of candidate domains and theories from our systematic review on a sample of six empirical case studies. We produced a new multi-level interdisciplinary framework called NASSS (Non-adoption or Abandonment of technology by individuals and difficulties achieving Scale-up, Spread and Sustainability), which incorporates and extends the theoretical frameworks and models listed in the previous paragraph. The NASSS framework is shown diagrammatically in Panel 1 and Fig. 1.

Panel 1: Domains and questions in the NASSS framework  Domain 1: the condition   1A. What is the nature of the condition or illness?   1B. What are the relevant co-morbidities?   1C. What are the relevant socio-cultural factors?  Domain 2: the technology   2A. What are the key features of the technology?   2B. What kind of knowledge does the technology bring into play?   2C. What knowledge and/or support is required to use the technology?   2D. What is the technology supply model?   2E. Who owns the intellectual property (IP) generated by the technology?  Domain 3: the value proposition and value chain   3A. What is the developer’s business case for the technology (supply-side value)?   3B. What is its desirability, efficacy, safety and cost-effectiveness (demand-side value)?  Domain 4: the adopter system   4A. What changes in staff roles, practices and identities are implied?   4B. What is expected of the patient (and/or immediate carer) — and is this achievable by and acceptable to them?   4C. What is assumed about the extended network of lay carers?  Domain 5: the organisation(s)   5A. What is the organisation’s capacity to innovate?   5B. How ready is the organisation for this technology-supported change?   5C. How easy will the adoption and funding decision be?   5D. What changes will be needed in team interactions and routines?   5E. What work is involved in implementation and who will do it?  Domain 6: the wider system   6A. What is the political context for programme development, implementation and roll-out?   6B. What is the regulatory context?   6C. What is the position of professional bodies?   6D. What is the socio-cultural context (public perception, interest, expectation)?   6E. What is the nature and extent of inter-organisational networking?  Domain 7: Embedding and adaptation over time   7A. How much scope is there for adapting and co-evolving the technology and the service over time?   7B. How resilient is the organisation to handling critical events and adapting to unforeseen eventualities?

The original questions guiding our empirical research were the following. (1) How can we improve the process by which health and care technologies are developed and implemented? (2) How can we support the customisation and use of such technologies in the home and/or the health or care setting? (3) How can we ensure that patients’ needs and concerns remain central when developing technology-supported service innovations [31, 32]? The study of complexity was not part of our original proposal, but it quickly emerged as the dominant theme in our empirical data (as well as a prominent element in the more recent literature we were identifying for our systematic review [29, 30]).

Our previous publication focused mainly on the secondary research component of the NASSS framework [16]. This paper presents a more detailed account of our empirical findings and illustrates how the framework allowed us to explore complexity in multiple interacting domains.

Complex systems and the NASSS framework

A system is defined as an assembly of agents that interact with each other. In a simple system (few agents and components) or a complicated one (many agents and components), the relationships between agents are well defined and stable, which means the overall behaviour of the system is predictable. In contrast, complex, adaptive systems are composed of agents with ill-defined and unstable boundaries that may act in unexpected ways, whose actions are interconnected so that one agent’s actions change the context for other agents [33]. Hence, in complex systems, agents interact with each other and with other systems in unexpected ways; their component agents (people, cells, technologies) can simultaneously be members of several systems. The complex system works by a fundamentally different logic, since its response to change is—to a greater or lesser extent—unpredictable and non-linear [34].

Against this background, the NASSS framework has been developed to encourage complex thinking (what Tsoukas calls ‘conjunctive theorising’ [35]) about technological innovations in healthcare. With a view to generating a rich narrative of events unfolding in a real-world setting, aspects of the different domains are first considered in terms of whether they are simple (straightforward, predictable, few components—as in making a sandwich), complicated (remains predictable but there are multiple interacting components or issues—as in building a rocket) or complex (dynamic, unpredictable, not easily disaggregated into constituent components—as in raising a child). Simple and complicated phenomena operate according to linear, Newtonian (predictive, cause-and-effect) logic; they can, for example, be meaningfully analysed in terms of their component parts. Complex phenomena operate according to different (non-linear) logic, in which a given cause may not always have the same effect. They exhibit broad patterns and emerge over time, but they are not predictable.

A simple illness or condition (domain 1 in the NASSS framework) is well characterised, well understood and predictable (though it may still be serious, e.g. deep venous thrombosis); its management is straightforward and is influenced only minimally by co-morbidities or socio-cultural factors. The same goes for a complicated condition (e.g. many cancers), though the logistics may be more challenging. Complex conditions, in contrast, are poorly characterised, poorly understood, unpredictable and/or strongly influenced by co-morbidities and socio-cultural factors (e.g. drug dependency, dementia).

The complexity dimension in domain 2 (technology) may refer to the technology’s material (including technical) properties, its ease of use, the kind of data it generates, its supply chain or the intellectual property associated with it. In all these sub-domains, complexity—which is impossible to define in rigid and universal terms—may relate to changeability, unpredictability, contestability (e.g. experts disagree on what the data mean and whether they can be trusted) and interdependence with other changeable, unpredictable or contestable aspects (such as availability of broadband).

The value proposition (domain 3) refers to both supply-side value (whether there is a straightforward and uncontested business case for generating revenue for the developer) and demand-side value (whether there is strong and uncontested evidence that the technology is desirable for patients, effective, safe and cost-effective). Complexity in this domain relates to (for example) multiple and perhaps interdependent assumptions on which the business case is based, a speculative or contested evidence base for effectiveness or cost-effectiveness, or gaps in the overall value chain.

Complexity in relation to intended adopters (domain 4) does not mean merely that some individuals need to learn new skills or procedures or adopt new staff roles. More challenging is the expectation that a staff member, patient or lay carer will need to take on a different identity (e.g. data processer, teacher) alongside their traditional one and/or make judgements that are difficult or unpredictable.

Complexity in the organisational setting (domain 5) relates in particular to the scope, scale, pace, resource requirements, and the logistical uncertainties and interdependencies of delivering the innovation and the associated new service model [36]. The introduction of a ’disruptive’ technology (that is, one that enables—and perhaps requires—organisational work to be done differently [37]) will be complex if known preconditions for innovation are not met (e.g. if there is weak leadership, poor managerial relations or severely stretched resources); if the technology is a poor strategic ‘fit’; if new work routines and/or inter-organisational co-operation are needed; or if a large amount of work is needed to build a vision, engage staff, implement the programme and develop ways of monitoring its impact.

Complexity in the external context (domain 6) means that there are tricky hurdles to be overcome in relation to political, financial, legal, regulatory or public concerns, or that inter-organisational networking and knowledge sharing are difficult. Again, the key issue is often the interdependency of different influences (which tends to mean that any one problem cannot be addressed without generating other problems elsewhere in the system).

Finally, complexity in domain 7 (adaptation over time) means that further adaptation or co-evolution of the technology is impossible because of lack of material or technical flexibility, and/or because the organisation(s) lack the resilience to adapt to changing external conditions (see Discussion).

Such were the findings of our systematic review that formed the theoretical basis of the NASSS framework [16]. In the remainder of this paper, we describe the empirical testing and refining of the NASSS domains across a maximum-variety sample of technology implementation case studies. The specific research question addressed in this paper was: Given that technology programmes in health and social care are often described as ‘complex’, what is the nature of this complexity and how might it affect the fortunes of a programme?

Context, governance and methodology

The research took place in various field sites across the UK. It embraced two research programmes: VOCAL (Virtual Online Consultations—Advantages and Limitations) and SCALS (Studies in Co-Creating Assisted Living Solutions). VOCAL (funded from 2015 to 2017, with an earlier set-up phase from 2011) was an in-depth study of the development, introduction and local roll-out of remote (video) consultations in three contrasting clinical departments, each on a different geographical site, in a large, multi-site UK hospital trust [38, 39]. SCALS (funded from 2015 to 2020, with some data collected from 2013) is an action research study of the challenges faced by UK health and social care organisations who introduce technology-supported new service models; it includes examples from healthcare (e.g. remote biomarker monitoring, video consultations, technologies for integrating care across organisations) and social care (safety alarms, GPS tracking, care organising apps) [32].

Both VOCAL and SCALS had an external steering group with a lay chair and representation from patients, front-line clinicians, the technology industry and local and national policymakers (including information leads at National Health Service (NHS) England). The VOCAL study also had a separate patient advisory group convened and chaired by a community anthropologist.

Case studies (all of which were drawn from VOCAL and SCALS) were sampled by a combination of responsiveness (health or care organisations sought our input to real-world implementation challenges), convenience (local initiatives caught our interest) and theoretical sampling (later cases were systematically sought to illustrate themes that had come up in our literature review but were not yet represented in our sample). The six prospective case studies reported below have so far been followed for up to 3 years. Additional, theoretically sampled case studies in the SCALS programme (added more recently and not reported here) will be explored in future papers.

Each case study has involved a flexible programme of qualitative interviews and observation (with patients, clinicians, managers, technical designers, commercial partners and—where relevant—investors), analysis of documents (correspondence, business plans, clinical records), ethnography (of technology use by patients/clients and staff, of meetings and events, and of technology design and functionality) and video-recording of both ends of remote consultations [31, 32]. In addition, in order to build up a rich picture of the national context in which technologies evolve, we used a combination of purposive and snowball sampling to identify 45 potential stakeholders from across government (e.g. NHS England), professional organisations (e.g. Royal College of Physicians, Medical Protection Society), patient groups (e.g. National Voices), industry (e.g. Microsoft) and charitable and third sector organisations (e.g. Health Foundation). We invited a maximum variety sample of 39 of these stakeholders to talk informally with the study team, of whom we spoke with 36 (the remaining 3 being uncontactable). We conducted formal semi-structured interviews with a purposive sample of 12 of these stakeholders (ensuring variation of groups and perspectives) and combined this with analysis of approximately 50 key national-level policy and policy-related documents published since 2000.

Outline of the six case studies

Case A (video outpatient consultations) included four clinical services in the NHS: adult diabetes, antenatal diabetes and cancer surgery, all using Skype™ [38], and a community-based, nurse-led heart failure service, using predominantly FaceTime™. Video consultation was offered to patients for whom it was judged clinically appropriate by the clinician. There was strong support from senior management and many (though not all) clinicians. Almost all patients volunteering for this option experienced it as convenient, technically straightforward and able to meet their clinical needs. But implementation proved logistically difficult, technically challenging, labour-intensive and slow. Video consulting was considered not clinically appropriate for many patients. By the end of the study period, video consultations had been abandoned in the antenatal diabetes service and put ‘on hold’ in the community heart failure service. In the adult diabetes and cancer surgery services, they continued and were being extended to other clinical services within the trust.

Case B (global positioning system [GPS] tracking for people with cognitive impairment) began when the SCALS team were approached by a local council in a deprived and multi-ethnic inner city borough and asked to help improve the take-up of devices to electronically track people with cognitive impairment who ‘wandered’ outside the home. We worked with the council and with linked voluntary sector groups to implement and adapt a selection of devices and a linked call centre and monitoring service [40, 41]. Whilst several hundred people in the catchment population had cognitive impairment, only 11 were ever identified as eligible for GPS tracking and 7 assented (of which only 3 continue to use the technology). Successful adoption of the technology was found to require a network of extended family and call centre staff who collectively ‘knew’ the client and his or her preferred walking route(s).

Case C (pendant alarms): Pendant alarms are worn around the neck (or on the wrist) and connected to a remote call centre. The client should press the alarm if he/she is in difficulty (e.g. fallen and cannot get up); the call centre will alert either a relative (on a retained contact list) or an emergency service. Pendant alarm services had been in widespread use for some years in two participating organisations in the SCALS study—both urban settings serving a mixed socio-demographic population. Various arrangements were in place for referring clients (including self, GP, social worker and local age charity) and fitting the alarm (typically a commercial supplier). In both sites, alarms were widely supplied and often ‘worked’ as intended, though they depended on a network of carers and professional staff whose collective knowledge of the client allowed them to interpret remote signals (e.g. judge whether a call was an emergency). In many instances, clients did not activate the alarm when care staff and relatives considered that they should have done so.

Case D (telehealth for heart failure) was the qualitative component of a multi-centre randomised controlled trial of biomarker monitoring (weight, blood pressure, heart rate) in heart failure (SUPPORT-HF). All participants in this trial were supplied with a tablet technology through which they could access their biomarker results, trends and educational material [42]. The intervention arm included active communication of results and recommendations to the patient’s general practitioner with the aim of increasing use of recommended medical therapy and improving patients’ well-being; in the control arm, data were available for the general practitioner to access if desired. Across participating sites, clinicians engaged variably with the study, occasionally leading to slower than predicted recruitment. Participants’ use of the technology also varied widely, influenced by various clinical, technical and logistical issues.

Case E (care organising software) followed the fortunes of two software products, each designed to help relatives and friends organise tasks and visits for someone with health and/or care needs. Product A, a web portal, had been developed in-house by a small software company. The business model was to sell the product (at a cost of several thousand pounds) to care organisations that would then provide it to their clients free of charge. Product B was a smartphone app (with linked web portal) that had been developed via publicly funded R&D using co-design methodology by a national carers’ charity; it was made available for individual download (e.g. via the Apple App Store) for £2.99. By the end of the study, very few families were using Product A, but around 7500 were registered to use Product B (a proportion of whom were also receiving a wider package of support from the care charity).

Case F (shared data warehouse for integrated case management of patients at risk of hospital admission) was introduced in 2009 to support a policy of coordinated, multi-disciplinary case management between health and social care services through assessment and care planning. It had been proposed as a solution to the growing challenge of emergency hospital admissions in older people with multiple health and care needs, reflecting national policy [43]. The cross-organisation data warehouse incorporating a predictive risk modelling tool was intended to automate the identification and stratification of people at high risk of hospital admission, and enable shared access to care plans, thus facilitating coordinated action and frequent dialogue between primary and secondary care providers and social services. However, the original vision of ‘integrated care’ achieved through the technology was only partially realised because, in practice, high-risk patients were identified and managed through a combination of risk stratification and data entry (using the technology) and clinical judgement and dialogue (bypassing the technology).

Data analysis and testing of the NASSS framework

We sought to analyse our six case studies both individually [44] and also as a theoretically sampled collection of cases representing maximum diversity in each of the NASSS domains [45]. This work both informed, and was informed by, our ongoing systematic review [16]. For example, the addition of domain 1 to the NASSS framework was prompted by a strong theme in our empirical data that non-adoption and abandonment of technologies were often explained by heterogeneity and unpredictability in the patient’s illness, co-morbidities and socio-cultural background. The addition of Case F (data warehouse) was prompted by the discovery from our secondary research that technologies intended for sharing data between organisations raised unique logistical, technical and professional challenges.

For each of the six case studies, we analysed qualitative data thematically and produced an initial narrative summary. We wove in quantitative data (e.g. uptake and usage rates) as part of that narrative and used longitudinal methods (e.g. repeat interviews, data trends) to build the narrative over time. We held a series of meetings (approximately monthly) to discuss each of the NASSS domains, singly and in combination, as they pertained to each case study—and tested emerging theory against these domains. Our refinement of the NASSS framework, and in particular the generation of key sub-questions within each domain, owed much to these cross-case meetings.

Domain 1: the condition

Differences in the underlying illness largely explained differences in the fortunes of the video consultation option in the four services studied in Case A. Routine check-ups for adults with diabetes and follow-up consultations after cancer surgery were mostly consistent and predictable (i.e. simple), and most unpredictable eventualities were of low risk. By the end of the study period, approximately 20% of all consultations in these clinics were being conducted by video.

In contrast, diabetes in pregnancy was an example of a complex condition. In pregnancy, diabetes tends to be metabolically volatile and if poorly controlled may lead to foetal abnormalities or death. Many pregnant women had developed diabetes only since becoming pregnant, and so were novices in self-administering insulin. The lead physician felt strongly accountable to the unborn child, and so only offered the video option to patients (3% of the total) judged to be ‘low risk’ (for example, those with high health literacy, good technological skills and fluency in English).

Heart failure is a serious, unpredictable and often unstable (hence, complex) illness whose effects vary from patient to patient and in the same patient over time [46]. It is mostly a condition of older people and occurs disproportionately in lower socio-economic groups. One of its common side effects is profound tiredness, and it is almost always associated with other co-morbidities (notably kidney disease, diabetes, depression or cognitive impairment). Heart failure nurses in our study made judgements about the stability or otherwise of the illness and about patients’ co-morbidities (including cognitive ability and mental health status), health and technological literacy, family support and technical set-up at home and motivation. As a result, the video consultation option was deemed inappropriate for many (at the time of writing, fewer than 20 such consultations had been undertaken across a clinic population of several hundred).

Complexity in the underlying condition was also associated with non-adoption, abandonment or limited usefulness of technologies in Cases B and C (in which dementia or multi-morbidity respectively made the patient unable or unwilling to use the supplied technology) and Case F (in which the predictive risk modelling tool selected multi-morbidity as a risk factor for hospital admission, but few such patients proved to be ‘textbook cases’ to manage).

Domain 2: the technology

The technologies used for video consultations, Skype™ and FaceTime™, are both mass-market software packages from large multi-national companies, presenting low risk of supplier withdrawal and relatively straightforward substitutability (hence, in these respects they could be classified as simple or complicated). However, there were elements of complexity. For example, they were run from NHS hardware (sometimes many years old) and from patient-held laptops, tablets or smartphones of varying quality and dependability. The software was sometimes logistically difficult to install on NHS computers (e.g. because of limited capacity of the IT support team and maintain ‘non-standard’ software environments), and even when installed, it was not 100% dependable for both technical (machine ‘crashing’) and human error (e.g. forgotten password) reasons. Workarounds tended to use low-tech, dependable solutions (e.g. community heart failure nurses defaulted to telephone consultations when video connection failed), thus reducing complexity.

Whilst video transmits speech and visual cues reasonably well (although variations in audio and video quality are quite common), clinical examination sometimes needs other modalities. In heart failure care, for example, the physical examination (e.g. blood pressure, heart rhythm, leg oedema) that the nurses considered essential was not easy in the remote environment—though it was sometimes possible with patient and carer assistance when the nurse knew the patient well. There was no easy or automated way of sharing and recording patient-held data such as home blood pressure readings (patients typically read out numbers but some misinterpreted the digital display or viewed it upside down).

The other case studies illustrated different aspects of complexity in the technology. GPS tracking technology (Case B), pendant alarms (Case C) and care organising software (Case E) all relied on bespoke solutions from small and medium-sized enterprise (SME) companies, and hence were vulnerable to withdrawal; these companies sometimes lacked capacity to meet the bureaucratic requirements for potential spread and scale-up. Case D used software that had been developed as part of an academic research study; importantly, the research nurse and technical team were co-located, allowing minor (but potentially critical) technical issues to be resolved in an ongoing way. Case F featured bespoke software supplied through a longstanding relationship with an SME that was subsequently acquired by a global company. These technologies all required considerable knowledge and skill to use to their full potential. An assumption underpinning the design of patient-held assistive technologies was that a group of relatives and/or friends would exist, live locally, be technology-savvy and be able and willing to collaborate around the care of the index case. In fact, such networks were rarely pre-existing; they often had to be built and nurtured. Cases B, C and D highlighted the role of both lay carers and professional staff in helping to set up and ‘service’ the technology and keep it in working order, a role that could be particularly onerous if the technology (as in case B) was not dependable.

The data warehouse for integrated case management in Case F was well embedded organisationally (in the sense that it was enshrined in national policy and local sub-contracts and data sharing agreements, and staff were employed to work on it). But it was not technically well embedded in the sense of seamless interoperability of data between participating organisations; significant workarounds were required to (for example) share care plans. The predictive risk modelling tool generated a different kind of risk estimate than a home visit from a clinician or social worker who knew the individual well and who had the capacity and authority to bear witness to a narrative and make contextual judgements. Our data illustrated that often neither approach alone offered the full picture that was sometimes necessary for making judgements. Thus, the output (risk score) generated by the technology was complex (in the sense that it was incomplete and contested).

Domain 3: the value proposition and overall value chain

The supply-side value proposition for video consulting in Case A currently appears complex. The multi-national companies behind Skype™ and FaceTime™ are also developing multiple other health products, especially directed towards the expanding ‘wellness and wearables’ market. From a purely financial perspective, such direct-to-consumer products may offer a more lucrative supply-side value proposition than investing in a major business venture to support video consultation products and services in the NHS, because working through technical and information governance challenges is a resource-intensive and time-consuming process with no guarantee of meeting shareholder or executive expectations at the end of it. Our interviews also suggest that companies are aware of the potential for reputational risk associated with seeking to profit from virtual consultations in the NHS.

The demand-side value (to patients) of video consultations is also complex, since the evidence base on which it rests is currently sparse. Whilst around 20 randomised controlled trials in a range of conditions have demonstrated equivalent efficacy and safety between video and face-to-face consultations [38], the samples for these trials are likely to have been carefully selected. Members of our VOCAL patient advisory group raised concerns about the risk of a ‘two-tier’ service in which demand-side value for a minority of patients will be gained at the expense of service cuts for the majority—a concern which, though speculative, has recently been echoed by professional bodies, clinicians and patients [47, 48].

In Case D, one aspect of the value proposition (which affects both supply- and demand-side value) is the potential of the data collected to inform the development of predictive algorithms based on biomarker changes over time and hence predict and pre-empt decompensation, hence averting hospital admission (rather than just prompting medication changes on the basis of, say, a rise or fall in blood pressure). This option creates new possibilities for ‘personalised’ medicine, but it also increases complexity. Whilst real-world value is hard to assess in the context of a randomised controlled trial, we note that the promise (or aspiration) of the telehealth package in Case D is highly complex, since it seeks to achieve multiple goals, including: (1) improving heart failure management in the community; (2) reducing demand on services (by allowing nurses to take on higher case loads); (3) preventing unplanned admissions; and (4) developing further predictive capabilities.

Case E illustrates two very different business models (and technology development models) for similar technologies and use cases. In one (Product A), the value proposition was highly speculative and little attempt was made to work with intended end-users to increase the technology’s fitness for purpose (and hence its desirability) during development. The implicit assumption was that the technology would be more or less plug-and-play; the business model rested on provider organisations paying for a block contract even though the intended benefits (and/or savings elsewhere in the system) were not clear. Product B included substantial up-front investment (from publicly funded R&D) to undertake co-design work; it was explicitly developed by a charity as a ‘public good’ in which costs to end-users would be minimal and viewed as a component of a wider (and ongoing) charity-supported package. These projects are both ongoing, but at the time of writing we would classify Product A’s value proposition as complex and Product B’s as complicated.

Domain 4: the adopter system

In the adoption of video consulting (Case A), there was considerable complexity in the adopter system. There was, for example, a striking difference between innovators (who embraced the new technology and way of working with enthusiasm) and other clinicians on the same teams who were reluctant to change, reflecting previous research showing that staff resistance is the single most important reason given for low uptake of remote health care [10, 49]. The hurdle was not merely learning to use a new technology but also accepting changes in identity (e.g. some staff did not view themselves as ‘techy’) and role (e.g. helping a remote patient troubleshoot technical problems with Skype™ or FaceTime™), dealing with perceptions of overload when running a new virtual service in parallel with a traditional face-to-face one and feeling under pressure to realise efficiency gains before the system had been fully redesigned to maximise such gains. New staff roles were not restricted to clinicians directly using the video technology; receptionists, clerks and technicians all had to accommodate new roles that the technology required in order to ‘work’. The same was true of the patient, who could sometimes but not always seek technical help from family members.

The other case studies illustrated additional complexities in the adopter system, such as some staff expressing ethical reservations about ‘tagging’ clients (Case B) or clinical concerns about the data generated by the technology (Case D—some clinicians were worried about possible legal liability if telehealth data, generated elsewhere and impossible to verify directly, were later found to be flawed). In Case C, clients sometimes rejected a pendant alarm because it symbolised dependence or because they were unwilling to pay a small monthly connection fee or be placed in a dependency relationship with a relative or neighbour.

Domain 5: the organisation(s)

The hospital trust that hosted the VOCAL study had strong leadership and good managerial relations; it met key criteria for technological innovativeness (e.g. it had previously won a national ‘Digital Trust of the Year’ award), and there was board-level enthusiasm for the introduction of video consultations. Despite these encouraging preconditions (‘simple’ in our taxonomy), other features of the organisation were highly complex. In particular, it had very limited spare staff time and resources (e.g. key posts were unfilled, and many clinic terminals were running outdated versions of software packages)—a problem known as ‘lack of organisational slack’ [18]. In addition, whilst many senior decision-makers assumed that the new service would save money by making services more efficient, the question of whether a video consultation would actually cost less to deliver was not easy to answer because of potential knock-ons in the system (e.g. the need for additional IT support and staff training; the fact that rooms still needed to be occupied, records retrieved and appointments booked even when the consultation was virtual; and the theoretical possibility of an increase in appointments as clinicians and patients found it easier to connect).

Another feature of complexity was that whilst video consultations between clinicians and selected patients usually worked well, the linked routines for booking appointments, managing the clinic list (e.g. registering when each patient had ‘arrived’ and ‘left’) and organising follow-up did not mesh well with a system that had evolved to process patients using their physical presence (waiting in line at a reception desk), manual transfer of paper records between different plastic ‘bins’ and sticky notes. Alignment with such routines was initially achieved using workarounds; by the end of the study, new (computer-based) routines had been developed by some but not all participating teams. Whilst Skype functionality increased access (by, for example, allowing patients to send messages to the clinician’s Skype account), and whilst this ease of access was sometimes clinically appropriate and encouraged (e.g. “drop me a message to confirm the change of insulin dose was OK”), it generated complexities elsewhere in the system, since the clinician then had to log the message on the medical record.

Similar complexity-related challenges were evident in the community heart failure study. The community trust was a digital innovator and had supplied all nurses with tablet computers to help them with various aspects of their work. Again, whilst video consultations worked well clinically for selected patients, at the time of writing they were not well integrated logistically with the administrative aspects of the service. There was no formal opposition from top management in the community trust to the nurse-led video consultation service. But neither was there strong enthusiasm, and there was limited spare capacity among front-line teams to undertake the work of making sense of the new approach, enrolling and training staff beyond initial enthusiasts, implementing new work routines and evaluating the service.

In our other case studies, enthusiasm at board level was sometimes absent (e.g. because business cases were weak) and/or key opponents were strategically placed and had high wrecking power (specific examples withheld). In Case D, wide variability in clinician engagement among the different study sites could be explained largely in terms of the extent to which the local team shared a vision for how remote biomarker monitoring for heart failure might enhance rather than threaten the existing service, an aspect of implementation work that May calls ‘coherence work’ [22].

In Case F, establishing integrated case management through shared data and predictive risk modelling technology was extremely complex because multiple organisations needed to be involved; the establishment and development of the programme unfolded over several years and relied on partnership working and contracting arrangements at different levels of multiple organisations. The technology was implemented, but the anticipated reduction in costs from reducing hospital admissions were not realised as real savings because of the complexities of reimbursement mechanisms and because case management was not always successful in avoiding admissions.

Domain 6: the wider system

The most significant system-level challenge to the scale-up and spread of video consultations in Case A was that there was no established national tariff for funding such consultations. In the VOCAL study, the local commissioning organisation proposed reimbursing such consultations at a rate (‘pass through tariff’ [50]) intermediate between a telephone consultation and a face-to-face one. But even though members of the relevant national policymaking team were on the study steering group and there was no opposition ’in principle’ to establishing a national tariff, this had still not been achieved at the time of writing—mainly because data to inform costing calculations were difficult to obtain and contested by some parties. This is a good example of how innovative, technology-supported service models can succeed as demonstration projects through local workarounds but will fail to spread or be sustained unless the regulatory and financial context is supportive [51]. Another contextual factor which added complexity in case study A was a mixed reaction from professional bodies, whose enthusiasm for new, potentially more efficient, models of care was tempered by concerns about workload and threats to equity.

Our national-level stakeholder interviews revealed another aspect of complexity relating to the nature and strength of evidence expected by different stakeholder groups. The technology industry typically moves quickly, with a development model based on rapid iterations of technologies and a pragmatic understanding of what works in practice (the ‘fail early, fail often’ principle). Policymakers, in contrast, tend to want what they call ‘gold standard’ evidence (for example, from randomised controlled trials) to ‘prove’ that a particular technology has the impacts claimed. This mismatch appeared to explain some of the slow progress on national-level policy in relation to video consultations.

The acute trust where our video consultation study was based was one of the first public sector providers in the UK to introduce this service model [39]. In the last year of the VOCAL study, more than 50 organisations contacted the lead clinician seeking advice or asking to visit to see the video consultation service in action. This is an example of the important role of inter-organisational networking in supporting the exchange of both explicit and tacit knowledge [18].

Domain 7: adaptation over time

The video consulting services in Case A illustrated both resistance to adaptation (through material limitations and institutionalised information governance regulations) and adaptiveness (through clinicians’ creative and responsive use of the technology). The NHS has a ‘locked-down’ computer environment in which any new hardware or software must be carefully considered and formally approved before being installed or upgraded (a characteristic that reduces complexity for IT managers but tends to increase complexity for front-line staff). Rapidly evolving software sits awkwardly in such an environment. In the VOCAL study, an automated upgrade to Skype™ made the system non-functional on clinical terminals until re-authorised by someone with administrator-level access rights—a problem that resulted in some remote clinics having to be done by telephone.

The material features of Skype™ enabled the development of ‘ad hoc’ consulting in the young adult diabetes clinic, for example, when the patient saw that the clinician was online and sent a text (SMS) message asking a question about a recently changed insulin dosage. The clinician could either reply by SMS message (within Skype™) or offer a real-time video consultation (typically very short). This adaptive use of video technology for patient-initiated consultations was viewed by clinicians as a game-changer for ‘challenging’ patients (characterised by high non-attendance rate at clinic, poor glycaemic control and a history of hospital admission for diabetic emergencies).

Several other cases in our dataset illustrated a similar tension between system rigidity (for contractual or cost reasons) and adaptation (through user creativity). In Case C, for example, potentially remediable problems occurred with some alarms, but adaptation was impossible because of the risk of loss of warranty. A pendant alarm service initially introduced to provide emergency physical support (e.g. for falls) was adapted over time to provide non-emergency emotional support for certain older people, who were encouraged by call centre staff to press the alarm button to trigger a supportive conversation when feeling lonely.

In Case F, following the introduction of the integrated case management data warehouse technology, clinical and administrative staff across different organisations collectively learnt and redefined what this technology could and could not do. They amended, adapted and worked around it. For example, clinicians and practitioners reviewed the outputs of the data-driven risk stratification model but also supplemented them with other data and used their judgement to target additional patients who had not been flagged as ‘high risk’ by the technological algorithm. Technical developers continually updated the user interface through an ongoing relationship with the procuring body. Notwithstanding these efforts, there was a brittleness about the technology (and the work routines it presupposed) that staff experienced as persistently frustrating. Thus, there was a sense that the technology had been ’successfully’ implemented and was for the moment being sustained (because it met policy expectations), but was not truly fit for purpose.