Below, we apply the different domains of the NASSS framework to our case studies before considering (in the Discussion) the implications of our findings in terms of complexity theory. We have presented Case A (video outpatient consultations) in depth and added additional data from other case studies where they add to the granularity of the analysis.
Domain 1: the condition
Differences in the underlying illness largely explained differences in the fortunes of the video consultation option in the four services studied in Case A. Routine check-ups for adults with diabetes and follow-up consultations after cancer surgery were mostly consistent and predictable (i.e. simple), and most unpredictable eventualities were of low risk. By the end of the study period, approximately 20% of all consultations in these clinics were being conducted by video.
In contrast, diabetes in pregnancy was an example of a complex condition. In pregnancy, diabetes tends to be metabolically volatile and if poorly controlled may lead to foetal abnormalities or death. Many pregnant women had developed diabetes only since becoming pregnant, and so were novices in self-administering insulin. The lead physician felt strongly accountable to the unborn child, and so only offered the video option to patients (3% of the total) judged to be ‘low risk’ (for example, those with high health literacy, good technological skills and fluency in English).
Heart failure is a serious, unpredictable and often unstable (hence, complex) illness whose effects vary from patient to patient and in the same patient over time [46]. It is mostly a condition of older people and occurs disproportionately in lower socio-economic groups. One of its common side effects is profound tiredness, and it is almost always associated with other co-morbidities (notably kidney disease, diabetes, depression or cognitive impairment). Heart failure nurses in our study made judgements about the stability or otherwise of the illness and about patients’ co-morbidities (including cognitive ability and mental health status), health and technological literacy, family support and technical set-up at home and motivation. As a result, the video consultation option was deemed inappropriate for many (at the time of writing, fewer than 20 such consultations had been undertaken across a clinic population of several hundred).
Complexity in the underlying condition was also associated with non-adoption, abandonment or limited usefulness of technologies in Cases B and C (in which dementia or multi-morbidity respectively made the patient unable or unwilling to use the supplied technology) and Case F (in which the predictive risk modelling tool selected multi-morbidity as a risk factor for hospital admission, but few such patients proved to be ‘textbook cases’ to manage).
Domain 2: the technology
The technologies used for video consultations, Skype™ and FaceTime™, are both mass-market software packages from large multi-national companies, presenting low risk of supplier withdrawal and relatively straightforward substitutability (hence, in these respects they could be classified as simple or complicated). However, there were elements of complexity. For example, they were run from NHS hardware (sometimes many years old) and from patient-held laptops, tablets or smartphones of varying quality and dependability. The software was sometimes logistically difficult to install on NHS computers (e.g. because of limited capacity of the IT support team and maintain ‘non-standard’ software environments), and even when installed, it was not 100% dependable for both technical (machine ‘crashing’) and human error (e.g. forgotten password) reasons. Workarounds tended to use low-tech, dependable solutions (e.g. community heart failure nurses defaulted to telephone consultations when video connection failed), thus reducing complexity.
Whilst video transmits speech and visual cues reasonably well (although variations in audio and video quality are quite common), clinical examination sometimes needs other modalities. In heart failure care, for example, the physical examination (e.g. blood pressure, heart rhythm, leg oedema) that the nurses considered essential was not easy in the remote environment—though it was sometimes possible with patient and carer assistance when the nurse knew the patient well. There was no easy or automated way of sharing and recording patient-held data such as home blood pressure readings (patients typically read out numbers but some misinterpreted the digital display or viewed it upside down).
The other case studies illustrated different aspects of complexity in the technology. GPS tracking technology (Case B), pendant alarms (Case C) and care organising software (Case E) all relied on bespoke solutions from small and medium-sized enterprise (SME) companies, and hence were vulnerable to withdrawal; these companies sometimes lacked capacity to meet the bureaucratic requirements for potential spread and scale-up. Case D used software that had been developed as part of an academic research study; importantly, the research nurse and technical team were co-located, allowing minor (but potentially critical) technical issues to be resolved in an ongoing way. Case F featured bespoke software supplied through a longstanding relationship with an SME that was subsequently acquired by a global company. These technologies all required considerable knowledge and skill to use to their full potential. An assumption underpinning the design of patient-held assistive technologies was that a group of relatives and/or friends would exist, live locally, be technology-savvy and be able and willing to collaborate around the care of the index case. In fact, such networks were rarely pre-existing; they often had to be built and nurtured. Cases B, C and D highlighted the role of both lay carers and professional staff in helping to set up and ‘service’ the technology and keep it in working order, a role that could be particularly onerous if the technology (as in case B) was not dependable.
The data warehouse for integrated case management in Case F was well embedded organisationally (in the sense that it was enshrined in national policy and local sub-contracts and data sharing agreements, and staff were employed to work on it). But it was not technically well embedded in the sense of seamless interoperability of data between participating organisations; significant workarounds were required to (for example) share care plans. The predictive risk modelling tool generated a different kind of risk estimate than a home visit from a clinician or social worker who knew the individual well and who had the capacity and authority to bear witness to a narrative and make contextual judgements. Our data illustrated that often neither approach alone offered the full picture that was sometimes necessary for making judgements. Thus, the output (risk score) generated by the technology was complex (in the sense that it was incomplete and contested).
Domain 3: the value proposition and overall value chain
The supply-side value proposition for video consulting in Case A currently appears complex. The multi-national companies behind Skype™ and FaceTime™ are also developing multiple other health products, especially directed towards the expanding ‘wellness and wearables’ market. From a purely financial perspective, such direct-to-consumer products may offer a more lucrative supply-side value proposition than investing in a major business venture to support video consultation products and services in the NHS, because working through technical and information governance challenges is a resource-intensive and time-consuming process with no guarantee of meeting shareholder or executive expectations at the end of it. Our interviews also suggest that companies are aware of the potential for reputational risk associated with seeking to profit from virtual consultations in the NHS.
The demand-side value (to patients) of video consultations is also complex, since the evidence base on which it rests is currently sparse. Whilst around 20 randomised controlled trials in a range of conditions have demonstrated equivalent efficacy and safety between video and face-to-face consultations [38], the samples for these trials are likely to have been carefully selected. Members of our VOCAL patient advisory group raised concerns about the risk of a ‘two-tier’ service in which demand-side value for a minority of patients will be gained at the expense of service cuts for the majority—a concern which, though speculative, has recently been echoed by professional bodies, clinicians and patients [47, 48].
In Case D, one aspect of the value proposition (which affects both supply- and demand-side value) is the potential of the data collected to inform the development of predictive algorithms based on biomarker changes over time and hence predict and pre-empt decompensation, hence averting hospital admission (rather than just prompting medication changes on the basis of, say, a rise or fall in blood pressure). This option creates new possibilities for ‘personalised’ medicine, but it also increases complexity. Whilst real-world value is hard to assess in the context of a randomised controlled trial, we note that the promise (or aspiration) of the telehealth package in Case D is highly complex, since it seeks to achieve multiple goals, including: (1) improving heart failure management in the community; (2) reducing demand on services (by allowing nurses to take on higher case loads); (3) preventing unplanned admissions; and (4) developing further predictive capabilities.
Case E illustrates two very different business models (and technology development models) for similar technologies and use cases. In one (Product A), the value proposition was highly speculative and little attempt was made to work with intended end-users to increase the technology’s fitness for purpose (and hence its desirability) during development. The implicit assumption was that the technology would be more or less plug-and-play; the business model rested on provider organisations paying for a block contract even though the intended benefits (and/or savings elsewhere in the system) were not clear. Product B included substantial up-front investment (from publicly funded R&D) to undertake co-design work; it was explicitly developed by a charity as a ‘public good’ in which costs to end-users would be minimal and viewed as a component of a wider (and ongoing) charity-supported package. These projects are both ongoing, but at the time of writing we would classify Product A’s value proposition as complex and Product B’s as complicated.
Domain 4: the adopter system
In the adoption of video consulting (Case A), there was considerable complexity in the adopter system. There was, for example, a striking difference between innovators (who embraced the new technology and way of working with enthusiasm) and other clinicians on the same teams who were reluctant to change, reflecting previous research showing that staff resistance is the single most important reason given for low uptake of remote health care [10, 49]. The hurdle was not merely learning to use a new technology but also accepting changes in identity (e.g. some staff did not view themselves as ‘techy’) and role (e.g. helping a remote patient troubleshoot technical problems with Skype™ or FaceTime™), dealing with perceptions of overload when running a new virtual service in parallel with a traditional face-to-face one and feeling under pressure to realise efficiency gains before the system had been fully redesigned to maximise such gains. New staff roles were not restricted to clinicians directly using the video technology; receptionists, clerks and technicians all had to accommodate new roles that the technology required in order to ‘work’. The same was true of the patient, who could sometimes but not always seek technical help from family members.
The other case studies illustrated additional complexities in the adopter system, such as some staff expressing ethical reservations about ‘tagging’ clients (Case B) or clinical concerns about the data generated by the technology (Case D—some clinicians were worried about possible legal liability if telehealth data, generated elsewhere and impossible to verify directly, were later found to be flawed). In Case C, clients sometimes rejected a pendant alarm because it symbolised dependence or because they were unwilling to pay a small monthly connection fee or be placed in a dependency relationship with a relative or neighbour.
Domain 5: the organisation(s)
The hospital trust that hosted the VOCAL study had strong leadership and good managerial relations; it met key criteria for technological innovativeness (e.g. it had previously won a national ‘Digital Trust of the Year’ award), and there was board-level enthusiasm for the introduction of video consultations. Despite these encouraging preconditions (‘simple’ in our taxonomy), other features of the organisation were highly complex. In particular, it had very limited spare staff time and resources (e.g. key posts were unfilled, and many clinic terminals were running outdated versions of software packages)—a problem known as ‘lack of organisational slack’ [18]. In addition, whilst many senior decision-makers assumed that the new service would save money by making services more efficient, the question of whether a video consultation would actually cost less to deliver was not easy to answer because of potential knock-ons in the system (e.g. the need for additional IT support and staff training; the fact that rooms still needed to be occupied, records retrieved and appointments booked even when the consultation was virtual; and the theoretical possibility of an increase in appointments as clinicians and patients found it easier to connect).
Another feature of complexity was that whilst video consultations between clinicians and selected patients usually worked well, the linked routines for booking appointments, managing the clinic list (e.g. registering when each patient had ‘arrived’ and ‘left’) and organising follow-up did not mesh well with a system that had evolved to process patients using their physical presence (waiting in line at a reception desk), manual transfer of paper records between different plastic ‘bins’ and sticky notes. Alignment with such routines was initially achieved using workarounds; by the end of the study, new (computer-based) routines had been developed by some but not all participating teams. Whilst Skype functionality increased access (by, for example, allowing patients to send messages to the clinician’s Skype account), and whilst this ease of access was sometimes clinically appropriate and encouraged (e.g. “drop me a message to confirm the change of insulin dose was OK”), it generated complexities elsewhere in the system, since the clinician then had to log the message on the medical record.
Similar complexity-related challenges were evident in the community heart failure study. The community trust was a digital innovator and had supplied all nurses with tablet computers to help them with various aspects of their work. Again, whilst video consultations worked well clinically for selected patients, at the time of writing they were not well integrated logistically with the administrative aspects of the service. There was no formal opposition from top management in the community trust to the nurse-led video consultation service. But neither was there strong enthusiasm, and there was limited spare capacity among front-line teams to undertake the work of making sense of the new approach, enrolling and training staff beyond initial enthusiasts, implementing new work routines and evaluating the service.
In our other case studies, enthusiasm at board level was sometimes absent (e.g. because business cases were weak) and/or key opponents were strategically placed and had high wrecking power (specific examples withheld). In Case D, wide variability in clinician engagement among the different study sites could be explained largely in terms of the extent to which the local team shared a vision for how remote biomarker monitoring for heart failure might enhance rather than threaten the existing service, an aspect of implementation work that May calls ‘coherence work’ [22].
In Case F, establishing integrated case management through shared data and predictive risk modelling technology was extremely complex because multiple organisations needed to be involved; the establishment and development of the programme unfolded over several years and relied on partnership working and contracting arrangements at different levels of multiple organisations. The technology was implemented, but the anticipated reduction in costs from reducing hospital admissions were not realised as real savings because of the complexities of reimbursement mechanisms and because case management was not always successful in avoiding admissions.
Domain 6: the wider system
The most significant system-level challenge to the scale-up and spread of video consultations in Case A was that there was no established national tariff for funding such consultations. In the VOCAL study, the local commissioning organisation proposed reimbursing such consultations at a rate (‘pass through tariff’ [50]) intermediate between a telephone consultation and a face-to-face one. But even though members of the relevant national policymaking team were on the study steering group and there was no opposition ’in principle’ to establishing a national tariff, this had still not been achieved at the time of writing—mainly because data to inform costing calculations were difficult to obtain and contested by some parties. This is a good example of how innovative, technology-supported service models can succeed as demonstration projects through local workarounds but will fail to spread or be sustained unless the regulatory and financial context is supportive [51]. Another contextual factor which added complexity in case study A was a mixed reaction from professional bodies, whose enthusiasm for new, potentially more efficient, models of care was tempered by concerns about workload and threats to equity.
Our national-level stakeholder interviews revealed another aspect of complexity relating to the nature and strength of evidence expected by different stakeholder groups. The technology industry typically moves quickly, with a development model based on rapid iterations of technologies and a pragmatic understanding of what works in practice (the ‘fail early, fail often’ principle). Policymakers, in contrast, tend to want what they call ‘gold standard’ evidence (for example, from randomised controlled trials) to ‘prove’ that a particular technology has the impacts claimed. This mismatch appeared to explain some of the slow progress on national-level policy in relation to video consultations.
The acute trust where our video consultation study was based was one of the first public sector providers in the UK to introduce this service model [39]. In the last year of the VOCAL study, more than 50 organisations contacted the lead clinician seeking advice or asking to visit to see the video consultation service in action. This is an example of the important role of inter-organisational networking in supporting the exchange of both explicit and tacit knowledge [18].
Domain 7: adaptation over time
The video consulting services in Case A illustrated both resistance to adaptation (through material limitations and institutionalised information governance regulations) and adaptiveness (through clinicians’ creative and responsive use of the technology). The NHS has a ‘locked-down’ computer environment in which any new hardware or software must be carefully considered and formally approved before being installed or upgraded (a characteristic that reduces complexity for IT managers but tends to increase complexity for front-line staff). Rapidly evolving software sits awkwardly in such an environment. In the VOCAL study, an automated upgrade to Skype™ made the system non-functional on clinical terminals until re-authorised by someone with administrator-level access rights—a problem that resulted in some remote clinics having to be done by telephone.
The material features of Skype™ enabled the development of ‘ad hoc’ consulting in the young adult diabetes clinic, for example, when the patient saw that the clinician was online and sent a text (SMS) message asking a question about a recently changed insulin dosage. The clinician could either reply by SMS message (within Skype™) or offer a real-time video consultation (typically very short). This adaptive use of video technology for patient-initiated consultations was viewed by clinicians as a game-changer for ‘challenging’ patients (characterised by high non-attendance rate at clinic, poor glycaemic control and a history of hospital admission for diabetic emergencies).
Several other cases in our dataset illustrated a similar tension between system rigidity (for contractual or cost reasons) and adaptation (through user creativity). In Case C, for example, potentially remediable problems occurred with some alarms, but adaptation was impossible because of the risk of loss of warranty. A pendant alarm service initially introduced to provide emergency physical support (e.g. for falls) was adapted over time to provide non-emergency emotional support for certain older people, who were encouraged by call centre staff to press the alarm button to trigger a supportive conversation when feeling lonely.
In Case F, following the introduction of the integrated case management data warehouse technology, clinical and administrative staff across different organisations collectively learnt and redefined what this technology could and could not do. They amended, adapted and worked around it. For example, clinicians and practitioners reviewed the outputs of the data-driven risk stratification model but also supplemented them with other data and used their judgement to target additional patients who had not been flagged as ‘high risk’ by the technological algorithm. Technical developers continually updated the user interface through an ongoing relationship with the procuring body. Notwithstanding these efforts, there was a brittleness about the technology (and the work routines it presupposed) that staff experienced as persistently frustrating. Thus, there was a sense that the technology had been ’successfully’ implemented and was for the moment being sustained (because it met policy expectations), but was not truly fit for purpose.
