Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

The evolution of mobile apps for asthma: an updated systematic assessment of content and tools

BMC Medicine201513:58

https://doi.org/10.1186/s12916-015-0303-x

Received: 4 December 2014

Accepted: 26 February 2015

Published: 23 March 2015

Abstract

Background

Interest in mobile apps that support long-term conditions such as asthma is matched by recognition of the importance of the quality and safety of apps intended for patient use. We assessed how changes over a 2-year period affected the clinical suitability of apps providing self-management information and tools for people with asthma by updating a review first performed in 2011.

Methods

Systematic content assessment of all apps for iOS and Android examining the comprehensiveness of asthma information, consistency with the evidence base for asthma self-management and adherence to best practice principles for trustworthy content, comparing the quality of apps available in 2011 to those released since.

Results

Between 2011 and 2013, numbers of asthma apps more than doubled from 93 to 191, despite withdrawal of 25% (n = 23/93) of existing apps. Newer apps were no more likely than those available in 2011 to include comprehensive information, such as the use of action plans, or offer guidance consistent with evidence; 13% (n = 19/147) of all apps, and 39% (n = 9/23) of those intended to manage acute asthma, recommended self-care procedures unsupported by evidence. Despite increases in the numbers of apps targeting specific skills, such as acute asthma management (n = 12 to 23) and inhaler technique (from n = 2 to 12), the proportion consistent with guidelines (17%, n = 4/23) and inhaler instructions (25%, n = 3/12), respectively, was low, and most apps provided only either basic information about asthma (50%, n = 75/147) or simple diary functions (24%, n = 36/147).

Conclusions

In addition to persisting questions about clinical quality and safety, dynamic aspects of app turnover and feature evolution affect the suitability of asthma apps for use in routine care. The findings underline the need for coordinated quality assurance processes that can adapt to changing clinical and information governance-related risks, ensure compliance with the evidence base and reflect local variations in clinical practice. It is unclear if substantial clinical benefits can be realized from a landscape dominated by low quality generic information apps and tools that do not adhere to accepted medical practice.

Keywords

Android App Asthma Cross-sectional study iPhone mHealth Mobile health Smartphone Systematic assessment Update

Background

Medical apps for patients are software programs that run on personal mobile devices and offer functions such as disease tracking, access to clinical and peer support, health information, and reminders [1]. Survey data indicate that approximately three fifths of US and UK adults own an app-capable smartphone [2,3] and around half of adults report knowing about and using apps [2]. In 2012, a fifth reported having downloaded a health or fitness app, increasing amongst people with recent changes in health status [4]. Industry estimates suggest that, by 2018, over one billion individuals worldwide will use a health or fitness app [5] drawn from a pool of consumer health apps estimated to number in excess of 15,000 [6]. In 2014, over a third of US clinicians reported having recommended an app to a patient in the past year [7], and a survey of US adults suggested that nine of ten would use one issued ‘on prescription’ by a healthcare professional [8]. Among policymakers and clinicians, interest in apps reflects the potential to improve patient support and, potentially, health outcomes by equipping individuals with tools for self-management and remote monitoring [9-11]. However, there is also increasing recognition of the importance of ensuring that medical apps are effective [12,13], safe [14,15], meet patient needs [16,17], and comply with existing evidence-based clinical practice [18-20].

In 2011, we mapped [21] the types of apps available for patients with asthma, chosen as a prevalent long-term condition with potential social and economic [22,23] impacts and which requires a range of self-management skills in everyday life [24]. We explored the degree to which apps conformed to evidence-based recommendations derived from international asthma guidelines. We characterized the range of possible functions supported by apps for asthma, including information provision, peak flow and symptom diaries, and medication trackers. While highlighting the potential to support tailored education and assist clinical decision-making using monitoring data, the review uncovered a range of potential issues. Few information apps addressed key educational topics recommended in international guidance. Many advocated strategies for acute asthma management and trigger avoidance that were unsupported by current evidence. Assessment and diagnostic tools were largely unvalidated, some had errors of design, and the small number of apps offering therapeutic interventions had no basis in evidence.

Rapid evolution is characteristic of mobile technologies. New hardware and software capabilities create opportunities for health; for example, remote monitoring enabled by always-on network connections and wearable sensors [25]. Modern development methods mean that apps commonly undergo cycles of refinement in response to testing and user feedback [26]. Changes in mobile devices and software have been accompanied by a broadening discussion about quality and safety that has involved clinicians [27,28], policy groups [29], and, more recently, regulators [30,31]. Compared to 2011, there is greater understanding of potential risks and more resources targeted towards medical app developers, aiming to improve the quality of medical apps [32-35]. As a result, there is an opportunity to explore the impact that these changes may have had on apps intended for patient use.

We aimed to explore changes in the content, function, and clinical quality of apps since 2011. We used a systematic approach, based on a systematic literature review, to identify, classify, and review apps for asthma. By updating our earlier review, we sought to differentiate persistent quality issues from those that are new or have proven transient.

Methods

App selection

All apps from the two most widely-used [36] mobile platforms (Android and iOS) incorporated in the original review [21] were automatically eligible for inclusion. We chose not to include apps running on other platforms, such as Blackberry, whose market share is currently declining [36]. To identify new apps, we performed a stepwise process of search, screening, and review. We searched the public app stores of the two most popular smartphone operating systems using the terms ‘asthma’, ‘inhaler’, ‘peak flow’, and related word variants, e.g., ‘asthmatic’. After removing apps already included from the previous study, results were screened to eliminate obviously irrelevant content. The remaining apps were downloaded and reviewed to reach a final decision for inclusion. Screening and review were performed by three reviewers (KH, JO, and AG), working independently. Final decisions were reached by consensus.

Free and paid-for smartphone and tablet apps were eligible for inclusion if they contained asthma-specific content or tools intended for use by patients or the public (see list below). Apps targeted exclusively towards clinicians were not included. Apps were excluded if they could not be downloaded from UK app stores or, after at least two attempts on different test devices, would not start. Apps from any country were eligible provided they supported English.

Inclusion criteria

  • Smartphone or tablet app

  • Free or paid

  • Running on Android or iOS platforms

  • Content or tools addressing one or more aspects of asthma diagnosis, management, or support as either the sole function or in a way that means asthma-related elements can be isolated from the rest of the content

  • Presenting content in any format

  • English language

  • Targeted at patients of any age

Exclusion criteria

  • Not available through an approved device marketplace

  • Explicitly disclaimed use for a health-related purpose

  • Could not be downloaded because of country restrictions that prevented access in the UK

  • Could not be used because of technical problems, after two attempts

App assessment

Apps were assessed using a systematic approach to characterize functions and content. Apps containing information about asthma were evaluated using a set of criteria derived from US [37], UK [24], and international [38] asthma guidelines. Information content was evaluated for coverage of eight domains identified as priorities for self-management education (basic facts about asthma, basic principles of treatment, trigger avoidance, how to use treatment, self-monitoring skills, the role of action plans, managing exacerbations, and personalized care), using a set of operational criteria derived from guidelines. These criteria were those used in our earlier review except for one criterion in the self-monitoring domain which was modified in response to peer feedback to acknowledge symptom-based methods for self-monitoring (criteria and change detailed in Additional file 1: Table AF1). Coverage of each educational domain was rated as either complete (all criteria satisfied), partial (some criteria satisfied), or absent. Content was also evaluated for consistency with evidence-based recommendations concerning eight aspects of asthma management derived from guidelines (Additional file 1: Table AF2). For those apps with content addressing a particular topic, the advice given was rated as being either consistent or inconsistent with underlying evidence. Consistency was judged both on the basis of factual accuracy and appropriate qualification of advice to reflect either uncertainty or limitations in the applicability of evidence, such as recommendations relevant only to individuals with more severe asthma, in which case we expected any information to be appropriately qualified. Information content was evaluated regardless of the delivery medium. However, apps focusing only on complementary and alternative medicine, using the definition of the NIH National Center for Complementary and Alternative Medicine [39], were excluded from this analysis because there was no basis to expect that they should comply with guideline recommendations. For diaries and other self-management tools, the nature of inputs and outputs and methods for summarizing and sharing data were characterized. For all apps with calculator and questionnaire functions, we attempted to locate validation information in the form of an authoritative citation. Where validation information was found, the behaviour of the app in handling inputs and producing appropriate outputs was assessed. All apps were assessed for compliance with principles for ethical disclosure of authorship and data protection practices, using criteria previously adapted from those assessing online health information devised by the Health on the Net Foundation [40] (criteria in Additional file 1: Table AF3). We also recorded any software issues that were encountered during testing, using a schema devised during the first review to classify problems such as crashes or errors in the user interface (detailed in Additional file 1: Table AF4).

Apps included in the original review had previously been assessed using these criteria. To ensure scoring consistency, we conducted a process of calibration in which a random selection of 10 apps was re-assessed and the outcomes compared to previous judgments. Apps that had been updated between 2011 and 2013 were also reassessed. Apps were reviewed by an author of the original review (KH) and one of two additional reviewers (JO and AG), working independently. After reconciliation, the results of the assessment process were recorded, along with basic characteristics of included apps, in a structured data extraction form (template in Additional file 1: Table AF5) for analysis.

Post hoc evaluation

In addition to the planned analyses described above, we performed a post hoc evaluation of inhaler technique guidance in response to the growing number of apps that include this function, using criteria derived from manufacturer guidance to assess key steps, such as inhaler preparation, necessary for effective drug delivery into the lungs (detailed in Additional file 1: Table AF6).

Data analysis

Apps available on multiple platforms were grouped for subsequent analysis. When an app was available as multiple versions on the same platform, for example ‘demo’ and ‘full’ editions, the most fully featured version was used for analysis (irrespective of free/paid status). The decision to define the unit of analysis as the platform-independent app reflected learning from the original review concerning the possible risk of bias resulting from double counting some apps. In that review, we treated each app version separately because we anticipated different quality issues arising from the need to tailor each app according to the platform and version. However, the final results suggested that this was not, in general, the case. While in 2011 only a small number of apps were affected (n = 13), and sensitivity analysis undertaken during that analysis demonstrated minimal impact on results, by 2013 the number of duplicates was substantially larger (n = 77) with consequently greater scope for bias.

Descriptive statistics were used to summarize the types, functions, and quality characteristics of apps available in 2011, those released since 2011, and overall. When reporting comparisons between 2011 and 2013, data are always written in chronological order, e.g., for “(X vs. Y)”, X are data from 2011 and Y reflect data for apps released since 2011. We hypothesized that newer apps might have different quality characteristics arising from greater experience amongst developers and discourse in both medical and computer science literature emphasizing the importance of high-quality medical apps. To explore differences in the quality of information apps, we compared the proportion of educational domains completely, partially, or not satisfied, and the proportion of guidance consistent with the evidence base in those available in 2011 and those released since. We also compared the proportion of apps satisfying principles for ethical disclosures of information and the proportion of apps with specific software issues. Apart from these planned comparisons, we performed a small number of additional tests to compare differences in the proportion of paid-for apps, apps addressing complementary and alternative medicine, and the incorporation of features such as support details, in-app help, and social sharing features. These are marked as post hoc comparisons in the results.

For all comparisons, we used a two-tailed Fisher’s exact test [41], with the Freeman-Halton extension [42] to handle two by three cases for information comprehensiveness and (post hoc) content orientation. To adjust for multiple comparisons, we used a Holm-Bonferroni [43] sequential adjustment to control the group-wise error rate at 5%, instead of specifying a fixed significance level for comparisons. All analyses were performed using R (Version 3.1.2).

Results

Searches performed in June, July, and August 2013 identified 764 apps (Figure 1, n = 538 Android, 226 Apple). After excluding unsuitable apps (n = 643), 191 apps were selected for evaluation. These included 70 apps identified in the 2011 review that remained available. Details of apps excluded at the final review stage are provided in Additional file 2: Table AF7.
Figure 1

Flowchart of app selection process. Flowchart details the process of selecting individual apps for inclusion in the study. After evaluation of 191 apps, duplicates available on both iOS and Android (n = 24) and, when more fully featured versions were available, ‘lite’ or demonstration versions (n = 20) were excluded, leaving 147 unique apps for subsequent analysis.

The number of apps for asthma more than doubled between 2011 and 2013 (from 93 to 191), dominated by growth in apps available for Android (from 41 to 103, 151% increase, vs. iOS 52 to 88, 69% increase, charted over time in Figure 2). Forty-one apps were available on both Android and iOS platforms and a further 36 had multiple versions on a single platform. After removing 24 platform duplicates and 20 demonstration versions where a more fully-featured version was available, analysis was performed on 147 unique apps. Over a quarter (28%, n = 22/78 unique apps) included in the original review had been withdrawn, and a similar proportion (24%, n = 19/78) updated, between 2011 and 2013. Withdrawals are included in reported 2011 statistics but not in overall 2013 totals. At the conclusion of analysis in June 2014, a further 20 apps had been withdrawn. These are retained for analysis, but annotated as withdrawn in data tables provided in Additional file 2.
Figure 2

Growth in numbers of asthma apps. Plot shows the total number of available apps for asthma against time, based on the original release date, for the period 2009 to mid-2013. Withdrawals are accounted for. Vertical lines indicate the dates of the 2011 and 2013 surveys. Data were not available for 5 apps; therefore, final n = 186.

Table 1 summarizes the basic characteristics of included apps. Compared to apps available in 2011, apps released since were significantly more likely to be free (33%, n = 26/78 vs. 64%, n = 59/92, P <0.001, post hoc) but also more likely to carry advertising (18%, n = 14/78 vs. 34%, n = 31/92, P = 0.024, post hoc), although the difference was not significant after sequential adjustment. Between 2011 and 2013 the median cost of paid-for apps decreased from £1.27 GBP to £0.89 (£1.12 overall), reflecting a reduction in the proportion of apps charging more than £1.00 (37%, n = 29/78 vs. 22%, n = 20/92). The proportion of apps offering in-app upgrades to unlock addition functions remained unchanged at 9% (n = 13/147) overall, (8%, n = 7/78 vs. 7%, n = 6/92, P = 0.576, post hoc). While the US remained the main source of apps for asthma, accounting for almost half of apps with a known country of origin available in 2013 (48%, n = 52/108), the number of countries represented increased from 13 to 21. Although the overall proportion of apps indicating that the product was targeted at a particular group remained low (16%, n = 24/147), apps released since 2011 were significantly more likely to provide information targeting parents/caregivers (4%, n = 3/78 vs. 16%, n = 15/92, P = 0.011, post hoc). There was also a non-significant increase in the proportion of apps targeting children (3%, n = 2/78 vs. 8%, n = 7/92, P = 0.182, post hoc).
Table 1

Basic characteristics of included apps

 

2011 (n = 78)

New 2013 (n = 92) ¤

All 2013 (n = 147) ¤

Characteristic

Yes

%

Yes

%

Yes

%

Platform

      

Android only

32

41%

52

57%

73

50%

iOS only

40

51%

28

30%

57

39%

Both

6

8%

12

13%

17

12%

Cost

      

Free

26

33%

59

64%

80

54%

Paid-for

52

67%

33

36%

67

46%

Median

£1.27

£0.89

£1.12

Country of origin

      

US

24

31%

31

34%

52

35%

UK

5

6%

6

7%

8

5%

India

7

9%

7

8%

8

5%

Australia

3

4%

4

4%

7

5%

Other

10

13%

24

26%

33

22%

Unknown

29

37%

20

22%

39

27%

Content orientation

      

Conventional medicine only

56

72%

59

64%

101

69%

Complementary/alternative medicine only

15

19%

17

18%

26

18%

Mixed

7

9%

16

17%

20

14%

Audience targeting

      

Children or young adults

2

3%

7

8%

8

5%

Parents or caregivers

3

4%

15

16%

19

13%

No audience stated

73

94%

73

79%

123

84%

Other features

      

Adverts

14

18%

31

34%

39

27%

Social sharing

13

17%

35

38%

61

41%

Cloud storage

3

4%

10

11%

16

11%

Clinical disclaimer

32

41%

37

40%

55

37%

In-app help

14

18%

28

30%

43

29%

Online help

5

6%

6

7%

10

7%

¤ New 2013, Apps available in 2013 that were not available in 2011; All 2013, All apps available in 2013, excluding those available in 2011 that were subsequently discontinued.

Four countries with highest overall number of apps in 2013 shown. Other countries were: Ireland (n = 5), Germany (n = 4), Switzerland (n = 4), Japan (n = 3), Romania (n = 2), Singapore (n = 2), Austria (n = 1), France (n = 1), Italy (n = 1), Norway (n = 1), Poland (n = 1), Portugal (n = 1), Spain (n = 1), and Sweden (n = 1).

Audience targeting was identified by specific claims made in the app title or description or content. We found no cases where apps explicitly indicated they were intended for adult use only. A small number of apps targeted both children and a parent or caregiver (2011, n = 0; New 2013, n = 3; All 2013, n = 3).

Information provision remained the function most commonly offered by nearly three fifths of apps for asthma (56%, n = 83/147; Table 2), followed by self-management tools in almost half (48%, n = 70/147). The number of apps combining self-management tools, such as diaries, with any form of information content grew from one (1%) in 2011 to nine (6%) in 2013. As in 2011, small numbers of apps available in 2013 provided functions for assessing asthma symptoms using questionnaires (n = 11) or sensors (n = 2) or assisting with aspects of therapy (n = 6). Although there was an increase in the proportion of apps released since 2011 incorporating mixed content addressing both conventional and alternative medical traditions, the change in relative frequencies was not significant (P = 0.268, post hoc; Table 2) and the overall proportion of apps presenting content oriented towards conventional medical practice changed minimally (69% n = 101/147 vs. 73% n = 57/78 in 2011).
Table 2

Breakdown of app functions

 

By date

By platform

 

2011 (n = 78)

New 2013 (n = 92) ¤

All 2013 (n = 147) ¤

All 2013 (n = 191)

Function

n

%

n

%

n

%

Android

iOS

Information

37

47%

60

65%

83

56%

66

44

General information

36

46%

51

55%

74

50%

56

39

Acute asthma management (First aid)

12

15%

18

20%

23

16%

19

14

Inhaler technique guidance

2

3%

10

11%

12

8%

6

13

Other therapeutic instructions

13

17%

10

11%

16

11%

15

9

Self-management tools

36

46%

38

41%

70

48%

42

49

Diaries and trackers

21

27%

18

20%

36

24%

23

26

Pollen status

3

4%

8

9%

12

8%

6

10

Pollution status

3

4%

6

7%

9

6%

2

9

Allergen database

4

5%

3

3%

7

5%

5

4

Forum

0

0%

1

1%

1

1%

1

0

Online pharmacy

0

0%

1

1%

1

1%

1

0

Combined with information

1

1%

8

9%

9

6%

6

7

Assessment tools

9

12%

8

9%

16

11%

10

11

Physiological measurement

2

3%

1

1%

3

2%

0

4

Assessment questionnaires

4

5%

7

8%

11

7%

7

9

Standalone calculators

2

3%

2

2%

4

3%

3

1

Therapeutic tools

2

3%

5

5%

6

4%

1

5

The table compares the types of functions present in apps for asthma available in 2011 and 2013, and provides a breakdown by platform for 2013; 25 apps included in the 2011 assessment were withdrawn and this accounts for the lower overall total in 2013. Some apps contained multiple functions and are counted more than once.

Apps available on both Android and iOS are counted only once.

¤ New 2013, Apps available in 2013 that were not available in 2011; All 2013, All apps available in 2013, excluding those available in 2011 that were subsequently discontinued.

Information apps

The number of apps offering information about asthma increased from 37 in 2011 to 83 by 2013, with 60 new apps and 14 withdrawals. Most information apps available in 2013 incorporated general information about asthma (89%, n = 74/83), although a growing proportion of apps (23%, n = 18/78 vs. 34%, n = 31/92, P = 0.174, post hoc) also incorporated specific instructions about the management of acute asthma, inhaler technique, and other aspects of self-management (discussed further in Therapeutic Instructions and Tools, below.) Apps used a range of media to communicate information including text (64%, n = 53/83), video and animation (10%, n = 8/83), or by combining different media types (27%, n = 22/83). A significantly greater proportion of apps released since 2011 used a medium other than text to communicate information (11%, n = 4/37 vs. 42%, n = 25/60, P = 0.001, post hoc). A new development was the use of gameplay (n = 3) to provide interactive education in three products targeted at children and teenagers. Full details of information-containing apps are provided in Additional file 2: Table AF8.

Although increasing in number, the proportion of products presenting information based exclusively around conventional medical practices was unchanged (51%, n = 19/37 vs. 50%, n = 30/60). Just over a fifth of all apps available in 2013 (23%, n = 19/83) combined alternative and conventional information and the remainder (28%, n = 23/83) presented information based on alternative medical principles alone. Consequently, there were 59 apps (vs. 23 in 2011) incorporating conventional medical information for which it was possible to assess coverage of self-management education topics and consistency with guideline recommendations.

Over two thirds (83%, n = 49/59) of all apps available in 2013 provided at least some details about basic aspects of asthma pathophysiology (Table 3). However, only just over half provided information about basic principles of medical management (53%, n = 31/59), and fewer addressed key self-management skills, including allergen and trigger avoidance (44%, n = 26/59), self-monitoring skills (39%, n = 23/59), or how to use treatment appropriately (39%, n = 23/59). Less than a third of apps addressed how to recognize the signs of deterioration (31%, n = 18/59), the importance of personalized treatment and goal setting (31%, n = 18/59), and the role of an action plan (22%, n = 13/59). There was no significant difference in the proportion of apps released since 2011 addressing domains other than basic facts (27%, n = 6/22 vs. 20%, n = 9/45, P = 0.542), nor in the proportion of apps addressing each domain considered individually (P values shown in Table 3). While the overall number of products addressing three or more educational domains, at least partially, grew from 14 to 31 between 2011 and 2013, new apps were not more likely to address multiple domains (64%, n = 14/22 vs. 51% n = 23/45, P = 0.435). In the same period, the number of products addressing all eight domains at least partially grew from two to five (with one withdrawal). As in 2011, however, no product provided complete coverage of all eight domains by satisfying all criteria.
Table 3

Coverage of asthma self-management education topics by information apps

 

2011 (n = 23)

New 2013 (n = 44) ¤

 

All 2013 (n = 59) ¤

Educational topic

No

Partially

Wholly

%

No

Partially

Wholly

%

P value

No

Partially

Wholly

%

Basic facts about the nature of the condition

2

15

6

91%

9

26

9

80%

0.510

10

37

12

83%

Allergen and trigger avoidance

9

12

2

61%

25

19

0

43%

0.094

33

24

2

44%

The nature of treatment: relievers and preventers

11

7

5

52%

22

18

4

50%

0.333

28

25

6

53%

Recognizing and responding appropriately to acute exacerbations

14

7

2

39%

30

9

5

32%

0.728

41

12

6

31%

How to use treatment

16

6

1

30%

27

17

0

39%

0.259

36

22

1

39%

Self-monitoring and assessment skills

16

6

1

30%

27

16

1

39%

0.632

36

20

3

39%

The role of a written, personalized action plan

18

2

3

22%

34

5

5

23%

1.000

46

6

7

22%

Personalizing the definition of good asthma control

17

5

1

26%

29

15

0

34%

0.244

41

18

0

31%

The table compares the proportion of apps containing asthma self-management education information that address guideline-recommended topics either partially or wholly between 2011 and 2013. Apps focusing solely on complementary and alternative medicine were excluded.

The proportion of information apps that wholly or partially address a given educational topic.

Two-tailed probability obtained from the Freeman-Halton extension to Fisher’s exact test of the comparison of topic coverage in apps available in 2011 and those released subsequently. After applying a Holm-Bonferroni correction to control the family-wise error rate at less than or equal to 5%, no comparisons achieved significance.

¤ New 2013, Apps available in 2013 that were not available in 2011; All 2013, All apps available in 2013, excluding those available in 2011 that were subsequently discontinued.

Thirty-six apps (vs. 14 in 2011) were evaluated for consistency of recommendations against nine evidence-based topics (Table 4) because they contained content addressing at least one topic. Smoking cessation and passive smoking avoidance were the topics most commonly addressed by apps (by n = 19 and 21, respectively) and, as in 2011, were correctly recommended as beneficial by all apps. All apps addressing immunotherapy (n = 5) and weight reduction (n = 7) correctly advocated their use in specific patient populations. However, apps were also likely to recommend actions for which there is unclear evidence of benefit, such as mold (n = 12/12) and cockroach (n = 7/7) avoidance measures, removal of pets (n = 7/8), and avoidance of air pollution (n = 15/17). Most apps covering seasonal influenza vaccination recommended it (n = 8/10) without acknowledging the uncertain impact on asthma [44]. Compared to 2011, there were no significant differences in the proportion of apps released since that presented advice consistent with the evidence base, either when considering each statement individually (exact P values computed from proportions shown in Table 4 were all 1.0) or when comparing the proportion of apps containing no statements contradicting evidence (36%, n = 5/14 vs. 43%, n = 12/28, P = 0.747). A new finding in 2012 concerned a small number apps (n = 6) containing advice emphasizing the possible harmful effects of conventional medicine and, in two cases, promoting a commercial product, for example a homeopathic remedy. Previously, apps containing mixed content had tended to emphasize the right to choose between medical approaches rather than denying accepted medical practice.
Table 4

Consistency of information apps with evidence-based recommendations

 

2011

New 2013 ¤

All 2013 ¤

Statement

n

Consistent

%

n

Consistent

%

n

Consistent

%

Removal of pets from the home (Qualified benefit)

3

1

33%

7

1

14%

8

1

13%

Fungal allergen avoidance and control measures (Qualified benefit)

8

0

0%

9

0

0%

13

0

0%

Cockroach avoidance and control measures (Qualified benefit)

6

0

0%

4

0

0%

7

0

0%

Cessation of active smoking (Beneficial)

7

7

100%

14

14

100%

19

19

100%

Avoidance of passive smoking (Beneficial)

8

8

100%

15

15

100%

21

21

100%

Avoidance of exposure to air pollution (Qualified benefit)

8

1

13%

14

1

7%

17

2

12%

Immunotherapy in atopic asthma (Beneficial)

1

1

100%

4

4

100%

5

5

100%

Weight reduction in obese patients (Beneficial)

3

3

100%

4

4

100%

7

7

100%

Seasonal influenza vaccination (Qualified benefit)**

4

1

25%

8

2

25%

10

2

20%

The table compares the proportion of apps containing particular management advice that is consistent with evidence-based recommendations between 2011 and 2013. The expected advice is shown in parentheses. We use the term ‘qualified benefit’ when additional factors, such as personal sensitivity to particular allergens or aspects of personal choice, are relevant concerns and unconditional recommendations are therefore inappropriate. Apps focusing solely on complementary and alternative medicine were excluded. Statistical comparison between apps in 2011 and new apps in 2013 was performed but is not shown because all comparisons, performed using Fisher’s exact test, yielded an exact probability of 1.0.

The number of apps that took a stance on a given topic, for example, by stating that a given action is definitely effective in controlling asthma symptoms.

The proportion of apps whose stance is consistent with the evidence base.

¤ New 2013, Apps available in 2013 that were not available in 2011; All 2013, All apps available in 2013, excluding those available in 2011 that were subsequently discontinued.

**Seasonal influenza vaccination is routinely offered to asthmatic patients; however, it is unclear if vaccination reduces the severity or frequency of flu-related asthma exacerbations [44].

Diaries and trackers

Thirty six apps available in 2013 (24%, vs. 27% in 2011) were diaries for recording asthma symptoms (n = 27), peak flow measurements (n = 24), and medication (n = 24, detailed in Additional file 2: Table AF9). Six apps allowed a user to document an action plan. Six incorporated inhaler trackers, including two where this was the sole function, designed to alert a user when a metered dose inhaler or other medication was running low. One app connected to an electronic inhaler to automatically log doses. Two thirds of diaries (67%, n = 24/36) offered a mechanism to summarize either recorded symptoms (n = 4), peak flow (n = 11), or both (n = 9) in chart form. Over half (55% n = 11/20) of apps charting peak flow used predicted (n = 6), personal best (n = 7), or manually entered (n = 3) thresholds to classify charted data to aid interpretation, for example, as ‘green’, ‘yellow’, and ‘red’ zones. Nine apps provided alerts based on changing input values. Seven provided specific advice, either by excerpting relevant parts of a user supplied-action plan (n = 3/6 apps with action plan support) or displaying standardized text (n = 4). Sixteen allowed reminders to be set to take peak flow measurements (n = 9), medication (n = 3), appointments (n = 2), and vaccination (n = 1). Compared to 2011, there were increases in the proportion of apps synchronizing and storing data online (10%, n = 2/21 vs. 44%, n = 8/18, P = 0.025, post hoc) but not those offering a mechanism for sharing data with clinicians (62%, n = 13/21 vs. 61%, n = 11/18, P = 1.000, post hoc), for example, email or file export.

Other self-management tools

Nineteen apps (compared to 6 in 2011, Table 2) provided status information about pollen (n = 10), air pollution (n = 7), or both (n = 2; Additional file 2: Table AF10). Most (n = 12) provided coverage of locations in the US. Only one product, which relied on used-supplied data, offered global coverage. Seven apps (vs. 4 in 2011) offered databases of chemical additives to allow patients to identify potential triggers in food and cosmetic ingredients. One app combined an electronic version of an action plan with links to information about asthma. Another product connected to a forum where users could view topics about asthma management and post their own questions. A third app offered UK patients the ability to purchase asthma medication online.

Diagnostic and assessment tools

Fifteen apps (vs. 6 in 2011) contained diagnostic or assessment functions. Ten products offered questionnaires for asthma diagnosis (n = 4) or assessment (n = 6; Additional file 2: Table AF11). Two apps used the validated Asthma Control Test under license and one app reused a questionnaire developed by the University of Maryland. The remaining apps either did not provide a citation or validation information or the information provided was insufficient to verify the basis for assessment. Nine products provided predicted peak flow calculators as either the main function (n = 3) or integrated into a diary (n = 6). No calculator app provided a clear citation, but the underlying formula was identified for two apps. Issues first identified in 2011 resulting in incorrect calculation output for two apps where the formula was deduced were unresolved. Three products used the device microphone to attempt to quantify lung volume (n = 1) and analyse breath sounds (n = 2), but were either unvalidated or experimental products that had not yet received regulatory approval.

Therapeutic instructions and tools

Forty-one apps (28% of n = 147) contained specific instructions for the use of inhalers (8%, n = 12/147), first aid for acute exacerbations (16%, n = 23/147), or other approaches to the management of asthma such as the use of herbal remedies (11%, n = 16/147, breakdown in Additional file 2: Table AF12). After pressurized metered dose inhalers (pMDI), either alone (100%, n = 12/12) or with spacer devices (92%, n = 11/12), the devices most commonly addressed by inhaler technique guidance were Accuhalers (58%, n = 7/12) and Turbohalers (50%, n = 6/12), although a range of additional devices were also covered (Additional file 2: Table AF12). Apps used text (33%, n = 4/12), video (25%, n = 3/12), animation (8%, n = 1/12), or a combination (33%, n = 4/12) to provide guidance. Post hoc evaluation (criteria in Additional file 2: Table AF6) examined the proportion of apps providing complete and correct information about inhaler preparation (67%, n = 8/12), positioning in the mouth (67%, n = 8/12), and the sequence of inhalation and actuation (50%, n = 6/12). While a quarter (25%, n = 3/12) addressed all three domains, the majority of apps omitted or misstated at least one step necessary for effective delivery of drugs to the lungs [45,46] for at least one of the inhalers discussed. The most common error (in 25%, n = 4/12) was the failure to explain how inspiration should start before, and continue after, actuation of a pMDI. A variable proportion of apps addressed other aspects of inhaler use such as holding the breath after inhalation (83%, n = 10/12), timing of a second dose (58%, n = 7/12), and mouth rinsing to reduce the risk of candidiasis with inhaled steroids (50%, n = 6/12). While five apps (42%) provided information about identifying empty Turbohalers or Accuhalers, no app addressed dose-tracking using pMDI. The two apps (17%) that covered inhaler care both provided incorrect information about spacer cleaning that risked static build-up interfering with subsequent doses.

Of the 23 apps incorporating advice about the management of acute asthma, approximately half (52%, n = 12/23) provided a clear description of the signs of acute asthma and gave practical advice such as sitting a patient up (48%, n = 11/23) and offering reassurance (52%, n = 12/23). Three fifths (61%, n = 14/23) provided appropriate instructions about when to seek further help. While almost two thirds (65%, n = 15/23) recommended the use of a rescue inhaler, less than a third (30%, n = 7/23) provided specific dose and timing instructions and only four apps (17%) provided instructions appropriate for children in addition to adults. Overall, four apps (17%) addressed all five components (Additional file 2: Table AF12). The proportion of apps released since 2011 that made treatment recommendations based on alternative medical practices did not change significantly (76%, n = 13/17 vs. 56%, n = 14/25, P = 0.207, post hoc), and almost three fifths of apps available in 2013 (58%, n = 19/33), and 39% (n = 9/23) of those intended to manage acute asthma, recommended procedures unsupported by evidence, for example, the use of a hot water bottle to warm the chest, cider vinegar administered orally, or fasting for the duration of an attack.

A small number of products (n = 6 vs. 2 in 2011) were tools intended to have a therapeutic effect. Three apps provided training in aspects of Buteyko breathing, which may have a future role in asthma management [47]. The remaining three were based on alternative medical principles using hypnosis, mantra, and motivational messages.

Disclosures and software issues

Results of assessment for compliance with ethical disclosures about authorship and data protection are summarized in Table 5 (based on criteria detailed in Additional file 1: Table AF4). Perhaps reflecting the increase in apps syncing data to a cloud service (4%, n = 3/78 vs. 11%, n = 10/92, P = 0.145, post hoc) or offer social sharing (17%, n = 13/78 vs. 38%, n = 35/92, P = 0.002, post hoc), apps released since 2011 were significantly more likely to have a privacy policy (9%, n = 7/78 vs. 36%, n = 33/92, P <0.001). New apps were also significantly more likely to provide contact details (41%, n = 32/78 vs. 64%, n = 59/92, P = 0.003). While no apps in 2011 had an editorial or advertising policy, five new apps did. In general, apps provided limited information about the information sources used for the advice they contained. The proportion of apps providing any kind of content attribution was unchanged (36%, n = 21/78 vs. 33%, n = 26/92, P = 0.856). Almost three fifths of information-containing apps (57%, n = 47/83) available in 2013 repackaged content available freely online from resources such as Wikipedia, a small increase from 2011 (54%, n = 19/35). The majority (77%, n = 36/47) of these apps did so without referencing the original content sources. Overall, 35% (n = 29/83) were ‘junk’ apps that reused, without attribution, information of generally low quality that was freely available online in a paid-for or advert-supported format.
Table 5

Summary of ethical disclosures and software issues

 

2011 (n = 78)

New 2013 (n = 92) ¤

 

All 2013 (n = 147) ¤

Quality domain

Yes

%

Yes

%

P value

Yes

%

Ethical disclosures

       

Attribution of authorship§

21

36%

26

33%

0.856

40

33%

Purpose clearly stated

69

88%

88

96%

0.090

141

96%

Privacy policy available

7

9%

33

36%

<0.001*

44

30%

Information versioning

1

1%

3

3%

0.626

4

3%

Contact details provided

32

41%

59

64%

0.003*

90

61%

Funding model clear

21

27%

30

33%

0.502

49

33%

Editorial/advertising policy

0

0%

5

5%

0.063

5

3%

Software issues

       

Data entry validation issues**

9

33%

8

32%

1.000

16

33%

Functionality issues

13

17%

9

10%

0.252

15

10%

User interface issues

25

32%

34

37%

0.522

45

31%

Crashes

8

10%

8

9%

0.796

13

9%

Network issues

3

4%

20

22%

<0.001*

25

17%

Other issues

10

13%

18

20%

0.301

28

19%

The table compares the proportion of apps satisfying best practices for content attribution and ethical disclosures (detailed in Additional file 1: Table AF3), and those with software issues identified during testing (detailed with examples in Additional file 1: Table AF4), between 2011 and 2013.

The proportion of information apps that satisfy a particular domain. Unless otherwise stated, the denominator is the total number of apps shown in the relevant heading.

Two-tailed probability obtained from Fisher’s exact test of the comparison of topic coverage in apps available in 2011 and those available since 2013. *After applying a Holm-Bonferroni correction to control the family-wise error rate within each domain at less than or equal to 5%, three comparisons were significant.

§Denominator for proportions reflects only apps with attributable content, e.g., written information, measurement scales, pollen/pollution data. Attribution of the software itself lies with the developers whose details are always released. Denominators: 2011 n = 59, New 2013 n = 80, 2013 n = 120.

¤ New 2013, Apps available in 2013 that were not available in 2011; All 2013, All apps available in 2013, excluding those available in 2011 that were subsequently discontinued.

**Data entry validation concerns steps taken to prevent out-of-range or inappropriate data being stored in an app, for example, textual values allowed in a numeric field. As a result, the denominator reflects only apps permitting data entry, e.g., calculators, diaries. Denominators: 2011 n = 27, New 2013 n = 25, 2013 n = 49.

We also assessed software issues encountered during the test process (using criteria detailed in Additional file 1: Table AF5). Compared to 2011, there was no change in the proportion of new apps lacking appropriate data entry validation to prevent, for example, textual data being entered in numeric data fields (33%, n = 9/27 vs. 32%, n = 8/25, P = 1.000). The proportion of apps with problems with the user interface, such as mislabelled data fields (32%, n = 25/78 vs. 37%, n = 34/92, P = 0.522), or core functionality, such as the correct operation of a calculator (17%, n = 13/78 vs. 10%, n = 9/92, P = 0.252), was also unchanged. However, there was a significant increase in apps released since 2011 experiencing network problems when trying to access online content or services (4%, n = 3/78 vs. 22%, n = 20/92, P <0.001). Crashes affected about one in ten apps during testing, unchanged from 2011 (9%, n = 13/147 overall).

Discussion

Our updated findings present a mixed picture for clinicians interested in integrating apps into routine care. While choice has increased, newer apps for asthma were no more likely than those available in 2011 to satisfy evidence-based recommendations for information content or the design of self-management tools. Our data extend findings of low overall compliance with evidence in recent assessments of medical apps for disease-management [20,48] and health-promotion [49,50] by suggesting that issues of quality are sustained and are not isolated to first-generation apps. However, as in 2011, our results question the scope for benefit offered by current apps for asthma. Firstly, rather than focusing on known gaps in self-management where there is evidence that intervention can improve outcomes, such as the use of action plans [22,51], most apps offer either very basic background information about asthma or simple peak flow diaries. Only a fifth of information apps available in 2013 addressed action plans and the proportion of diaries offering a mechanism for recording a plan was smaller. Secondly, although it is recognized that information provision alone has a limited impact on health behaviour [52], the proportion of apps combining information and tools remains small. While apps using multimedia methods, such as video and gameplay, presented welcome new opportunities for tailored support that focuses on specific needs, such as improving inhaler technique [53], or on specific groups, such as those with limited literacy [54], our assessment highlights the risk of clinically-relevant impacts resulting from inadequate content. For example, three quarters of inhaler training apps left out at least one step considered necessary for effective delivery of drug into the lungs. Thirdly, while apps incorporate a range of features, such as reminders, that could plausibly play a role in interventions designed to improve asthma self-management, apps themselves do not currently provide effective scaffolding to support individuals in changing, and sustaining, behaviour. For example, although many apps offered the ability to send data by email, and the proportion of apps using a mechanism to share data online almost doubled between 2011 and 2013 to 52%, no app offered a built-in mechanism for communication between a professional and patient, potentially limiting the scope for feedback that could assist self-management. Nor was it clear that the online services linked to currently available apps could support the kind of summary and flagging activities necessary to manage cohorts rather than single patients. Evidence is needed that app-supported remote monitoring can generate reliable data, prompt consistent and sustained clinical improvements, and meet the needs of patients across the spectrum of asthma severity [55]. Although more apps allowed information to be shared using a social network like Facebook or Twitter, this was never in the context of a structured activity relating to asthma self-management and no apps took advantage of the possibilities for peer support. Similar to the findings of others [19,56], only one asthma app appeared to be linked to an active research project [57], highlighting a broader disconnect between guidance on the design of interventions that incorporate evidence-based components and evaluation [58] and what occurs in practice.

Dynamic aspects of the asthma app marketplace, including rapid growth and turnover, have potential consequences for both clinical services and individuals. Over a quarter of apps included in the original review, including those from reputable content providers, had been withdrawn. While patients may choose apps to suit their changing needs, the costs associated with app selection, staff training, setting up support mechanisms, and appropriate governance create incentives for planned care to be based around small numbers of trusted apps. Unexpected withdrawal of an app may be an inconvenience for individual users, but can threaten the function of an established clinical service. App updates are also potentially disruptive, requiring testing and training. As a result, formal arrangements with app developers, such as service-level contracts, should be considered where apps are being used as part of planned care. Systematic identification of factors influencing app lifespan would be helpful and future work should address this.

A new finding concerned apps that repackaged existing information either to create content focused on promoting specific commercial products, or to serve as a more general vehicle for hosting adverts operating on a pay-per-click basis. A small number specifically emphasized negative effects associated with conventional medical practice, with the intention of promoting commercial alternatives. Some apps were developed using online tools that specifically target developers with no or limited coding experience. Marketing materials for these tools emphasized their ability to produce apps quickly and in number. If the goal of these apps is simply to generate advertising revenue or an initial purchase, content need only to appear superficially plausible. This kind of low-quality information, coupled with high volume distribution and commercial motive, shares similarities with spam or junk email and websites [59]. Indeed, some apps for asthma reused material originally produced by online content farms [60]. If sustained, the junk medical app would be an unwelcome development, making it harder for users to locate higher-quality and unbiased content. Unlike the web, where search engines tend to prioritize higher-quality content in results [61], conventional app stores offer limited mechanisms for filtering out lower quality and junk apps.

Apps offering unvalidated diagnostic, assessment, and therapeutic functions with no basis in evidence are a continuing issue that raises questions about patient safety and clinical liability [62]. While some apps are likely to fall under the scope of medical app regulation recently proposed in both the US and Europe [30,31], aspects of how this will work in practice are yet to be agreed, not least because regulators must balance competing demands of risk, costs, and the possible impact on innovation [14]. As a result, rapid action appears unlikely, even where standards already exist. For example, while physical sensors, such as peak flow meters, are classified as moderate risk (class 2) medical devices, neither of the two asthma apps that use a device microphone to analyse breath sounds appear to have undergone regulatory approval, and both remain available in November 2014, over 1 year since guidance first came out. Regulation is also constrained by the scope of current legislation which applies only to apps intended to diagnose, treat, or prevent disease. While not ruling out future changes, regulators have stated that information apps do not satisfy this definition, even if they contain procedural guidance such as first aid [63], and that they intend to apply discretion when enforcing rules for apps that probably qualify as medical devices but pose a low overall risk such as diaries, pollen alerts, and inhaler trackers [14]. While ensuring that potentially useful apps are not burdened by costly and time-consuming approvals, some apps will fall outside established reporting mechanisms that would allow a more complete picture of risk to emerge over time. Further work is necessary to understand the consequences of known defects in tools such as predicted peak flow calculators and identify unanticipated risks, for example, those arising from novel services like online pharmacies that do not involve a usual clinician.

Our findings have relevance for emerging medical app accreditation programs [32,64,65], which aim to reduce the burden of assessment placed on professionals and patients, provide recognizable badges of app quality, and collate certified apps to make them easily accessible. Rapid app turnover and feature evolution suggest that accreditation processes will need to be revised regularly and programs will need to allocate appropriate resources to reassess their growing cohort of existing certified apps as they are updated. Programs must also consider the possibility of best practices that vary by location and target group. For example, some US inhaler technique guidance apps advised actuating a metered dose inhaler in front of an open mouth, a method not taught in UK self-management education. The growth in connected apps, and those based around advertiser models, highlights the importance of assessment processes which examine privacy and security-related risks as personal and medical data start to be shared online. While we did not explore the governance arrangements of cloud-based apps, privacy and security are topical concerns for patients [55,66]. Additionally, recent findings highlighting gaps in privacy disclosures [67,68] and security arrangements [69,70] in popular health apps suggest that further scrutiny is required. High rates of app turnover emphasize the importance of due-diligence that focuses on the commitment of developers to maintaining their products. The range of functions seen within a single long-term condition in this study highlights the potential complexity of defining comprehensive assessment standards. Similar to the content-independent criteria applied in this study, simplified assessments which focus on high-risk functions and proxy markers for quality, such as the involvement of clinicians in design, have been proposed [71,72], but their predictive validity is unknown. While most accreditation programs appear to be developing their own criteria, a collaborative, open approach to standard development could share the workload, for example, by involving professional bodies with specific content expertise, enable transparent input by patients, professionals, and developers, and permit evaluation. Further work should explore the extent to which developers can be engaged proactively to increase awareness of accreditation requirements and the potential value of patient and professional input into health app design, which, in turn, may increase the likelihood that apps meet accreditation requirements without potentially costly cycles of rejection and resubmission. Finally, because accreditation programs are voluntary and have yet to prove their ability to control quality issues at scale, most health apps available in public app stores will not be badged for quality. Patient education could address this by, for example, explaining the phenomenon of junk medical apps and characterizing the risks as similar to those presented by the range of information available on the general internet.

The study used a systematic method to exhaustively sample apps from the two most popular app stores and included steps to minimize bias during the assessment process. By repeating an assessment first performed in 2011, we were able to explore aspects of the dynamics of app turnover and feature development. However, the study had a number of limitations. Apps in languages other than English and those available on platforms other than Android and iOS were not included. It is possible that there are systematic differences in app quality by language or platform that this study could not detect. Because the study involved only two time points, there should be substantial caution in inferring that any changes observed are indicative of trends. Our data reflect issues present in apps available at particular points in time, and while representative categories are likely to persist, the specifics will change, necessitating updated assessment. Finally, because of limitations in the data provided by app stores, we were unable to precisely quantify the scale of app use by patients nor the ways in which apps are used. The potential for a mismatch between available features and those desired by patients was highlighted by a recent review of apps for patients with Parkinson’s disease [73]. Future work could explore this issue in relation to asthma, examine whether user reviews and ratings have any predictive validity for app quality, and continue to develop the currently limited evidence-base specifically addressing the design of apps for asthma self-management [74].

Conclusions

In addition to established findings about variable clinical quality and possible safety risks, dynamic aspects of the asthma app marketplace, such as rapid growth and turnover, affect the suitability of apps for use in clinical settings. Efforts to assure the quality of medical apps need to be responsive to the changing features and risks posed by apps, consider governance as well as clinical factors, and recognize the differing needs of patients using apps on their own initiative and those supported by clinical care. Accreditation programs may need to be supplemented by proactive intervention by regulators to address potentially unsafe apps and advice to help patients navigate the increasing numbers of poor quality or ‘junk’ apps for asthma in general app stores.

Declarations

Acknowledgements

This work received no specific funding. However, KH’s doctoral studies are funded by the National Institute for Health Research (NIHR) under the Collaborations for Leadership in Applied Health Research and Care (CLAHRC) programme for North West London. They played no role in study design, in the collection, analysis, and interpretation of data, in the writing of the report, or in the decision to submit the article for publication. The views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

Authors’ Affiliations

(1)
Global eHealth Unit, Imperial College London
(2)
Health Services and Outcomes Research Programme, Lee Kong Chian School of Medicine, Nanyang Technological University

References

  1. Klasnja P, Pratt W. Healthcare in the pocket: mapping the space of mobile-phone health interventions. J Biomed Inform. 2011;45:184–98.View ArticlePubMedPubMed CentralGoogle Scholar
  2. Ofcom. The communications market 2014 (August). 2014. http://stakeholders.ofcom.org.uk/market-data-research/market-data/communications-market-reports/cmr14/.
  3. Pew Research Internet Project. Mobile technology fact sheet. http://www.pewinternet.org/fact-sheets/mobile-technology-fact-sheet/.
  4. Fox S, Duggan M. Mobile health 2012. Washington DC: Pew Research Center; 2012.Google Scholar
  5. research2guidance. Mobile health market report 2013–2017. http://www.research2guidance.com/shop/index.php/mhealth-report-2.
  6. Aitken M, Gauntlett C. Patient apps for improved healthcare – from novelty to mainstream. Parsippany, NJ: IMS Institute for Healthcare Informatics; 2013.Google Scholar
  7. Manhattan Research. 2014 Environment. http://manhattanresearch.com/Products-and-Services/Physician/Taking-the-Pulse-U-S.
  8. Digitas Health. mBook 2013 edition – marketing mobile health. 2013. http://m.2013.digitashealth.com/Mbook2013_web.pdf.
  9. Steinhubl SR, Muse ED, Topol EJ. Can mobile health technologies transform health care? JAMA. 2013;310:2395–6.View ArticlePubMedGoogle Scholar
  10. European Commission. Green paper on mobile health ("mHealth"). Brussels: EC; 2014.Google Scholar
  11. England NHS. Five year forward view. London: HM Government; 2014.Google Scholar
  12. Rosser BA, Eccleston C. Smartphone applications for pain management. J Telemed Telecare. 2011;17:308–12.View ArticlePubMedGoogle Scholar
  13. McCartney M. How do we know whether medical apps work? BMJ. 2013;346:f1811.View ArticlePubMedGoogle Scholar
  14. Cortez NG, Cohen IG, Kesselheim AS. FDA regulation of mobile health technologies. N Engl J Med. 2014;371:372–9.View ArticlePubMedGoogle Scholar
  15. Wolf JA, Moreau JF, Akilov O, Patton T, English 3rd JC, Ho J, et al. Diagnostic inaccuracy of smartphone applications for melanoma detection. JAMA Dermatol. 2013;149:422–6.View ArticlePubMedPubMed CentralGoogle Scholar
  16. Huckvale K, Car J. Implementation of mobile health tools. JAMA. 2014;311:1447–8.View ArticlePubMedGoogle Scholar
  17. Atienza AA, Patrick K. Mobile health: the killer app for cyberinfrastructure and consumer health. Am J Prev Med. 2011;40:S151–3.View ArticlePubMedGoogle Scholar
  18. Buijink AWG, Visser BJ, Marshall L. Medical apps for smartphones: lack of evidence undermines quality and safety. Evid Based Med. 2013;18:90–2.View ArticlePubMedGoogle Scholar
  19. De La Vega R, Miro J. mHealth: a strategic field without a solid scientific soul. A systematic review of pain-related apps. PLoS One. 2014;9:e101312.View ArticlePubMedPubMed CentralGoogle Scholar
  20. Bender JL, Yue RY, To MJ, Deacken L, Jadad AR. A lot of action, but not in the right direction: systematic review and content analysis of smartphone applications for the prevention, detection, and management of cancer. J Med Internet Res. 2013;15:e287.View ArticlePubMedPubMed CentralGoogle Scholar
  21. Huckvale K, Car M, Morrison C, Car J. Apps for asthma self-management: a systematic assessment of content and tools. BMC Med. 2012;10:144.View ArticlePubMedPubMed CentralGoogle Scholar
  22. Partridge MR, Negro RWD, Olivieri D. Understanding patients with asthma and COPD: insights from a European study. Prim Care Respir J. 2011;20:315–23.View ArticlePubMedGoogle Scholar
  23. Masoli M, Fabian D, Holt S, Beasley R, Global Initiative for Asthma (GINA) Program. The global burden of asthma: executive summary of the GINA dissemination committee report. Allergy. 2004;59:469–78.View ArticlePubMedGoogle Scholar
  24. British Thoracic Society, Scottish Intercollegiate Guidelines Network. British guideline on the management of asthma – a national clinical guideline. May 2008, revised January 2012. Edinburgh: SIGN; 2012.Google Scholar
  25. Patel S, Park H, Bonato P, Chan L, Rodgers M. A review of wearable sensors and systems with application in rehabilitation. J Neuroeng Rehabil. 2012;9:21.View ArticlePubMedPubMed CentralGoogle Scholar
  26. Heredia A, Garcia-Guzman J, Amescua-Seco A, Velasco-Diego M. Agile practices adapted to mass-market application development. J Softw (Malden). 2014;26:818–28.Google Scholar
  27. Charani E, Castro-Sanchez E, Moore L, Holmes A. Do smartphone applications in healthcare require a governance and legal framework? It depends on the application! BMC Med. 2014;12:29.View ArticlePubMedPubMed CentralGoogle Scholar
  28. Barton A. The regulation of mobile health applications. BMC Med. 2012;10:46.View ArticlePubMedPubMed CentralGoogle Scholar
  29. Alliance ECD. Green paper on mHealth apps – input from the European Chronic Disease Alliance (ECDA). Brussels: ECDA; 2014.Google Scholar
  30. Food US, Administration D. Mobile medical applications – guidance for industry and food and drug administration staff. US Department of Health and Human Services: Rockville, MD; 2013.Google Scholar
  31. Medicines and Healthcare Regulatory Agency. Guidance on medical device stand-alone software (including apps). http://www.mhra.gov.uk/Howweregulate/Devices/Software/index.htm.
  32. Ferrero-Alvarez-Rementeria J, Santana-Lopez V, Escobar-Ubreva A, Vazquez-Vazquez M. Quality and safety strategy for mobile health applications: a certification programme. Eur J ePractice. 2013;21:14–26.Google Scholar
  33. Marvel FA, Chase J, Madruga M. Ideas to iPhones: A 10-step framework for creating mobile medical applications with case report from Madruga and Marvel’s Medical Black Book app. J Mob Technol Med. 2014;3:35–41.View ArticleGoogle Scholar
  34. Information Commissioner's Office. Privacy in mobile apps – guidance for app developers. https://ico.org.uk/media/for-organisations/documents/1596/privacy-in-mobile-apps-dp-guidance.pdf.
  35. Macharia E. Five tips for healthcare app development. http://www.software.ac.uk/blog/2013-10-08-five-tips-healthcare-app-development.
  36. IDC. Smartphone OS market share, Q3 2014. http://www.idc.com/prodserv/smartphone-os-market-share.jsp.
  37. National Asthma Education and Prevention Program. Expert panel report 3: guidelines for the diagnosis and management of asthma. Full report 2007. Bethseda, MD: National Institutes of Health; 2007.Google Scholar
  38. Global Initiative for Asthma. Global strategy for asthma management and prevention. Updated 2012. http://www.ginasthma.org.
  39. National Centre for Complementary and Alternative Medicine (NCCAM). Complementary, alternative, or integrative health: what’s in a name? http://nccam.nih.gov/health/whatiscam.
  40. Health on the Net Foundation. Operational definition of the HONcode principles. Guidelines v2.0. http://www.hon.ch/HONcode/Guidelines/guidelines.html#Review.
  41. Fisher RA. On the interpretation of X2 from contingency tables, and the calculation of P. J R Stat Soc. 1922;85:87–94.View ArticleGoogle Scholar
  42. Freeman GH, Halton JH. Note on an exact treatment of contingency, goodness of fit and other problems of significance. Biometrika. 1951;38:141–9.View ArticlePubMedGoogle Scholar
  43. Holm S. A simple sequentially rejective multiple test procedure. Scand J Stat. 1979;6:65–70.Google Scholar
  44. Cates CJ, Rowe BH. Vaccines for preventing influenza in people with asthma. Cochrane Database Syst Rev. 2013;2, CD000364.PubMedGoogle Scholar
  45. van der Palen J, Klein JJ, Kerkhoff AHM, van Herwaarden CLA, Seydel ER. Evaluation of the long-term effectiveness of three instruction modes for inhaling medicines. Patient Educ Couns. 1997;32:S87–95.View ArticlePubMedGoogle Scholar
  46. Melani AS, Bonavia M, Cilenti V, Cinti C, Lodi M, Martucci P, et al. Inhaler mishandling remains common in real life and is associated with reduced disease control. Respir Med. 2011;105:930–8.View ArticlePubMedGoogle Scholar
  47. Freitas DA, Holloway EA, Bruno SS, Chaves GS, Fregonezi GA, Mendonca KP. Breathing exercises for adults with asthma. Cochrane Database Syst Rev. 2013;10, CD001277.PubMedGoogle Scholar
  48. Chomutare T, Fernandez-Luque L, Årsand E, Hartvigsen G. Features of mobile diabetes applications: review of the literature and analysis of current applications compared against evidence-based guidelines. J Med Internet Res. 2011;13:e65.View ArticlePubMedPubMed CentralGoogle Scholar
  49. Abroms LC, Lee Westmaas J, Bontemps-Jones J, Ramani R, Mellerson J. A content analysis of popular smartphone apps for smoking cessation. Am J Prev Med. 2013;45:732–6.View ArticlePubMedGoogle Scholar
  50. Breton E, Fuemmeler B, Abroms L. Weight loss – there is an app for that! But does it adhere to evidence-informed practices? Behav Med Pract Policy Res. 2011;1:523–9.View ArticleGoogle Scholar
  51. Gibson PG, Powell H. Written action plans for asthma: an evidence-based review of the key components. Thorax. 2004;59:94–9.View ArticlePubMedPubMed CentralGoogle Scholar
  52. Bodenheimer T, Lorig K, Holman H, Grumbach K. Patient self-management of chronic disease in primary care. JAMA. 2002;288:2469–75.View ArticlePubMedGoogle Scholar
  53. Newcomb PA, McGrath KW, Covington JK, Lazarus SC, Janson SL. Barriers to patient-clinician collaboration in asthma management: the patient experience. J Asthma. 2010;47:192–7.View ArticlePubMedGoogle Scholar
  54. Paasche-Orlow MK, Riekert KA, Bilderback A, Chanmugam A, Hill P, Rand CS, et al. Tailored education may reduce health literacy disparities in asthma self-management. Am J Respir Crit Care Med. 2005;172:980–6.View ArticlePubMedPubMed CentralGoogle Scholar
  55. Hall JL, McGraw D. For telehealth to succeed, privacy and security risks must be identified and addressed. Health Aff (Millwood). 2014;33:216–21.View ArticleGoogle Scholar
  56. Mosnaim GS, Powell LH, Rathkopf M. A review of published studies using interactive internet tools or mobile devices to improve asthma knowledge or health outcomes. Pediatr Allergy Immunol Pulmonol. 2012;25:55–63.View ArticleGoogle Scholar
  57. Burnay E, Cruz-Correia R, Jacinto T, Sousa AS, Fonseca J. Challenges of a mobile application for asthma and allergic rhinitis patient enablement-interface and synchronization. Telemed J E Health. 2013;19:13–8.View ArticlePubMedGoogle Scholar
  58. Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med. 2013;46:81–95.View ArticlePubMedGoogle Scholar
  59. Markines B, Cattuto C, Menczer F. Social spam detection. 5th International Workshop on Adversarial Information Retrieval on the Web. 20 April 2009; Madrid: ACM; 2009. p. 41–8.Google Scholar
  60. McCreadie R, Macdonald C, Ounis I, Giles J, Jabr F. An examination of content farms in web search using crowdsourcing. 21st ACM International Conference on Information and Knowledge Management. 29 October – 2 November 2012. Maui, Hawaii: ACM; 2012. p. 2551–4.Google Scholar
  61. Colledge A, Car J, Donnelly A, Majeed A. Health information for patients: time to look beyond patient information leaflets. J R Soc Med. 2008;101:447–53.View ArticlePubMedPubMed CentralGoogle Scholar
  62. Yang YT, Silverman RD. Mobile health applications: the patchwork of legal and liability issues suggests strategies to improve oversight. Health Aff (Millwood). 2014;33:222–7.View ArticleGoogle Scholar
  63. Food US, Administration D. Appendix A – examples of mobile apps that are not medical devices. Mobile Medical Applications – Guidance for Industry and Food and Drug Administration Staff. Rockville, MD: US Department of Health and Human Services; 2013.Google Scholar
  64. Agencia de Calidad Sanitaria de Andalucia. [Distintivo app saludable]. http://www.calidadappsalud.com/.
  65. NHS Health Apps Library. Review process. http://apps.nhs.uk/review-process.
  66. King J. “How come I’m allowing strangers to go through my phone?” – smartphones and privacy expectations. Symposium On Usable Privacy and Security (SOUPS) 2012. July 11th–13th, 2012. Washington, DC; 2012.Google Scholar
  67. Office of the Privacy Commissioner of Canada. Results of the 2014 Global Privacy Enforcement Network sweep. https://www.priv.gc.ca/media/nr-c/2014/bg_140910_e.asp.
  68. Sunyaev A, Dehling T, Taylor PL, Mandl KD. Availability and quality of mobile health app privacy policies. J Am Med Inform Assoc. 2014.Google Scholar
  69. Privacy Rights Clearinghouse. Fact Sheet 39: Mobile health and fitness apps: what are the privacy risks? https://www.privacyrights.org/mobile-health-and-fitness-apps-what-are-privacy-risks.
  70. Dolan B. Happtique suspends mobile health app certification program. http://mobihealthnews.com/28165/happtique-suspends-mobile-health-app-certification-program/.
  71. Lewis TL. A systematic self-certification model for mobile medical apps. J Med Internet Res. 2013;15:e89.View ArticlePubMedPubMed CentralGoogle Scholar
  72. Albrecht UV, von Jan U, Jungnickel T, Pramann O. App-synopsis – standard reporting for medical apps. Stud Health Technol Inform. 2013;192:1154.PubMedGoogle Scholar
  73. Divakaruni A, Armstrong M, Gruber-Baldini A, Shulman L. Discordance between medication apps and needs of patients with Parkinson's disease. Neurology. 2014;82:Supplement P3.067.Google Scholar
  74. Marcano Belisario JS, Huckvale K, Greenfield G, Car J, Gunn LH. Smartphone and tablet self management apps for asthma. Cochrane Database Syst Rev. 2013;11, CD010013.PubMedGoogle Scholar

Copyright

© Huckvale et al.; licensee BioMed Central. 2015

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Advertisement