Skip to main content
Download PDF

Validation of a deep-learning-based retinal biomarker (Reti-CVD) in the prediction of cardiovascular disease: data from UK Biobank

BMC Medicine volume 21, Article number: 28 (2023) Cite this article

Abstract

Background

Currently in the United Kingdom, cardiovascular disease (CVD) risk assessment is based on the QRISK3 score, in which 10% 10-year CVD risk indicates clinical intervention. However, this benchmark has limited efficacy in clinical practice and the need for a more simple, non-invasive risk stratification tool is necessary. Retinal photography is becoming increasingly acceptable as a non-invasive imaging tool for CVD. Previously, we developed a novel CVD risk stratification system based on retinal photographs predicting future CVD risk. This study aims to further validate our biomarker, Reti-CVD, (1) to detect risk group of ≥ 10% in 10-year CVD risk and (2) enhance risk assessment in individuals with QRISK3 of 7.5–10% (termed as borderline-QRISK3 group) using the UK Biobank.

Methods

Reti-CVD scores were calculated and stratified into three risk groups based on optimized cut-off values from the UK Biobank. We used Cox proportional-hazards models to evaluate the ability of Reti-CVD to predict CVD events in the general population. C-statistics was used to assess the prognostic value of adding Reti-CVD to QRISK3 in borderline-QRISK3 group and three vulnerable subgroups.

Results

Among 48,260 participants with no history of CVD, 6.3% had CVD events during the 11-year follow-up. Reti-CVD was associated with an increased risk of CVD (adjusted hazard ratio [HR] 1.41; 95% confidence interval [CI], 1.30–1.52) with a 13.1% (95% CI, 11.7–14.6%) 10-year CVD risk in Reti-CVD-high-risk group. The 10-year CVD risk of the borderline-QRISK3 group was greater than 10% in Reti-CVD-high-risk group (11.5% in non-statin cohort [n = 45,473], 11.5% in stage 1 hypertension cohort [n = 11,966], and 14.2% in middle-aged cohort [n = 38,941]). C statistics increased by 0.014 (0.010–0.017) in non-statin cohort, 0.013 (0.007–0.019) in stage 1 hypertension cohort, and 0.023 (0.018–0.029) in middle-aged cohort for CVD event prediction after adding Reti-CVD to QRISK3.

Conclusions

Reti-CVD has the potential to identify individuals with ≥ 10% 10-year CVD risk who are likely to benefit from earlier preventative CVD interventions. For borderline-QRISK3 individuals with 10-year CVD risk between 7.5 and 10%, Reti-CVD could be used as a risk enhancer tool to help improve discernment accuracy, especially in adult groups that may be pre-disposed to CVD.

Peer Review reports

Background

In the United Kingdom (UK), around 1 in 9 people live with cardiovascular disease (CVD) [1, 2]. A formal risk assessment tool, QRISK3, was developed using data from a cohort of 1.28 million individuals, and it is currently recommended by the National Institute for Health and Care Excellence (NICE) for CVD risk assessment and guidance for primary prevention strategies [3]. In 2014, NICE updated the requirements for statin initiation in individuals at risk of developing CVD, increasing the number of eligible individuals for statin treatment [4]. NICE also updated the requirements for antihypertensive initiation in individuals with stage 1 hypertension (clinically-measured blood pressure ranging from 140/90 to 159/99 mmHg) in 2019 [5]. The current recommendation for initiating statins for primary CVD prevention and antihypertensives for stage 1 hypertension is a calculated QRISK3 score of 10% [4, 5]. Therefore, the benchmark of 10% for 10-year CVD risk is very important in determining clinical intervention.

Despite these updated guidelines, their effect on clinical practice has been limited, with little change in statin prescription behavior [6]. Moreover, considering that 10-year CVD risk might underestimate the lifetime probability of developing CVD, NICE recommends that antihypertensive treatment should also be considered for people under 60 who have a predicted 10-year CVD risk below 10%. Practice-changing guidelines would require time-consuming longitudinal prospective studies. A simple and personalized CVD risk refinement tool may be helpful in not only minimizing time and various resource costs, but also finding consensus on statin and antihypertensive treatment initiation.

Retinal photography is a simple, effective, and non-invasive imaging tool that provides fast and accurate information on the human vasculature that may not be clearly visible to the human eye especially in recent years with the advent of deep learning (DL) [7,8,9]. In our previous study, we developed a DL-based three-tier CVD risk stratification system, RetiCAC, using coronary artery calcium (CAC) as a ground truth, which allowed us to stratify CVD risk in the UK Biobank and a diversified Asian population [10]. However, this study had no comparison of RetiCAC with QRISK3 and was not UK-specific, thereby limiting its application in clinical decision-making such as statin and anti-hypertensive medication initiation in the UK.

In this study, we aimed to optimize the operating threshold of RetiCAC, herein referred to as Reti-CVD, via simple retinal imaging. Using Reti-CVD, we aimed (1) to detect risk group of ≥ 10% in 10-year CVD risk and (2) to evaluate the potential of Reti-CVD as a risk enhancer when applied alongside QRISK3 for borderline-QRISK3 groups (i.e., individuals with a 10-year CVD risk of 7.5–10%) in the UK Biobank.

Methods

Ethics statement

This retrospective study was deemed exempt from institutional review board (IRB) review by the SingHealth Centralised Institutional Review Board (CIRB). This study adhered to the tenets of the Declaration of Helsinki. Written informed consent was obtained from the participants of the original studies [10, 11].

Study population

We used clinical data and retinal photographs from the UK Biobank, a prospective population-based cohort in the UK [12]. The UK Biobank protocol is available online [13].

We excluded (1) duplicated retinal photographs (n = 18,423), (2) those who had type 1 diabetes (n = 288), (3) those with pre-existing CVD at baseline (n = 7624), (4) poor-quality photographs (n = 11,115), and (5) those who were < 40 years old (n = 1) (Additional file 1: eFigure 1). Pre-existing CVD was defined as previous history of coronary heart disease, other heart diseases, stroke, transient ischemic attack, peripheral arterial disease, or cardiovascular surgery, and patients who have undergone cardiovascular procedures based on the Classification of Interventions and Procedures version 4 (OPCS-4) [10].

A total of 48,260 participants, representing the general population without a history of CVD, were included for analysis. In addition, we also defined three at-risk subgroups: (1) the non-statin cohort, individuals not taking a statin; (2) the stage 1 hypertension cohort, individuals with stage 1 hypertension and not taking any antihypertensive medications; and (3) the middle-aged cohort, individuals aged 40–64 years at baseline (Additional file 1: eFigure 1). Since the purpose of Reti-CVD is aimed at primary prevention of CVD, we focus on individuals who display a lack of awareness towards the risk factors associated with cardiovascular disease. In a previous publication, the authors reported based on age-stratified analyses that unawareness rates were highest in individuals aged between 40 and 49 years old and lowest in individuals aged above 70 years old [14]. Therefore, our middle-aged cohort is classified based on vulnerability.

Retinal photographs included in the study were taken using the Topcon 3D OCT-1000 Mark II (Topcon Corporation) between 7 December 2009 and 21 July 2010. Retinal cameras used in the training set include AFP-210 non-mydriatic auto retinal camera (NIDEK Corporation, Aichi, Japan), TRC-NW8 non-mydriatic retinal camera (Topcon Corporation, Tokyo, Japan), and Nonmyd A-D (Kowa Co. Ltd., Shizuoka, Japan). We did not include Topcon 3D OCT-1000 Mark II (Topcon Corporation) in our training set.

The other variables used in this study were defined as follows. Pre-diabetes and diabetes were defined based on (1) medical history and (2) glucose ≥ 5.5 mmol/L. Medical history of high cholesterol, type 1 diabetes, antihypertensive were self-declared and collected from a baseline assessment questionnaire on medical conditions. Smoking status was self-declared as well and categorized into “life-time smoker” and “never.”

Definition of cardiovascular disease events

In the UK Biobank, we used hospitalization and mortality data provided by the National Health Service (NHS) registers. The main outcome of interest in the current study reflected the outcome used in the QRISK3 risk score: fatal CVD events (ICD-10 I00-99, F01, Q20-Q28, C38.0, P29, G45) [15] or nonfatal coronary heart disease, ischemic stroke, or transient ischemic attack (ICD-10 G45, I20–24, and I63–64) [16].

Calculation of the QRISK3 score and borderline-QRISK3 group

For each individual, the QRISK3 score was calculated using R package, version 3.6 [17]. The distribution of the QRISK3 score is provided in Additional file 2: eFigure 2. For comparison with three-strata Reti-CVD groups, we divided the subjects into 5 groups based on the QRISK3 score (%) (≥ 0 to < 5; ≥ 5 to > 10; ≥ 10 to > 15; ≥ 15 to < 20; and ≥ 20). In addition, as the recommended threshold for initiating statins and antihypertensive medication from a 10-year risk of CVD is 10%, we defined “borderline-risk” group as those who had QRISK3 score between 7.5 and 10%. Specifically, we divided the subjects into 5 groups as compared to the previous three-risk-strata group of QRISK3 as outlined in the NICE guidelines. Herein lies the difference in intervals used to evaluate CVD events. We used 5% intervals compared to the previous 10% intervals so that a more detailed comparison could be done with Reti-CVD.

New retinal-based cardiovascular disease risk stratification system ≈

Details of RetiCAC model development and previous validations have been described elsewhere [9]. Briefly, the RetiCAC score was defined based on a probability score derived from our deep-learning algorithm of binary classification (absence vs presence of coronary artery calcium [CAC]). The probability scores ranged from zero to one, with a high value indicating a high probability of the presence of CAC. First, we enhanced the RetiCAC algorithm using more datasets which includes both retinal photographs and CT scans from Korea (Additional file 3: eDocument 1). Second, we proposed new cardiovascular disease risk stratification groups (i.e., Reti-CVD) with optimized cut-off values based on 40th percentile and 95th percentile of Reti-CVD score among 48,260 participants after exclusion. The cut-off values were determined to have similar incidence rate: the low-risk group of Reti-CVD was designed to have similar incidence rate to those of the 0 to 5% QRISK3 risk group (i.e., 2.5 per 1000 person-years), and the moderate-risk of Reti-CVD group was designed to have similar incidence rate to those of the 5 to 10% QRISK3 risk group (i.e., 7.0 per 1000 person-years). We used these proposed cut-off values to further stratify the CVD risk in the UK Biobank participants.

Statistical analysis

Analyses were done using p < 0.05 as the significance level, Stata/MP version 14.0 for survival analysis, and R version 3.4.4 for estimation of net reclassification index (NRI) using the R package survIDINRI [18]. Descriptive statistics are provided for all participants and also according to the three risk groups by Reti-CVD.

In the UK Biobank, hospitalization and mortality data were available up to May 05, 2021, at the time of analysis and each participant was followed up to 11.4 years from the date of baseline visit. In survival analysis, each patient was followed up to 11.4 years (median follow-up, 11.0 years) from the date of baseline visit to the last follow-up date or the date of the CVD events.

In all populations, the cumulative incidence of cardiovascular events rate was evaluated across the three groups (low, moderate, and high risk) defined by the Reti-CVD using Kaplan-Meier method. Cox proportional hazards model was used to estimate the hazard ratios (HRs) and trends in HRs and respective p-values were examined by fitting a linear model for the three categories. Unadjusted HR was provided according to three risk groups of Reti-CVD, and QRISK3-ajudted HR trend was provided.

To help future patients make an informed decision on statin and antihypertensive medication initiation, we only included the borderline-QRISK3 group who had QRISK3 score between 7.5 and 10% of 10-year CVD risk. In the borderline QRISK3 group, cumulative incidence of cardiovascular events rate was evaluated across the three groups (low, moderate, and high risk) according to the Reti-CVD and compared with participants with QRISK3 5–7.5% and 10–12.5%. The same analysis was repeated for middle-aged group (40 to 64 years).

The incremental prognostic value of the Reti-CVD over the QRISK3 in the prediction of CVD events was assessed using C-statistics and continuous net reclassification index (NRI) [18]. In addition, decision curve analysis was used to compare the net-benefit of models at different thresholds over QRISK3 model and Reti-CVD plus age and gender model. Age and gender were included for fair comparison because QRISK3 is based on survival model including various risk factors including age, gender, smoking, and comorbidities. Continuous models were presented as decision-analysis curves (demonstrating potential outcomes of using any threshold in that model) and models with higher net-benefit were considered higher performing.

Results

Study population characteristics

Table 1 details the clinical characteristics of the participants by the three groups of Reti-CVD score (categorized as low, moderate, and high-risk groups).

Table 1 Characteristics of the study population
Full size table

Among 48,260 included participants from the UK Biobank, the median QRISK3 10-year CVD risk was 4.5% (IQR 2.2–8.0%, SD, 4.2%); CVD events occurred in 2766 (5.7%) participants during the follow-up. The Spearman’s rank correlation coefficient between Reti-CVD and QRISK3 score was 0.50 (p < 0.001).

CVD events rate were 2.8% (545/19,304) in the low-risk group, 7.2% (1900/26,543) in the moderate-risk group, and 13.3% (321/2413) in the high-risk group of Reti-CVD. Among the low-risk group of Reti-CVD, 84.3% had QRISK3 score of 0 to < 5%, and among the high-risk group of Reti-CVD, 8.6% had QRISK3 score of 0 to 5%, 40.1% had QRISK3 score of ≥ 5 to < 10%, and 36.3% had QRISK3 score of ≥ 10 to < 15%.

Performance of Reti-CVD in prediction of CVD event

Kaplan-Meier curves were described in all participants according to five groups of QRISK3 and three groups of Reti-CVD (Fig. 1). During the follow-up (median 11.0 years; interquartile range [IQR], 10.9–11.1 years), 513,714.3 person-years were analyzed. As expected, QRISK3 stratified CVD risk well in general population of the UK Biobank (Fig. 1A). Reti-CVD also shows distinct CVD risk stratification based on the three groups within the general population of the UK Biobank (Fig. 1B). Based on Reti-CVD, the incidence of CVD per 1000 person-years was 2.6 (95% CI, 2.4–2.8) in the low-risk group, 6.8 (6.5–7.1) in the moderate-risk group, and 13.1 (11.7–14.6) in the high-risk group (Table 2), indicating a 13.1% 10-year CVD risk in Reti-CVD-high-risk group. Analysis for non-statin cohort and stage 1 hypertension cohort are provided in Additional file 4: eFigure 3 and Additional file 5: eTable 1.

Fig. 1
figure 1

Kaplan-Meier curves according to Reti-CVD and QRISK3. Cardiovascular disease (CVD) event rate according to QRISK3-five groups (A), and Reti-CVD-three groups (B) in all participants (n = 48,260) were presented. In the general population, the high-risk group of Reti-CVD (B) shows a similar incident CVD rate to that of the QRISK 10–15% group (A), indicating that if a participant is identified as Reti-CVD-high risk by retinal photography, participants may be advised to undergo a full CVD risk assessment via the NHS primary care service

Full size image
Table 2 Risk of cardiovascular events by the deep-learning-based retinal CVD biomarker (Reti-CVD) in all participants
Full size table

Additional file 6: eTable 2 describes the performance of Reti-CVD in predicting the incidence of CVD events when applied to at-risk subgroup populations. Incidence of CVD and QRISK3-adjusted HR trends were provided in 3 subgroups: individuals with body-mass index (BMI) above 25 kg/m2, hypertension, and pre-diabetes/diabetes. Our results show that the three-strata Reti-CVD system can further stratify CVD risk in all at-risk subgroup populations. Specifically, in the subgroup of individuals who were taking antihypertensive medication, the incidence of CVD was as high as 17.7 (95% CI, 15.0–20.8) in Reti-CVD-high-risk group.

Performance of Reti-CVD as a risk enhancer in borderline-QRISK3 group

The Kaplan-Meier analysis of CVD events in the UK Biobank according to the three strata of Reti-CVD among borderline group individuals (QRISK3 score between 7.5 and 10%) is provided in Fig. 2. Although QRISK3 was less than 10% in the borderline group, the cumulative CVD events rate in Reti-CVD-high-risk group was consistently higher than 10% of 10-year CVD risk.

Fig. 2
figure 2

Kaplan-Meier curves according to Reti-CVD in borderline-QRISK3 group. Cardiovascular disease (CVD) event rate according to Reti-CVD-three groups in borderline-QRISK3 group who had QRISK3 score between 7.5 and 10% were presented in non-statin cohort (A), stage 1 hypertension cohort (B), and middle-aged (40–64 years) cohorts (C). Considering that statin and antihypertensive pharmacotherapy initiation is recommended at QRISK of ≥ 10%, Reti-CVD can be used as a risk enhancer in borderline-QRISK3 groups of 7.5–10% to reach consensus on statin initiation. In addition, although most of risk assessment systems were derived from cohorts of primarily middle-aged people and typically well-functioning individuals, Reti-CVD can still be a risk enhancer in in borderline-QRISK3 groups of 7.5–10% middle-aged people

Full size image

Specifically, in this borderline-QRISK3 group, the 10-year CVD risk was higher than 10% in Reti-CVD-high-risk group with 11.5% (8.9–15.0%) in non-statin cohort (n = 45,473), 11.5% (7.4–17.8%) in stage 1 hypertension cohort (n = 11,966), and 14.2% (10.3–19.5%) in middle-age cohort (n = 38,941) (Table 3).

Table 3 Risk of cardiovascular events by the deep-learning-based retinal CVD biomarker (Reti-CVD) in participants who had 7.5–10% QRISK3 score
Full size table

Risk discrimination and incremental value

We then tested the incremental value of Reti-CVD when applied alongside QRISK3 in predicting incident CVD events among non-statin, stage 1 hypertension, and middle-aged cohorts (Table 4). By adding Reti-CVD score to QRISK3, C statistics increased by 0.014 (95% CI, 0.010–0.017) in non-statin cohort, 0.013 (0.007–0.019) in stage 1 hypertension cohort, and 0.023 (0.018–0.029) in middle-aged cohort for prediction of CVD events. Also, by adding Reti-CVD score to QRISK3, the continuous NRI was 0.133 (95% CI, 0.088–0.173) in non-statin cohort, 0.094 (0.008–0.174) in stage 1 hypertension cohort, and 0.248 (0.190–0.301) in middle-aged cohort.

Table 4 Prognostic performance with the addition of Reti-CVD to the QRISK3 in the UK biobank dataset
Full size table

Additional file 7: eFigure 4 summarizes net-benefit comparisons using a decision-analysis curve. In entire UK Biobank participants, Additional file 7: eFigure 4A presents the decision-analysis curves for QRISK3 score and Reti-CVD plus age, and gender model, indicating that the two models are almost similar in terms of net-benefit. In the hypertensive patients and the pre-diabetes and diabetes subgroup (Additional file 7: eFigure 4B and 4C), the Reti-CVD plus age, and gender model demonstrated a higher net-benefit than QRISK3 score in general.

Discussion

In this study, we optimized the operating threshold of RetiCAC, which was originally based on the Korean cohort. This optimized biomarker, Reti-CVD, stratified the general UK population into three risk groups and presented with significant hazard ratio (HR) trends. Specifically, the high-risk group in Reti-CVD adequately identified individuals who had a 10-year CVD risk greater than 10%, with a CVD incidence of 13.1 per 1000 person-years. Moreover, we quantified the cumulative CVD events and prognostic value for risk stratification when adding Reti-CVD alongside QRISK3 in the borderline-QRISK3 group (10-year CVD risk of 7.5–10%). Due to the new cut-off threshold of 10% for statin and antihypertensive treatment initiation, we focused on individuals who had a QRISK3 score between 7.5 and 10% and showed that Reti-CVD could further stratify the CVD risk into low, moderate, and high-risk groups. Specifically, in the borderline-risk group, individuals who were stratified as high-risk using Reti-CVD had a 10-year CVD risk greater than 10% and was comparable to the CVD incidence of QRISK3 (10 to 12.5%) group, indicating that Reti-CVD could be used as a risk enhancer.

Like other areas of medicine, there is now a paradigm shift in cardiovascular medicine, towards using digital innovations especially those related to artificial intelligence to enhance diagnosis and risk stratification [19]. Retinal imaging has been posited as a potential imaging application for risk assessment in a wide variety of diseases. Reti-CVD, as a further extension of RetiCAC, when applied alongside QRISK3, provides a good reflection of CVD severity in the UK population. This underscores the importance of the retinal imaging-based Reti-CVD tool for clinical CVD risk stratification.

Previous studies using retinal imaging as a tool for predicting CVD risk have focused largely on other risk factors such as age, hypertension, and dyslipidemia [20,21,22]. Other studies have investigated the use of retinal imaging to determine the risk of specific CVD subtypes such as coronary heart disease [23] and peripheral arterial disease [24]. Compared to these studies, our study focuses on predicting future CVD events in general instead of diagnosis of current subtypes of CVD. Studies that have used DL-based retinal imaging to predict coronary artery calcium (CAC) scores were either applied on small sample sizes and/or did not evaluate the ability of the deep learning systems to predict future CVD events [25, 26]. In particular, a study by Chang et al. evaluated the value-added ability of its DL-based system in predicting CVD mortality from retinal imaging when added to the Framingham risk scoring (FRS) system [27]. However, Hippisley-Cox et al has suggested that the FRS algorithm has ethnicity bias and may be less suited to the UK cohort compared to QRISK algorithms [3]. Therefore, our novel focus in applying Reti-CVD to QRISK3 reflects both the UK Biobank demographic population and NICE guidelines, as QRISK3 was recommended by NICE.

In the general population, we have shown that Reti-CVD can identify individuals with a CVD risk greater than 10% in our high-risk Reti-CVD group. This finding is useful for two reasons because a CVD risk of 10% is the latest threshold for statin and anti-hypertensive medication initiation. First, at the primary care level, Reti-CVD provides GPs with a simple and effective way of providing an initial assessment on asymptomatic patients. Ophthalmologists, optometrists, and opticians can also use existing retinal photographs for opportunistic eye screening. Second, Reti-CVD can aid the CVD prevention program that NHS England has developed. The NHS CVD primary prevention focusses on three main components: atrial fibrillation, blood pressure, and cholesterol [28]. Given that our study included individuals who are not on statin or anti-hypertensive medication in the general population, physicians can use Reti-CVD while monitoring patients’ blood pressure and cholesterol levels to better predict CVD risk. If identified as high-risk by Reti-CVD, patients can be encouraged to undergo a NHS health check-up for further evaluation. This could boost the compliance rate for NHS health screening, which has been estimated to be less than 50% [29]. This low-cost opportunistic CVD risk screening based on routine eye checks may help to include ‘hard to reach' populations in CVD risk assessment.

An additional finding that we want to highlight is the fact that Reti-CVD identified high-risk individuals who were otherwise missed by QRISK3. 8.6% of patients identified as high-risk by our new stratification tool had a QRISK3 score between 0 to 5 and 40.1% of them were categorized in the QRISK3 ≥ 5 to < 10% group. This demonstrates the potential of Reti-CVD as an adjunct tool to identify future high-risk individuals who may be left out if CVD risk stratification is dependent solely on QRISK3 scoring. Consequently, when we incorporate only two basic demographic factors of age and gender to Reti-CVD, this model was superior to QRISK3 model across the range of risk thresholds. Moreover, in our explorative analysis in eTable 3, we confirmed that upon conducting a more detailed QRISK3 analysis in the borderline QRISK3 group where individuals were divided based on 0.5 intervals between 7.5 and 10%, more than 1.6% of individuals were classified in the Reti-CVD-high-risk groups in all subgroups. This indicates that Reti-CVD can be used as an independent risk enhancer in this range of QRISK3.

By comparing Reti-CVD to QRISK3, this study demonstrates the potential value and scope of Reti-CVD to improve discernment accuracy in the UK population since QRISK3 is part of the up-to-date guidelines that reflects the demographic profile of the UK Biobank. Study limitations include a possibility of systematic errors in the form of misclassification bias, since hospitalization and mortality data provided by the NHS registers were used for CVD definition. However, a study by Kivimaki et al. has analyzed the validity of and reported good agreement between CVD event predictions and the use of UK National Health Electronic Records, therefore minimizing the likelihood of CVD event misdiagnoses [30]. We also excluded poor quality images during the development and validation of Reti-CVD. The quality of retinal photographs available in clinical settings may vary, and therefore this may pose challenges for real-world implementation.

Conclusions

In conclusion, a DL and retinal photograph-derived new CVD biomarker, Reti-CVD, could effectively stratify CVD risk in the general population. We also confirmed that Reti-CVD has the potential to identify individuals with ≥ 10% 10-year CVD risk who are likely to benefit from preventative CVD interventions. In addition, for people with a QRISK3 score between 7.5 and 10%, Reti-CVD could be used as a risk stratification enhancer for identifying high risk patients for aggressive CVD prevention.

Availability of data and materials

The UK Biobank test dataset was obtained from UK Biobank (application number 68428). Data cannot be shared publicly due to the violation of patient privacy and the absence of informed consent for data sharing.

Abbreviations

CVD:

Cardiovascular disease

UK:

United Kingdom

NICE:

National Institute for Health and Care Excellence

DL:

Deep-learning

CAC:

Coronary artery calcium

IRB:

Institutional Review Board

CIRB:

SingHealth Centralized Institutional Review Board

NHS:

National Health Service

NRI:

Net Reclassification Index

BMI:

Body-mass index

HR:

Hazard ratio

FRS:

Framingham risk score

GP:

General practitioner

References

  1. Deshmukh M, Liu YC, Rim TH, Venkatraman A, Davidson M, Yu M, et al. Automatic segmentation of corneal deposits from corneal stromal dystrophy images via deep learning. Comput Biol Med. 2021;137:104675.

    Article  Google Scholar 

  2. Ng WY, Zhang S, Wang Z, Ong CJT, Gunasekeran DV, Lim GYS, et al. Updates in deep learning research in ophthalmology. Clin Sci (Lond). 2021;135(20):2357–76.

    Article  Google Scholar 

  3. Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, May M, Brindle P. Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study. BMJ. 2007;335(7611):136.

    Article  Google Scholar 

  4. Cardiovascular disease: risk assessment and reduction, including lipid modification [https://www.nice.org.uk/guidance/cg181/resources]. 2014.

  5. Jones NR, McCormack T, Constanti M, McManus RJ. Diagnosis and management of hypertension in adults: NICE guideline update 2019. Br J Gen Pract. 2020;70(691):90–1.

    Article  Google Scholar 

  6. Pate A, Emsley R, van Staa T. Impact of lowering the risk threshold for statin treatment on statin prescribing: a descriptive study in English primary care. Br J Gen Pract. 2020;70(700):e765–e71.

    Article  Google Scholar 

  7. Betzler BK, Yang HHS, Thakur S, Yu M, Quek TC, Soh ZD, et al. Gender prediction for a multiethnic population via deep learning across different retinal fundus photograph fields: retrospective cross-sectional study. JMIR Med Inform. 2021;9(8):e25165.

    Article  Google Scholar 

  8. Tseng RMWW, Rim TH, Cheung CY, Wong TY. Artificial intelligence using the eye as a biomarker of systemic risk. Artificial Intelligence in Ophthalmology: Springer; 2021. p. 243–55.

    Google Scholar 

  9. Rim TH, Lee G, Kim Y, Tham YC, Lee CJ, Baik SJ, et al. Prediction of systemic biomarkers from retinal photographs: development and validation of deep-learning algorithms. Lancet Digit Health. 2020;2(10):e526–e36.

    Article  Google Scholar 

  10. Rim TH, Lee CJ, Tham Y-C, Cheung N, Yu M, Lee G, et al. Deep-learning-based cardiovascular risk stratification using coronary artery calcium scores predicted from retinal photographs. Lancet Digital Health. 2021;3(5):e306–e16.

    Article  Google Scholar 

  11. Sudlow C, Gallacher J, Allen N, Beral V, Burton P, Danesh J, et al. UK biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS Med. 2015;12(3):e1001779.

    Article  Google Scholar 

  12. Palmer LJ. UK Biobank: bank on it. Lancet. 2007;369(9578):1980–2.

    Article  Google Scholar 

  13. UK Biobank: Protocol for a large-scale prospective epidemiological resource. 2007.

    Google Scholar 

  14. Man REK, Gan AHW, Fenwick EK, Gan ATL, Gupta P, Sabanayagam C, et al. Prevalence, determinants and association of unawareness of diabetes, hypertension and hypercholesterolemia with poor disease control in a multi-ethnic Asian population without cardiovascular disease. Popul Health Metrics. 2019;17(1):17.

    Article  Google Scholar 

  15. UK Factsheet [https://www.bhf.org.uk/-/media/files/research/heart-statistics/bhf-cvd-statistics---ukfactsheet.pdf]. 2022.

  16. Welsh C, Welsh P, Celis-Morales CA, Mark PB, Mackay D, Ghouri N, et al. Glycated hemoglobin, prediabetes, and the links to cardiovascular disease: data from UK Biobank. Diabetes Care. 2020;43(2):440–5.

    Article  CAS  Google Scholar 

  17. Li Y, Sperrin M, van Staa T. R package “QRISK3”: an unofficial research purposed implementation of ClinRisk’s QRISK3 algorithm into R. F1000Research. 2020;8:2139.

    Article  Google Scholar 

  18. Pencina MJ, D'Agostino RB Sr, D'Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157–72 discussion 207-12.

    Article  Google Scholar 

  19. Vardas PE, Asselbergs FW, van Smeden M, Friedman P. The year in cardiovascular medicine 2021: digital health and innovation. Eur Heart J. 2022;43(4):271–9.

    Article  Google Scholar 

  20. Zhang L, Yuan M, An Z, Zhao X, Wu H, Li H, et al. Prediction of hypertension, hyperglycemia and dyslipidemia from retinal fundus photographs via deep learning: A cross-sectional study of chronic diseases in central China. PLoS One. 2020;15(5):e0233166.

    Article  CAS  Google Scholar 

  21. Poplin R, Varadarajan AV, Blumer K, Liu Y, McConnell MV, Corrado GS, et al. Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning. Nat Biomed Eng. 2018;2(3):158–64.

    Article  Google Scholar 

  22. Gerrits N, Elen B, Craenendonck TV, Triantafyllidou D, Petropoulos IN, Malik RA, et al. Age and sex affect deep learning prediction of cardiometabolic risk factors from retinal images. Sci Rep. 2020;10(1):9432.

    Article  CAS  Google Scholar 

  23. Wang JJ, Liew G, Wong TY, Smith W, Klein R, Leeder SR, et al. Retinal vascular calibre and the risk of coronary heart disease-related death. Heart. 2006;92(11):1583–7.

    Article  CAS  Google Scholar 

  24. Mueller S, Wintergerst MWM, Falahat P, Holz FG, Schaefer C, Schahab N, et al. Multiple instance learning detects peripheral arterial disease from high-resolution color fundus photography. Sci Rep. 2022;12(1):1389.

    Article  CAS  Google Scholar 

  25. Barriada RG, Simó-Servat O, Planas A, Hernández C, Simó R, Masip D. Deep Learning of Retinal Imaging: A Useful Tool for Coronary Artery Calcium Score Prediction in Diabetic Patients. Appl Sci. 2022;12:1401. https://doi.org/10.3390/app12031401.

  26. Son J, Shin JY, Chun EJ, Jung K-H, Park KH, Park SJ. Predicting high coronary artery calcium score from retinal fundus images with deep learning algorithms. Transl Vis Sci Technol. 2020;9(2):28.

    Article  Google Scholar 

  27. Chang J, Ko A, Park SM, Choi S, Kim K, Kim SM, et al. Association of cardiovascular mortality and deep learning-funduscopic atherosclerosis score derived from retinal fundus images. Am J Ophthalmol. 2020;217:121–30.

    Article  Google Scholar 

  28. Arnould L, Guenancia C, Bourredjem A, Binquet C, Gabrielle PH, Eid P, et al. Prediction of cardiovascular parameters with supervised machine learning from singapore “I” vessel assessment and OCT-angiography: a pilot study. Transl Vision Sci Technol. 2021;10(13):20.

    Article  Google Scholar 

  29. Martin A, Saunders CL, Harte E, Griffin SJ, MacLure C, Mant J, et al. Delivery and impact of the NHS health check in the first 8 years: a systematic review. Br J Gen Pract. 2018;68(672):e449–e59.

    Article  Google Scholar 

  30. Kivimäki M, Batty GD, Singh-Manoux A, Britton A, Brunner EJ, Shipley MJ. Validity of cardiovascular disease event ascertainment using linkage to UK hospital records. Epidemiology. 2017;28(5):735–9.

    Article  Google Scholar 

  31. UK Biobank research ethics approval. 2021. [https://www.ukbiobank.ac.uk/learn-more-about-ukbiobank/about-us/ethics].

Download references

Acknowledgements

Not applicable.

Funding

This project is supported by the Agency for Science, Technology and Research (A*STAR) under its RIE2020 Health and Biomedical Sciences (HBMS) Industry Alignment Fund Pre-Positioning (IAF-PP) Grant No. H20c6a0031. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of the A*STAR.

Author information

Author notes

  1. Rachel Marjorie Wei Wen Tseng and Tyler Hyungtaek Rim contributed equally to this work.

Authors and Affiliations

  1. Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore

    Rachel Marjorie Wei Wen Tseng, Tyler Hyungtaek Rim, Sahil Thakur, Simon Nusinovici, Qingsheng Peng, Marco Yu, Yih-Chung Tham, Tien Yin Wong & Ching-Yu Cheng

  2. Duke-NUS Medical School, Singapore, Singapore

    Rachel Marjorie Wei Wen Tseng

  3. Ophthalmology and Visual Sciences Academic Clinical Program (Eye ACP), Duke-NUS Medical School, Singapore, Singapore

    Tyler Hyungtaek Rim, Simon Nusinovici, Marco Yu, Yih-Chung Tham & Ching-Yu Cheng

  4. Mediwhale Inc., Seoul, South Korea

    Tyler Hyungtaek Rim, Hyeonmin Kim & Geunyoung Lee

  5. Department of Primary Care and Mental Health, University of Liverpool, Liverpool, UK

    Eduard Shantsila

  6. Albert Einstein College of Medicine, New York, NY, USA

    Joseph K. Yi

  7. Division of Cardiology, Severance Cardiovascular Hospital, Yonsei University College of Medicine, Seoul, South Korea

    Sungha Park & Chan Joo Lee

  8. Division of Retina, Severance Eye Hospital, Yonsei University College of Medicine, Seoul, South Korea

    Sung Soo Kim

  9. Clinical and Translational Sciences Program, Duke-NUS Medical School, Singapore, Singapore

    Qingsheng Peng

  10. Center for Innovation and Precision Eye Health, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore

    Yih-Chung Tham & Ching-Yu Cheng

  11. Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore

    Yih-Chung Tham & Ching-Yu Cheng

  12. Royal Free Hospital London NHS Foundation Trust, London, UK

    Ameet Bakhai

  13. Cardiology Department, Barnet General Hospital, Thames House, Enfield, UK

    Ameet Bakhai

  14. Cardiovascular Clinical Research Facility, RDM Division of Cardiovascular Medicine, University of Oxford, Oxford, UK

    Paul Leeson

  15. Liverpool Centre for Cardiovascular Science, University of Liverpool and Liverpool John Moores University and Liverpool Heart & Chest Hospital, Liverpool, United Kingdom; and Department of Clinical Medicine, Aalborg University, Aalborg, Denmark

    Gregory Y.H. Lip

  16. Tsinghua Medicine, Tsinghua University, Beijing, China

    Tien Yin Wong

Authors
  1. Rachel Marjorie Wei Wen Tseng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tyler Hyungtaek Rim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eduard Shantsila
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joseph K. Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sungha Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sung Soo Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Chan Joo Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Sahil Thakur
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Simon Nusinovici
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Qingsheng Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Hyeonmin Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Geunyoung Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Marco Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Yih-Chung Tham
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Ameet Bakhai
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Paul Leeson
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Gregory Y.H. Lip
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Tien Yin Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Ching-Yu Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

T.H.R. conceptualized the study. G.L. developed the algorithm. R.M.W.W.T. and T.H.R. wrote the manuscript and researched its contents and edited all versions. All authors substantially revised the manuscript, read, and approved the final manuscript.

Corresponding author

Correspondence to Tyler Hyungtaek Rim.

Ethics declarations

Ethics approval and consent to participate

The UK Biobank dataset was obtained ethically from UK Biobank (application number 68428). The UK Biobank has also received Research Tissue Bank approval from North West Multi-centre Research Ethics Committee and the Human Tissue Authority license [31].

Consent for publication

Not applicable.

Competing interests

T.H.R. was a former scientific adviser and owns stock of Mediwhale Inc. H.K. and G.L. are employees of Mediwhale Inc. T.H.R. and G.L. hold the following patents that might be affected by this study: 10–2018–0166720(KR), 10–2018–0166721(KR), 10–2018–0166722(KR), 62/694,901(US), 62/715,729(US), and 62/776,345 (US). All other authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: eFigure 1.

Study flow chart.

Additional file 2: eFigure 2.

Distribution of the QRISK3 score in the UK Biobank.

Additional file 3: eDocument 1.

RetiCAC model updates.

Additional file 4: eFigure 3.

Kaplan-Meier curves according to Reti-CVD and QRISK3.

Additional file 5: eTable 1.

Risk of cardiovascular events by the deep-learning-based retinal CVD biomarker (Reti-CVD).

Additional file 6: eTable 2.

Subgroup analysis in the UK Biobank.

Additional file 7: eFigure 4.

Decision curve analysis.

Additional file 8: eTable 3.

Detailed distribution of Reti-CVD risk groups among those between 7.5% and 10% QRISK3 score.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tseng, R.M.W.W., Rim, T.H., Shantsila, E. et al. Validation of a deep-learning-based retinal biomarker (Reti-CVD) in the prediction of cardiovascular disease: data from UK Biobank. BMC Med 21, 28 (2023). https://doi.org/10.1186/s12916-022-02684-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12916-022-02684-8

Keywords

  • Artificial intelligence
  • Cardiovascular disease
  • Deep learning
  • Retinal imaging
  • Retinal photograph
  • Risk stratification
  • Risk stratification system
  • UK Biobank

BMC Medicine

ISSN: 1741-7015

Contact us