- Research article
- Open Access
Validation of a deep-learning-based retinal biomarker (Reti-CVD) in the prediction of cardiovascular disease: data from UK Biobank
BMC Medicine volume 21, Article number: 28 (2023)
Currently in the United Kingdom, cardiovascular disease (CVD) risk assessment is based on the QRISK3 score, in which 10% 10-year CVD risk indicates clinical intervention. However, this benchmark has limited efficacy in clinical practice and the need for a more simple, non-invasive risk stratification tool is necessary. Retinal photography is becoming increasingly acceptable as a non-invasive imaging tool for CVD. Previously, we developed a novel CVD risk stratification system based on retinal photographs predicting future CVD risk. This study aims to further validate our biomarker, Reti-CVD, (1) to detect risk group of ≥ 10% in 10-year CVD risk and (2) enhance risk assessment in individuals with QRISK3 of 7.5–10% (termed as borderline-QRISK3 group) using the UK Biobank.
Reti-CVD scores were calculated and stratified into three risk groups based on optimized cut-off values from the UK Biobank. We used Cox proportional-hazards models to evaluate the ability of Reti-CVD to predict CVD events in the general population. C-statistics was used to assess the prognostic value of adding Reti-CVD to QRISK3 in borderline-QRISK3 group and three vulnerable subgroups.
Among 48,260 participants with no history of CVD, 6.3% had CVD events during the 11-year follow-up. Reti-CVD was associated with an increased risk of CVD (adjusted hazard ratio [HR] 1.41; 95% confidence interval [CI], 1.30–1.52) with a 13.1% (95% CI, 11.7–14.6%) 10-year CVD risk in Reti-CVD-high-risk group. The 10-year CVD risk of the borderline-QRISK3 group was greater than 10% in Reti-CVD-high-risk group (11.5% in non-statin cohort [n = 45,473], 11.5% in stage 1 hypertension cohort [n = 11,966], and 14.2% in middle-aged cohort [n = 38,941]). C statistics increased by 0.014 (0.010–0.017) in non-statin cohort, 0.013 (0.007–0.019) in stage 1 hypertension cohort, and 0.023 (0.018–0.029) in middle-aged cohort for CVD event prediction after adding Reti-CVD to QRISK3.
Reti-CVD has the potential to identify individuals with ≥ 10% 10-year CVD risk who are likely to benefit from earlier preventative CVD interventions. For borderline-QRISK3 individuals with 10-year CVD risk between 7.5 and 10%, Reti-CVD could be used as a risk enhancer tool to help improve discernment accuracy, especially in adult groups that may be pre-disposed to CVD.
In the United Kingdom (UK), around 1 in 9 people live with cardiovascular disease (CVD) [1, 2]. A formal risk assessment tool, QRISK3, was developed using data from a cohort of 1.28 million individuals, and it is currently recommended by the National Institute for Health and Care Excellence (NICE) for CVD risk assessment and guidance for primary prevention strategies . In 2014, NICE updated the requirements for statin initiation in individuals at risk of developing CVD, increasing the number of eligible individuals for statin treatment . NICE also updated the requirements for antihypertensive initiation in individuals with stage 1 hypertension (clinically-measured blood pressure ranging from 140/90 to 159/99 mmHg) in 2019 . The current recommendation for initiating statins for primary CVD prevention and antihypertensives for stage 1 hypertension is a calculated QRISK3 score of 10% [4, 5]. Therefore, the benchmark of 10% for 10-year CVD risk is very important in determining clinical intervention.
Despite these updated guidelines, their effect on clinical practice has been limited, with little change in statin prescription behavior . Moreover, considering that 10-year CVD risk might underestimate the lifetime probability of developing CVD, NICE recommends that antihypertensive treatment should also be considered for people under 60 who have a predicted 10-year CVD risk below 10%. Practice-changing guidelines would require time-consuming longitudinal prospective studies. A simple and personalized CVD risk refinement tool may be helpful in not only minimizing time and various resource costs, but also finding consensus on statin and antihypertensive treatment initiation.
Retinal photography is a simple, effective, and non-invasive imaging tool that provides fast and accurate information on the human vasculature that may not be clearly visible to the human eye especially in recent years with the advent of deep learning (DL) [7,8,9]. In our previous study, we developed a DL-based three-tier CVD risk stratification system, RetiCAC, using coronary artery calcium (CAC) as a ground truth, which allowed us to stratify CVD risk in the UK Biobank and a diversified Asian population . However, this study had no comparison of RetiCAC with QRISK3 and was not UK-specific, thereby limiting its application in clinical decision-making such as statin and anti-hypertensive medication initiation in the UK.
In this study, we aimed to optimize the operating threshold of RetiCAC, herein referred to as Reti-CVD, via simple retinal imaging. Using Reti-CVD, we aimed (1) to detect risk group of ≥ 10% in 10-year CVD risk and (2) to evaluate the potential of Reti-CVD as a risk enhancer when applied alongside QRISK3 for borderline-QRISK3 groups (i.e., individuals with a 10-year CVD risk of 7.5–10%) in the UK Biobank.
This retrospective study was deemed exempt from institutional review board (IRB) review by the SingHealth Centralised Institutional Review Board (CIRB). This study adhered to the tenets of the Declaration of Helsinki. Written informed consent was obtained from the participants of the original studies [10, 11].
We used clinical data and retinal photographs from the UK Biobank, a prospective population-based cohort in the UK . The UK Biobank protocol is available online .
We excluded (1) duplicated retinal photographs (n = 18,423), (2) those who had type 1 diabetes (n = 288), (3) those with pre-existing CVD at baseline (n = 7624), (4) poor-quality photographs (n = 11,115), and (5) those who were < 40 years old (n = 1) (Additional file 1: eFigure 1). Pre-existing CVD was defined as previous history of coronary heart disease, other heart diseases, stroke, transient ischemic attack, peripheral arterial disease, or cardiovascular surgery, and patients who have undergone cardiovascular procedures based on the Classification of Interventions and Procedures version 4 (OPCS-4) .
A total of 48,260 participants, representing the general population without a history of CVD, were included for analysis. In addition, we also defined three at-risk subgroups: (1) the non-statin cohort, individuals not taking a statin; (2) the stage 1 hypertension cohort, individuals with stage 1 hypertension and not taking any antihypertensive medications; and (3) the middle-aged cohort, individuals aged 40–64 years at baseline (Additional file 1: eFigure 1). Since the purpose of Reti-CVD is aimed at primary prevention of CVD, we focus on individuals who display a lack of awareness towards the risk factors associated with cardiovascular disease. In a previous publication, the authors reported based on age-stratified analyses that unawareness rates were highest in individuals aged between 40 and 49 years old and lowest in individuals aged above 70 years old . Therefore, our middle-aged cohort is classified based on vulnerability.
Retinal photographs included in the study were taken using the Topcon 3D OCT-1000 Mark II (Topcon Corporation) between 7 December 2009 and 21 July 2010. Retinal cameras used in the training set include AFP-210 non-mydriatic auto retinal camera (NIDEK Corporation, Aichi, Japan), TRC-NW8 non-mydriatic retinal camera (Topcon Corporation, Tokyo, Japan), and Nonmyd A-D (Kowa Co. Ltd., Shizuoka, Japan). We did not include Topcon 3D OCT-1000 Mark II (Topcon Corporation) in our training set.
The other variables used in this study were defined as follows. Pre-diabetes and diabetes were defined based on (1) medical history and (2) glucose ≥ 5.5 mmol/L. Medical history of high cholesterol, type 1 diabetes, antihypertensive were self-declared and collected from a baseline assessment questionnaire on medical conditions. Smoking status was self-declared as well and categorized into “life-time smoker” and “never.”
Definition of cardiovascular disease events
In the UK Biobank, we used hospitalization and mortality data provided by the National Health Service (NHS) registers. The main outcome of interest in the current study reflected the outcome used in the QRISK3 risk score: fatal CVD events (ICD-10 I00-99, F01, Q20-Q28, C38.0, P29, G45)  or nonfatal coronary heart disease, ischemic stroke, or transient ischemic attack (ICD-10 G45, I20–24, and I63–64) .
Calculation of the QRISK3 score and borderline-QRISK3 group
For each individual, the QRISK3 score was calculated using R package, version 3.6 . The distribution of the QRISK3 score is provided in Additional file 2: eFigure 2. For comparison with three-strata Reti-CVD groups, we divided the subjects into 5 groups based on the QRISK3 score (%) (≥ 0 to < 5; ≥ 5 to > 10; ≥ 10 to > 15; ≥ 15 to < 20; and ≥ 20). In addition, as the recommended threshold for initiating statins and antihypertensive medication from a 10-year risk of CVD is 10%, we defined “borderline-risk” group as those who had QRISK3 score between 7.5 and 10%. Specifically, we divided the subjects into 5 groups as compared to the previous three-risk-strata group of QRISK3 as outlined in the NICE guidelines. Herein lies the difference in intervals used to evaluate CVD events. We used 5% intervals compared to the previous 10% intervals so that a more detailed comparison could be done with Reti-CVD.
New retinal-based cardiovascular disease risk stratification system ≈
Details of RetiCAC model development and previous validations have been described elsewhere . Briefly, the RetiCAC score was defined based on a probability score derived from our deep-learning algorithm of binary classification (absence vs presence of coronary artery calcium [CAC]). The probability scores ranged from zero to one, with a high value indicating a high probability of the presence of CAC. First, we enhanced the RetiCAC algorithm using more datasets which includes both retinal photographs and CT scans from Korea (Additional file 3: eDocument 1). Second, we proposed new cardiovascular disease risk stratification groups (i.e., Reti-CVD) with optimized cut-off values based on 40th percentile and 95th percentile of Reti-CVD score among 48,260 participants after exclusion. The cut-off values were determined to have similar incidence rate: the low-risk group of Reti-CVD was designed to have similar incidence rate to those of the 0 to 5% QRISK3 risk group (i.e., 2.5 per 1000 person-years), and the moderate-risk of Reti-CVD group was designed to have similar incidence rate to those of the 5 to 10% QRISK3 risk group (i.e., 7.0 per 1000 person-years). We used these proposed cut-off values to further stratify the CVD risk in the UK Biobank participants.
Analyses were done using p < 0.05 as the significance level, Stata/MP version 14.0 for survival analysis, and R version 3.4.4 for estimation of net reclassification index (NRI) using the R package survIDINRI . Descriptive statistics are provided for all participants and also according to the three risk groups by Reti-CVD.
In the UK Biobank, hospitalization and mortality data were available up to May 05, 2021, at the time of analysis and each participant was followed up to 11.4 years from the date of baseline visit. In survival analysis, each patient was followed up to 11.4 years (median follow-up, 11.0 years) from the date of baseline visit to the last follow-up date or the date of the CVD events.
In all populations, the cumulative incidence of cardiovascular events rate was evaluated across the three groups (low, moderate, and high risk) defined by the Reti-CVD using Kaplan-Meier method. Cox proportional hazards model was used to estimate the hazard ratios (HRs) and trends in HRs and respective p-values were examined by fitting a linear model for the three categories. Unadjusted HR was provided according to three risk groups of Reti-CVD, and QRISK3-ajudted HR trend was provided.
To help future patients make an informed decision on statin and antihypertensive medication initiation, we only included the borderline-QRISK3 group who had QRISK3 score between 7.5 and 10% of 10-year CVD risk. In the borderline QRISK3 group, cumulative incidence of cardiovascular events rate was evaluated across the three groups (low, moderate, and high risk) according to the Reti-CVD and compared with participants with QRISK3 5–7.5% and 10–12.5%. The same analysis was repeated for middle-aged group (40 to 64 years).
The incremental prognostic value of the Reti-CVD over the QRISK3 in the prediction of CVD events was assessed using C-statistics and continuous net reclassification index (NRI) . In addition, decision curve analysis was used to compare the net-benefit of models at different thresholds over QRISK3 model and Reti-CVD plus age and gender model. Age and gender were included for fair comparison because QRISK3 is based on survival model including various risk factors including age, gender, smoking, and comorbidities. Continuous models were presented as decision-analysis curves (demonstrating potential outcomes of using any threshold in that model) and models with higher net-benefit were considered higher performing.
Study population characteristics
Table 1 details the clinical characteristics of the participants by the three groups of Reti-CVD score (categorized as low, moderate, and high-risk groups).
Among 48,260 included participants from the UK Biobank, the median QRISK3 10-year CVD risk was 4.5% (IQR 2.2–8.0%, SD, 4.2%); CVD events occurred in 2766 (5.7%) participants during the follow-up. The Spearman’s rank correlation coefficient between Reti-CVD and QRISK3 score was 0.50 (p < 0.001).
CVD events rate were 2.8% (545/19,304) in the low-risk group, 7.2% (1900/26,543) in the moderate-risk group, and 13.3% (321/2413) in the high-risk group of Reti-CVD. Among the low-risk group of Reti-CVD, 84.3% had QRISK3 score of 0 to < 5%, and among the high-risk group of Reti-CVD, 8.6% had QRISK3 score of 0 to 5%, 40.1% had QRISK3 score of ≥ 5 to < 10%, and 36.3% had QRISK3 score of ≥ 10 to < 15%.
Performance of Reti-CVD in prediction of CVD event
Kaplan-Meier curves were described in all participants according to five groups of QRISK3 and three groups of Reti-CVD (Fig. 1). During the follow-up (median 11.0 years; interquartile range [IQR], 10.9–11.1 years), 513,714.3 person-years were analyzed. As expected, QRISK3 stratified CVD risk well in general population of the UK Biobank (Fig. 1A). Reti-CVD also shows distinct CVD risk stratification based on the three groups within the general population of the UK Biobank (Fig. 1B). Based on Reti-CVD, the incidence of CVD per 1000 person-years was 2.6 (95% CI, 2.4–2.8) in the low-risk group, 6.8 (6.5–7.1) in the moderate-risk group, and 13.1 (11.7–14.6) in the high-risk group (Table 2), indicating a 13.1% 10-year CVD risk in Reti-CVD-high-risk group. Analysis for non-statin cohort and stage 1 hypertension cohort are provided in Additional file 4: eFigure 3 and Additional file 5: eTable 1.
Additional file 6: eTable 2 describes the performance of Reti-CVD in predicting the incidence of CVD events when applied to at-risk subgroup populations. Incidence of CVD and QRISK3-adjusted HR trends were provided in 3 subgroups: individuals with body-mass index (BMI) above 25 kg/m2, hypertension, and pre-diabetes/diabetes. Our results show that the three-strata Reti-CVD system can further stratify CVD risk in all at-risk subgroup populations. Specifically, in the subgroup of individuals who were taking antihypertensive medication, the incidence of CVD was as high as 17.7 (95% CI, 15.0–20.8) in Reti-CVD-high-risk group.
Performance of Reti-CVD as a risk enhancer in borderline-QRISK3 group
The Kaplan-Meier analysis of CVD events in the UK Biobank according to the three strata of Reti-CVD among borderline group individuals (QRISK3 score between 7.5 and 10%) is provided in Fig. 2. Although QRISK3 was less than 10% in the borderline group, the cumulative CVD events rate in Reti-CVD-high-risk group was consistently higher than 10% of 10-year CVD risk.
Specifically, in this borderline-QRISK3 group, the 10-year CVD risk was higher than 10% in Reti-CVD-high-risk group with 11.5% (8.9–15.0%) in non-statin cohort (n = 45,473), 11.5% (7.4–17.8%) in stage 1 hypertension cohort (n = 11,966), and 14.2% (10.3–19.5%) in middle-age cohort (n = 38,941) (Table 3).
Risk discrimination and incremental value
We then tested the incremental value of Reti-CVD when applied alongside QRISK3 in predicting incident CVD events among non-statin, stage 1 hypertension, and middle-aged cohorts (Table 4). By adding Reti-CVD score to QRISK3, C statistics increased by 0.014 (95% CI, 0.010–0.017) in non-statin cohort, 0.013 (0.007–0.019) in stage 1 hypertension cohort, and 0.023 (0.018–0.029) in middle-aged cohort for prediction of CVD events. Also, by adding Reti-CVD score to QRISK3, the continuous NRI was 0.133 (95% CI, 0.088–0.173) in non-statin cohort, 0.094 (0.008–0.174) in stage 1 hypertension cohort, and 0.248 (0.190–0.301) in middle-aged cohort.
Additional file 7: eFigure 4 summarizes net-benefit comparisons using a decision-analysis curve. In entire UK Biobank participants, Additional file 7: eFigure 4A presents the decision-analysis curves for QRISK3 score and Reti-CVD plus age, and gender model, indicating that the two models are almost similar in terms of net-benefit. In the hypertensive patients and the pre-diabetes and diabetes subgroup (Additional file 7: eFigure 4B and 4C), the Reti-CVD plus age, and gender model demonstrated a higher net-benefit than QRISK3 score in general.
In this study, we optimized the operating threshold of RetiCAC, which was originally based on the Korean cohort. This optimized biomarker, Reti-CVD, stratified the general UK population into three risk groups and presented with significant hazard ratio (HR) trends. Specifically, the high-risk group in Reti-CVD adequately identified individuals who had a 10-year CVD risk greater than 10%, with a CVD incidence of 13.1 per 1000 person-years. Moreover, we quantified the cumulative CVD events and prognostic value for risk stratification when adding Reti-CVD alongside QRISK3 in the borderline-QRISK3 group (10-year CVD risk of 7.5–10%). Due to the new cut-off threshold of 10% for statin and antihypertensive treatment initiation, we focused on individuals who had a QRISK3 score between 7.5 and 10% and showed that Reti-CVD could further stratify the CVD risk into low, moderate, and high-risk groups. Specifically, in the borderline-risk group, individuals who were stratified as high-risk using Reti-CVD had a 10-year CVD risk greater than 10% and was comparable to the CVD incidence of QRISK3 (10 to 12.5%) group, indicating that Reti-CVD could be used as a risk enhancer.
Like other areas of medicine, there is now a paradigm shift in cardiovascular medicine, towards using digital innovations especially those related to artificial intelligence to enhance diagnosis and risk stratification . Retinal imaging has been posited as a potential imaging application for risk assessment in a wide variety of diseases. Reti-CVD, as a further extension of RetiCAC, when applied alongside QRISK3, provides a good reflection of CVD severity in the UK population. This underscores the importance of the retinal imaging-based Reti-CVD tool for clinical CVD risk stratification.
Previous studies using retinal imaging as a tool for predicting CVD risk have focused largely on other risk factors such as age, hypertension, and dyslipidemia [20,21,22]. Other studies have investigated the use of retinal imaging to determine the risk of specific CVD subtypes such as coronary heart disease  and peripheral arterial disease . Compared to these studies, our study focuses on predicting future CVD events in general instead of diagnosis of current subtypes of CVD. Studies that have used DL-based retinal imaging to predict coronary artery calcium (CAC) scores were either applied on small sample sizes and/or did not evaluate the ability of the deep learning systems to predict future CVD events [25, 26]. In particular, a study by Chang et al. evaluated the value-added ability of its DL-based system in predicting CVD mortality from retinal imaging when added to the Framingham risk scoring (FRS) system . However, Hippisley-Cox et al has suggested that the FRS algorithm has ethnicity bias and may be less suited to the UK cohort compared to QRISK algorithms . Therefore, our novel focus in applying Reti-CVD to QRISK3 reflects both the UK Biobank demographic population and NICE guidelines, as QRISK3 was recommended by NICE.
In the general population, we have shown that Reti-CVD can identify individuals with a CVD risk greater than 10% in our high-risk Reti-CVD group. This finding is useful for two reasons because a CVD risk of 10% is the latest threshold for statin and anti-hypertensive medication initiation. First, at the primary care level, Reti-CVD provides GPs with a simple and effective way of providing an initial assessment on asymptomatic patients. Ophthalmologists, optometrists, and opticians can also use existing retinal photographs for opportunistic eye screening. Second, Reti-CVD can aid the CVD prevention program that NHS England has developed. The NHS CVD primary prevention focusses on three main components: atrial fibrillation, blood pressure, and cholesterol . Given that our study included individuals who are not on statin or anti-hypertensive medication in the general population, physicians can use Reti-CVD while monitoring patients’ blood pressure and cholesterol levels to better predict CVD risk. If identified as high-risk by Reti-CVD, patients can be encouraged to undergo a NHS health check-up for further evaluation. This could boost the compliance rate for NHS health screening, which has been estimated to be less than 50% . This low-cost opportunistic CVD risk screening based on routine eye checks may help to include ‘hard to reach' populations in CVD risk assessment.
An additional finding that we want to highlight is the fact that Reti-CVD identified high-risk individuals who were otherwise missed by QRISK3. 8.6% of patients identified as high-risk by our new stratification tool had a QRISK3 score between 0 to 5 and 40.1% of them were categorized in the QRISK3 ≥ 5 to < 10% group. This demonstrates the potential of Reti-CVD as an adjunct tool to identify future high-risk individuals who may be left out if CVD risk stratification is dependent solely on QRISK3 scoring. Consequently, when we incorporate only two basic demographic factors of age and gender to Reti-CVD, this model was superior to QRISK3 model across the range of risk thresholds. Moreover, in our explorative analysis in eTable 3, we confirmed that upon conducting a more detailed QRISK3 analysis in the borderline QRISK3 group where individuals were divided based on 0.5 intervals between 7.5 and 10%, more than 1.6% of individuals were classified in the Reti-CVD-high-risk groups in all subgroups. This indicates that Reti-CVD can be used as an independent risk enhancer in this range of QRISK3.
By comparing Reti-CVD to QRISK3, this study demonstrates the potential value and scope of Reti-CVD to improve discernment accuracy in the UK population since QRISK3 is part of the up-to-date guidelines that reflects the demographic profile of the UK Biobank. Study limitations include a possibility of systematic errors in the form of misclassification bias, since hospitalization and mortality data provided by the NHS registers were used for CVD definition. However, a study by Kivimaki et al. has analyzed the validity of and reported good agreement between CVD event predictions and the use of UK National Health Electronic Records, therefore minimizing the likelihood of CVD event misdiagnoses . We also excluded poor quality images during the development and validation of Reti-CVD. The quality of retinal photographs available in clinical settings may vary, and therefore this may pose challenges for real-world implementation.
In conclusion, a DL and retinal photograph-derived new CVD biomarker, Reti-CVD, could effectively stratify CVD risk in the general population. We also confirmed that Reti-CVD has the potential to identify individuals with ≥ 10% 10-year CVD risk who are likely to benefit from preventative CVD interventions. In addition, for people with a QRISK3 score between 7.5 and 10%, Reti-CVD could be used as a risk stratification enhancer for identifying high risk patients for aggressive CVD prevention.
Availability of data and materials
The UK Biobank test dataset was obtained from UK Biobank (application number 68428). Data cannot be shared publicly due to the violation of patient privacy and the absence of informed consent for data sharing.
National Institute for Health and Care Excellence
Coronary artery calcium
Institutional Review Board
SingHealth Centralized Institutional Review Board
National Health Service
Net Reclassification Index
Framingham risk score
Deshmukh M, Liu YC, Rim TH, Venkatraman A, Davidson M, Yu M, et al. Automatic segmentation of corneal deposits from corneal stromal dystrophy images via deep learning. Comput Biol Med. 2021;137:104675.
Ng WY, Zhang S, Wang Z, Ong CJT, Gunasekeran DV, Lim GYS, et al. Updates in deep learning research in ophthalmology. Clin Sci (Lond). 2021;135(20):2357–76.
Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, May M, Brindle P. Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study. BMJ. 2007;335(7611):136.
Cardiovascular disease: risk assessment and reduction, including lipid modification [https://www.nice.org.uk/guidance/cg181/resources]. 2014.
Jones NR, McCormack T, Constanti M, McManus RJ. Diagnosis and management of hypertension in adults: NICE guideline update 2019. Br J Gen Pract. 2020;70(691):90–1.
Pate A, Emsley R, van Staa T. Impact of lowering the risk threshold for statin treatment on statin prescribing: a descriptive study in English primary care. Br J Gen Pract. 2020;70(700):e765–e71.
Betzler BK, Yang HHS, Thakur S, Yu M, Quek TC, Soh ZD, et al. Gender prediction for a multiethnic population via deep learning across different retinal fundus photograph fields: retrospective cross-sectional study. JMIR Med Inform. 2021;9(8):e25165.
Tseng RMWW, Rim TH, Cheung CY, Wong TY. Artificial intelligence using the eye as a biomarker of systemic risk. Artificial Intelligence in Ophthalmology: Springer; 2021. p. 243–55.
Rim TH, Lee G, Kim Y, Tham YC, Lee CJ, Baik SJ, et al. Prediction of systemic biomarkers from retinal photographs: development and validation of deep-learning algorithms. Lancet Digit Health. 2020;2(10):e526–e36.
Rim TH, Lee CJ, Tham Y-C, Cheung N, Yu M, Lee G, et al. Deep-learning-based cardiovascular risk stratification using coronary artery calcium scores predicted from retinal photographs. Lancet Digital Health. 2021;3(5):e306–e16.
Sudlow C, Gallacher J, Allen N, Beral V, Burton P, Danesh J, et al. UK biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS Med. 2015;12(3):e1001779.
Palmer LJ. UK Biobank: bank on it. Lancet. 2007;369(9578):1980–2.
UK Biobank: Protocol for a large-scale prospective epidemiological resource. 2007.
Man REK, Gan AHW, Fenwick EK, Gan ATL, Gupta P, Sabanayagam C, et al. Prevalence, determinants and association of unawareness of diabetes, hypertension and hypercholesterolemia with poor disease control in a multi-ethnic Asian population without cardiovascular disease. Popul Health Metrics. 2019;17(1):17.
UK Factsheet [https://www.bhf.org.uk/-/media/files/research/heart-statistics/bhf-cvd-statistics---ukfactsheet.pdf]. 2022.
Welsh C, Welsh P, Celis-Morales CA, Mark PB, Mackay D, Ghouri N, et al. Glycated hemoglobin, prediabetes, and the links to cardiovascular disease: data from UK Biobank. Diabetes Care. 2020;43(2):440–5.
Li Y, Sperrin M, van Staa T. R package “QRISK3”: an unofficial research purposed implementation of ClinRisk’s QRISK3 algorithm into R. F1000Research. 2020;8:2139.
Pencina MJ, D'Agostino RB Sr, D'Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157–72 discussion 207-12.
Vardas PE, Asselbergs FW, van Smeden M, Friedman P. The year in cardiovascular medicine 2021: digital health and innovation. Eur Heart J. 2022;43(4):271–9.
Zhang L, Yuan M, An Z, Zhao X, Wu H, Li H, et al. Prediction of hypertension, hyperglycemia and dyslipidemia from retinal fundus photographs via deep learning: A cross-sectional study of chronic diseases in central China. PLoS One. 2020;15(5):e0233166.
Poplin R, Varadarajan AV, Blumer K, Liu Y, McConnell MV, Corrado GS, et al. Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning. Nat Biomed Eng. 2018;2(3):158–64.
Gerrits N, Elen B, Craenendonck TV, Triantafyllidou D, Petropoulos IN, Malik RA, et al. Age and sex affect deep learning prediction of cardiometabolic risk factors from retinal images. Sci Rep. 2020;10(1):9432.
Wang JJ, Liew G, Wong TY, Smith W, Klein R, Leeder SR, et al. Retinal vascular calibre and the risk of coronary heart disease-related death. Heart. 2006;92(11):1583–7.
Mueller S, Wintergerst MWM, Falahat P, Holz FG, Schaefer C, Schahab N, et al. Multiple instance learning detects peripheral arterial disease from high-resolution color fundus photography. Sci Rep. 2022;12(1):1389.
Barriada RG, Simó-Servat O, Planas A, Hernández C, Simó R, Masip D. Deep Learning of Retinal Imaging: A Useful Tool for Coronary Artery Calcium Score Prediction in Diabetic Patients. Appl Sci. 2022;12:1401. https://doi.org/10.3390/app12031401.
Son J, Shin JY, Chun EJ, Jung K-H, Park KH, Park SJ. Predicting high coronary artery calcium score from retinal fundus images with deep learning algorithms. Transl Vis Sci Technol. 2020;9(2):28.
Chang J, Ko A, Park SM, Choi S, Kim K, Kim SM, et al. Association of cardiovascular mortality and deep learning-funduscopic atherosclerosis score derived from retinal fundus images. Am J Ophthalmol. 2020;217:121–30.
Arnould L, Guenancia C, Bourredjem A, Binquet C, Gabrielle PH, Eid P, et al. Prediction of cardiovascular parameters with supervised machine learning from singapore “I” vessel assessment and OCT-angiography: a pilot study. Transl Vision Sci Technol. 2021;10(13):20.
Martin A, Saunders CL, Harte E, Griffin SJ, MacLure C, Mant J, et al. Delivery and impact of the NHS health check in the first 8 years: a systematic review. Br J Gen Pract. 2018;68(672):e449–e59.
Kivimäki M, Batty GD, Singh-Manoux A, Britton A, Brunner EJ, Shipley MJ. Validity of cardiovascular disease event ascertainment using linkage to UK hospital records. Epidemiology. 2017;28(5):735–9.
UK Biobank research ethics approval. 2021. [https://www.ukbiobank.ac.uk/learn-more-about-ukbiobank/about-us/ethics].
This project is supported by the Agency for Science, Technology and Research (A*STAR) under its RIE2020 Health and Biomedical Sciences (HBMS) Industry Alignment Fund Pre-Positioning (IAF-PP) Grant No. H20c6a0031. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of the A*STAR.
Ethics approval and consent to participate
The UK Biobank dataset was obtained ethically from UK Biobank (application number 68428). The UK Biobank has also received Research Tissue Bank approval from North West Multi-centre Research Ethics Committee and the Human Tissue Authority license .
Consent for publication
T.H.R. was a former scientific adviser and owns stock of Mediwhale Inc. H.K. and G.L. are employees of Mediwhale Inc. T.H.R. and G.L. hold the following patents that might be affected by this study: 10–2018–0166720(KR), 10–2018–0166721(KR), 10–2018–0166722(KR), 62/694,901(US), 62/715,729(US), and 62/776,345 (US). All other authors declare no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Additional file 1: eFigure 1.
Study flow chart.
Additional file 2: eFigure 2.
Distribution of the QRISK3 score in the UK Biobank.
Additional file 3: eDocument 1.
RetiCAC model updates.
Additional file 4: eFigure 3.
Kaplan-Meier curves according to Reti-CVD and QRISK3.
Additional file 5: eTable 1.
Risk of cardiovascular events by the deep-learning-based retinal CVD biomarker (Reti-CVD).
Additional file 6: eTable 2.
Subgroup analysis in the UK Biobank.
Additional file 7: eFigure 4.
Decision curve analysis.
Additional file 8: eTable 3.
Detailed distribution of Reti-CVD risk groups among those between 7.5% and 10% QRISK3 score.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Tseng, R.M.W.W., Rim, T.H., Shantsila, E. et al. Validation of a deep-learning-based retinal biomarker (Reti-CVD) in the prediction of cardiovascular disease: data from UK Biobank. BMC Med 21, 28 (2023). https://doi.org/10.1186/s12916-022-02684-8
- Artificial intelligence
- Cardiovascular disease
- Deep learning
- Retinal imaging
- Retinal photograph
- Risk stratification
- Risk stratification system
- UK Biobank