Skip to main content

Prediction of cardiovascular risk in patients with chronic obstructive pulmonary disease: a study of the National Health and Nutrition Examination Survey database

Abstract

Background

Cardiovascular disease (CVD) is a common comorbidity associated with chronic obstructive pulmonary disease (COPD), but few studies have been conducted to identify CVD risk in COPD patients. This study was to develop a predictive model of CVD in COPD patients based on the National Health and Nutrition Examination Survey (NHANES) database.

Methods

A total of 3,226 COPD patients were retrieved from NHANES 2007–2012, dividing into the training (n = 2351) and testing (n = 895) sets. The prediction models were conducted using the multivariable logistic regression and random forest analyses, respectively. Receiver operating characteristic (ROC) curves, area under the curves (AUC) and internal validation were used to assess the predictive performance of models.

Results

The logistic regression model for predicting the risk of CVD was developed regarding age, gender, body mass index (BMI), high-density lipoprotein (HDL), glycosylated hemoglobin (HbA1c), family history of heart disease, and stayed overnight in the hospital due to illness last year, which the AUC of the internal validation was 0.741. According to the random forest analysis, the important variables-associated with CVD risk were screened including smoking (NNAL and cotinine), HbA1c, HDL, age, gender, diastolic blood pressure, poverty income ratio, BMI, systolic blood pressure, and sedentary activity per day. The AUC of the internal validation was 0.984, indicating the random forest model for predicting the CVD risk in COPD cases was superior to the logistic regression model.

Conclusion

The random forest model performed better predictive effectiveness for the cardiovascular risk among COPD patients, which may be useful for clinicians to guide the clinical practice.

Peer Review reports

Background

Chronic obstructive pulmonary disease (COPD), a common preventable and treatable disease, is characterized by persistent airflow limitation, COPD is frequently associated with increased airway and lung inflammatory responses to noxious particles or gases, [1], and will be the third leading cause of deaths all over the world by 2030 [2]. It occurs in approximately 10% of adults older than 40 years [3]. Disease management is complicated by the presence of comorbidities, which are an important component of COPD [4, 5]. Furthermore, significant comorbidities may have an impact on the incidence and mortality of COPD [6]. The common comorbidities of COPD include cardiovascular disease (CVD), skeletal muscle wasting, and stroke [7,8,9]. Of which, CVD is widely considered to have the greatest impact on COPD patients and is associated with disease progression, clinical outcomes, and mortality [10, 11].

Several mechanisms have been proposed to explain the link between COPD and increased risk of CVD [12,13,14]. COPD patients are at greater risk of CVD compared with age-matched and sex-matched individuals without COPD [7, 15]. In addition, COPD patients with CVD report more dyspnea, poorer quality of life, more frequent hospitalizations, and higher mortality than those with COPD alone [16]. CVD and COPD have similar risk factors, which are frequently coexist, such as aging, history of cigarette smoking, and a sedentary lifestyle [17,18,19]. However, the risk of CVD in most COPD patients has not yet been identified [20]. Predicting CVD risk is of great significance for disease management of COPD, including timely intervention and rational drug use.

In the current study, we assessed the variables of CVD risk in patients with COPD, and developed models that using multivariable logistic regression and random forest analysis to predict the risk of CVD in COPD patients. Also, the performance of these models was investigated with the internal validation.

Methods

Study design and data source

The data were extracted from NHANES (2007–2012) database [21], a cross-sectional survey of the U.S. civilian. Information was collected via household interviews and standardized physical examinations in specially equipped mobile examination centers. A total of 3226 COPD adults aged 40 to 79 years were enrolled in this study, dividing into the training (n = 2351) and testing (n = 895) sets. The approval from the Institutional Review Board of Beijing Friendship Hospital, Capital Medical University was not required because the data accessed from NHANES were freely available.

Measurement of diseases

COPD was confirmed based on the Medical Conditions Questionnaire (MCQ), including “Ever told you had emphysema” (MCQ160G) and “Ever told you had chronic bronchitis” (MCQ160K). The participants would be diagnosed as COPD if one of the two questions were answered yes. In addition, if the subjects underwent two pulmonary function measurements, the mean value of baseline 1st test spirometry-forced expiratory volume in the first 1.0s (SPXNFEV1) and bronchodilator 2nd test spirometry-forced expiratory volume in the first 1.0s (SPXBFEV1) was taken as FEV1, and the mean value of baseline 1st test spirometry-forced vital capacity (SPXNFVC) and bronchodilator 2nd test spirometry- forced vital capacity (SPXBFVC) was taken as FVC. If only one pulmonary function measurement was performed, the ratios of FEV1 (measured by SPXNFEV1) and FVC (measured by SPXNFVC) were calculated. FEV1 was predicted by gender, age and height in the whole population according to different races. The participants would be diagnosed as COPD when the actual FEV1 was less than 80% of the predicted value, and the actual FEV1/FVC was less than 70% of the predicted value.

CVD were determined respectively according to the questions “Ever told you had angina or heart failure” (MCQ160B), “Ever told you had heart attack” (MCQ160E), and “Has a doctor or other health professional ever told you that you had coronary heart disease” (MCQ160C). The individuals would be diagnosed as CVD if one of the three questions were answered yes.

Determinants

Variables were extracted from the NHANES database containing demographic information, health-related characteristics, and healthcare-related characteristics. Demographic information was as follows: age, gender, ethnicity, education, and poverty income ratio. Health-related characteristics included general health, general health compared with last year, body mass index (BMI), smoking, anyone smoking inside the home, total smokers inside the home, cotinine (nicotine metabolites), 4-(methylnitrosamino)-1-(3-pyridyl)-1-butanol (NNAL, nicotine metabolites), sedentary activity per day, systolic blood pressure, diastolic blood pressure, high-density lipoprotein (HDL), glycosylated hemoglobin (HbA1c), family history of heart disease, and asthma. Healthcare-related characteristics covered healthcare place type, number of times received healthcare over past year, stayed overnight in the hospital due to illness last year, and number of overnight stays in the hospital due to illness.

Statistical analysis

All statistical analyses were performed using SAS software (version 9.4) and scikit-learn (version 0.23.1). Scikit-learn (version 0.23.1) is a Python-based open-source machine learning library and used to implement the random forest model. The quantitative data were respectively described as the median and quartile [M (Q1, Q3)] through Mann–Whitney U test. N (%) was used to express the categorical data using \({\upchi }^{2}\) test. Data distribution were adjusted with the sample weight of the Mobile Examination Center (MEC) to deal with the oversample of the data itself. Since the data missing ratio was less than 10%, the mean or mode of the weighted sample was directly used to filling. All COPD cases were divided into the training and testing sets. Variables with significant differences were into the multivariable logistic regression model using the training set, and then internal validation with the testing set were to assess the predictive effectiveness of the CVD risk among COPD patients. Similarly, a random forest model for the risk prediction of CVD was carried out using the training set, and then the testing set was used to internally validate the model performance. P < 0.05 was considered statistically significant.

Results

Baseline characteristics

A total of 3226 COPD cases were included form the NHANES (2007–2012). Of which, 2351 COPD patients identified through the spirometry data were included in the training set, while another 895 COPD patients identified by questionnaire data were included in the testing set. The characteristics of the training set were shown in Table 1. There were 428 patients in COPD & CVD group and 1923 cases in COPD group, respectively. Compared with COPD group, the patient’s age (Z = − 2097.465, P < 0.001), male ratio (χ2 = 13.470, P < 0.001), education (χ2 = 8.905, P = 0.012), and poverty income ratio (Z = − 648.776, P < 0.001) were higher in COPD & CVD group. The overall health status of patients in COPD group was better than that in COPD & CVD group (χ2 = 57.185, P < 0.001), while the BMI (Z = 1084.647, P < 0.001), smoking ratio (χ2 = 16.890, P < 0.001), sedentary activity per day (Z = − 894.344, P < 0.001), systolic blood pressure (Z = 34.157, P < 0.001), HDL (Z = − 1256.720, P < 0.001), HbA1c (Z = 1539.918, P < 0.001), and family history of heart disease ratio (χ2 = 20.298, P < 0.001) were lower in COPD group than those in COPD & CVD group. The number of times received healthcare over past year (χ2 = 53.250, P < 0.001) and number of overnight stays in the hospital due to illness (χ2 = 501.298, P < 0.001) were less in COPD group than that in COPD & CVD group.

Table 1 Univariable analysis between COPD & CVD and COPD groups

The characteristics of cases between COPD & CVD and COPD groups

The proportion of patients with chronic bronchitis or emphysema was shown in Fig. 1. The results indicated that the COPD & CVD group patients had higher proportion of chronic bronchitis (19.45% vs 8.29%, χ2 = 16.689, P < 0.001) and emphysema (21.88% vs 9.96%, χ2 = 15.207, P < 0.001) than those in COPD group. In addition, there was no statistical difference in the distribution of angina, heart attack, heart failure, and coronary between males and females in the COPD & CVD group.

Fig. 1
figure 1

The proportion of patients with chronic bronchitis or emphysema in the COPD & CVD and COPD groups. A Patients with chronic bronchitis; B patients with emphysema

The Logistic regression model for predicting CVD risk in COPD patients

The multivariate Logistic regression analysis was carried out to assess the determinants of CVD risk in COPD cases (Table 2). The results showed that age (OR 1.073, 95% CI 1.054 to 1.092), BMI (OR 1.025, 95% CI 1.003 to 1.048), HbA1c (OR 1.192, 95% CI 1.073 to 1.323), family history of heart disease (OR 2.665, 95% CI 1.79 to 3.967), and stayed overnight in the hospital due to illness last year (OR 2.314, 95% CI 1.543 to 3.551) were determinants for the risk of CVD among COPD patients. In addition, elevated HDL (OR 0.417, 95% CI 0.255 to 0.681) and females (OR 0.537, 95% CI 0.384 to 0.751) were associated with a reduced risk of CVD in COPD patients.

Table 2 Multivariable Logistic regression model for predicting CVD risk in COPD patients

The Logistic regression model was established based on these determinants using the training set. The receiver operating characteristic (ROC) curves of the predictive model were plotted in Fig. 2. The area under curve (AUC) of the model was 1.000. The result of the internal validation with the test set showed that the AUC was 0.741, suggesting the model could be used to predict CVD risk in COPD patients.

Fig. 2
figure 2

Receiver operating characteristic (ROC) curves and area under the curves (AUC) of Logistic regression model on the training set (A) and test set (B)

The random forest model for predicting CVD risk in COPD patients

Totally 1024 decision trees were used in the random forest analysis, the maximum number of sampling features was 4, and the rest were set using default parameter. The important variables-associated with CVD risk were NNAL, HbA1c, HDL, age, gender, diastolic blood pressure, cotinine, poverty income ratio, BMI, systolic blood pressure, and sedentary activity per day. The importance of variables was listed in Fig. 3.

Fig. 3
figure 3

Importance of variables in the random forest model. NNAL-4-(methylnitrosamino)-1-(3-pyridyl)-1-butanol (nicotine metabolites), PIR-poverty income ratio

The ROC curves of the random forest model were displayed in Fig. 4. The AUCs of this model and the internal validation were 1.000 and 0.948, respectively. It was indicated that the random forest model was performed well predictive effectiveness in predicting CVD risk among COPD patients, which may be used to guide the clinical practice.

Fig. 4
figure 4

ROC curves and AUC of random forest model on the training set (A) and test set (B)

Discussion

Two predictive models based on the NHANES database were carried out to identify CVD risk in COPD patients. The results showed that the AUC was 0.741 in the logistic regression model, and age, gender, BMI, HDL, HbA1c, family history of heart disease, and stayed overnight in the hospital due to illness last year were the determinants for the risk of CVD in COPD patients. The elevated HDL and females were associated with a reduced risk of CVD in COPD patients. In addition, the AUC of the random forest model was 0.948, and NNAL, HbA1c, HDL, age, gender, diastolic blood pressure, cotinine, poverty income ratio, BMI, systolic blood pressure, and sedentary activity per day were important variables-associated with CVD risk. The predictive efficiency of the random forest model was superior to the logistic regression model, indicating that the random forest model may perform better effectiveness in predicting the cardiovascular risk among cases with COPD.

Previous studies have reported a high incidence of COPD patients with CVD, leading to poor quality of life, dyspnea, low exercise tolerance and high risk of hospitalization [15, 22]. To reduce the risk of poor prognosis in COPD patients, it is necessary to effectively identify CVD risk. It was reported that multiple major risk factors-associated with CVD were found in COPD patients [23]. In our study, age, gender, BMI, HDL, HbA1c, were the important determinants of CVD risk in COPD patients in the logistic regression model. In the random forest model, metabolites associated with smoking (NNAL and cotinine), HbA1c, HDL, age, gender were the important variables of CVD risk in COPD patients. Variables such as age, gender, and HDL were important for CVD risk in COPD patients in both models. The study of Cazzola et al. indicated that patients > 35 years had higher odds ratio of simultaneous CVD and COPD [24]. Gunay et al. found that HDL level of COPD patients was significantly lower than that of healthy subjects [25]. Our results showed that higher age, males, and lower HDL level were associated with an increased risk of CVD in COPD patients.

To our best knowledge, the identification of cardiovascular risk in patients with COPD is currently unclear [20]. Few previous studies have developed a clinical predictive model that can be used to identify CVD risk in COPD patients. An early study showed that less than one-third of COPD patients were diagnosed with CVD by electrocardiographic images [26]. A simple, safe and effective method was needed to identify CVD risk in COPD patients. One recent study reported a model for predicting cardiovascular risk in patients with COPD, and their overall cardiovascular risk model C-statistic was 0.689 [27]. Our study provided two predictive models to identify CVD risk in COPD patients, especially the random forest model had a better predictive effect with an AUC of 0.948. The study of Adamson et al. indicated that cardiac troponin I concentrations were a specific and major biomarker of CVD risk in COPD patients [28]. In further studies, some important biomarkers can be included to improve the prediction effect of the model.

At present, the mechanism link between COPD and CVD is complex, multifactorial, and not yet fully understood [29]. Models that predicting CVD risk in COPD patients play an important role. The random forest model performed an excellent predictive effect and was simple on predicting CVD risk in COPD patients, which has the potential to be further applied to clinical practice. In future studies, more effective predictive model will be established to identify CVD risk among COPD patients. However, our study has some limitations. First, the diagnosis criteria of some COPD patients and all CVD patients relied on questionnaire data, which may affect the results of the model. However, in the random forest model, the model’s AUC had a small difference between the training set (COPD identified by spirometry data) and the test set (COPD identified through questionnaire data), indicating that the difference between COPD patients identified by questionnaire data and spirometry data was small. Second, an independent external validation study would be a more rigorous test and should be conducted. Third, the samples were mainly American, and there would be selection bias.

Conclusion

This study provided two predictive models to identify CVD risk in COPD patients. The AUC was 0.741 in the logistic regression model, and age, gender, BMI, HDL, HbA1c, family history of heart disease, and stayed overnight in the hospital due to illness last year were the influencing factors for CVD in COPD patients. In the random forest model, the AUC was 0.948, and NNAL, HbA1c, HDL, age, gender, diastolic blood pressure, cotinine, poverty income ratio, BMI, systolic blood pressure, and sedentary activity per day were important variables-associated with CVD risk. The random forest model performed an excellent predictive effect on predicting CVD risk in COPD patients, which has the potential to be further applied to clinical practice.

Abbreviations

CVD:

Cardiovascular disease

COPD:

Chronic obstructive pulmonary disease

ROC:

Receiver operating characteristic

AUC:

Area under the curves

BMI:

Body mass index

HDL:

High-density lipoprotein

HbA1c:

Glycosylated hemoglobin

MCQ:

Medical Conditions Questionnaire

FEV1:

Forced expiratory volume in one second

FVC:

Forced vital capacity

References

  1. Global strategy for the diagnosis, management, and prevention of chronic obstructive pulmonary disease (2020 report). Global Initiative for Chronic Obstructive Lung Disease (GOLD) https://www.goldcopdorg/gold-reports/ 2020.

  2. Mathers CD, Loncar D. Projections of global mortality and burden of disease from 2002 to 2030. PLoS Med. 2006;3:e442.

    Article  Google Scholar 

  3. Rabe KF, Watz H. Chronic obstructive pulmonary disease. Lancet. 2017;389:1931–40.

    Article  Google Scholar 

  4. Soriano J, Lamprecht B. Chronic obstructive pulmonary disease: a worldwide problem. Med Clin North Am. 2012;96:671–80.

    Article  Google Scholar 

  5. Celli B, MacNee W. Standards for the diagnosis and treatment of patients with COPD: a summary of the ATS/ERS position paper. Eur Respir J. 2004;23:932–46.

    Article  CAS  Google Scholar 

  6. Lozano R, Naghavi M, Foreman K, Lim S, Shibuya K, Aboyans V, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380:2095–128.

    Article  Google Scholar 

  7. Curkendall S, DeLuise C, Jones J, Lanes S, Stang M, Goehring E, et al. Cardiovascular disease in patients with chronic obstructive pulmonary disease, Saskatchewan Canada cardiovascular disease in COPD patients. Ann Epidemiol. 2006;16:63–70.

    Article  Google Scholar 

  8. Magnussen H, Disse B, Rodriguez-Roisin R, Kirsten A, Watz H, Tetzlaff K, et al. Withdrawal of inhaled glucocorticoids and exacerbations of COPD. N Engl J Med. 2014;371:1285–94.

    Article  Google Scholar 

  9. Perera W, Hurst J, Wilkinson T, Sapsford R, Müllerova H, Donaldson G, et al. Inflammatory changes, recovery and recurrence at COPD exacerbation. Eur Respir J. 2007;29:527–34.

    Article  CAS  Google Scholar 

  10. Vestbo J, Hurd S, Agustí A, Jones P, Vogelmeier C, Anzueto A, et al. Global strategy for the diagnosis, management, and prevention of chronic obstructive pulmonary disease: GOLD executive summary. Am J Respir Crit Care Med. 2013;187:347–65.

    Article  CAS  Google Scholar 

  11. Patel A, Hurst J. Extrapulmonary comorbidities in chronic obstructive pulmonary disease: state of the art. Expert Rev Respir Med. 2011;5:647–62.

    Article  Google Scholar 

  12. Sin D, Man S. Why are patients with chronic obstructive pulmonary disease at increased risk of cardiovascular diseases? The potential role of systemic inflammation in chronic obstructive pulmonary disease. Circulation. 2003;107:1514–9.

    Article  Google Scholar 

  13. Maclay J, McAllister D, Mills N, Paterson F, Ludlam C, Drost E, et al. Vascular dysfunction in chronic obstructive pulmonary disease. Am J Respir Crit Care Med. 2009;180:513–20.

    Article  Google Scholar 

  14. Watz H, Waschki B, Meyer T, Magnussen H. Physical activity in patients with COPD. Eur Respir J. 2009;33:262–72.

    Article  CAS  Google Scholar 

  15. Mannino D, Thorn D, Swensen A, Holguin F. Prevalence and outcomes of diabetes, hypertension and cardiovascular disease in COPD. Eur Respir J. 2008;32:962–9.

    Article  CAS  Google Scholar 

  16. Wang L, Zhu Y, Cui J, Yin K, Liu S, Gao Y. Subclinical atherosclerosis risk markers in patients with chronic obstructive pulmonary disease: a systematic review and meta-analysis. Respir Med. 2017;123:18–27.

    Article  Google Scholar 

  17. Divo M, Cote C, de Torres J, Casanova C, Marin J, Pinto-Plata V, et al. Comorbidities and risk of mortality in patients with chronic obstructive pulmonary disease. Am J Respir Crit Care Med. 2012;186:155–61.

    Article  Google Scholar 

  18. Smith M, Wrobel J. Epidemiology and clinical impact of major comorbidities in patients with COPD. Int J Chron Obstruct Pulmon Dis. 2014;9:871–88.

    Article  Google Scholar 

  19. Bhatt S, Dransfield M. AECOPD: Acute exacerbations of chronic obstructive cardiopulmonary disease? Am J Respir Crit Care Med. 2013;188:1046–8.

    Article  Google Scholar 

  20. Morgan A, Zakeri R, Quint J. Defining the relationship between COPD and CVD: what are the implications for clinical practice? Ther Adv Respir Dis. 2018;12:1753465817750524.

    Article  Google Scholar 

  21. National Health and Nutrition Examination Survey (2007–2012). Centers for Disease Control and Prevention https://www.cdcgov/nchs/nhanes/indexhtm?CDC_AA_refVal=https%3A%2F%2Fwwwcdcgov%2Fnchs%2Fnhaneshtm 2014.

  22. Miller J, Edwards L, Agustí A, Bakke P, Calverley P, Celli B, et al. Comorbidity, systemic inflammation and outcomes in the ECLIPSE cohort. Respir Med. 2013;107:1376–84.

    Article  Google Scholar 

  23. Leong P, Macdonald M, Ko B, Bardin P. Coexisting chronic obstructive pulmonary disease and cardiovascular disease in clinical practice: a diagnostic and therapeutic challenge. Med J Aust. 2019;210:417–23.

    Article  Google Scholar 

  24. Cazzola M, Calzetta L, Rinaldi B, Page C, Rosano G, Rogliani P, et al. Management of chronic obstructive pulmonary disease in patients with cardiovascular diseases. Drugs. 2017;77:721–32.

    Article  CAS  Google Scholar 

  25. Gunay S, Sariaydin M, Acay A. New predictor of atherosclerosis in subjects with COPD: atherogenic indices. Respir Care. 2016;61:1481–7.

    Article  Google Scholar 

  26. Brekke P, Omland T, Smith P, Søyseth V. Underdiagnosis of myocardial infarction in COPD - Cardiac Infarction Injury Score (CIIS) in patients hospitalised for COPD exacerbation. Respir Med. 2008;102:1243–7.

    Article  Google Scholar 

  27. Fermont J, Fisk M, Bolton C, MacNee W, Cockcroft J, Fuld J, et al. Cardiovascular risk prediction using physical performance measures in COPD: results from a multicentre observational study. BMJ Open. 2020;10:e038360.

    Article  Google Scholar 

  28. Adamson PD, Anderson JA, Brook RD, Calverley PMA, Celli BR, Cowans NJ, et al. Cardiac Troponin I and cardiovascular risk in patients with chronic obstructive pulmonary disease. J Am Coll Cardiol. 2018;72:1126–37.

    Article  CAS  Google Scholar 

  29. Decramer M, Janssens W. Chronic obstructive pulmonary disease and comorbidities. Lancet. 2013;1:73–83.

    Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

Not applicable.

Author information

Authors and Affiliations

Authors

Contributions

YS and YH designed this study. JZ extracted the data from the NHANES database. Data analysis was conducted by YS and JZ. YS and YH wrote the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Yingshuo Huang.

Ethics declarations

Ethics approval and consent to participate

The approval from the Institutional Review Board of Beijing Friendship Hospital, Capital Medical University was not required because the data accessed from NHANES were freely available.

Consent for publication

Not applicable.

Availability of data and materials

The datasets generated and/or analyzed during the current study are available in the NHANES, https://www.cdc.gov/nchs/nhanes/index.html.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, Y., Zhang, J. & Huang, Y. Prediction of cardiovascular risk in patients with chronic obstructive pulmonary disease: a study of the National Health and Nutrition Examination Survey database. BMC Cardiovasc Disord 21, 417 (2021). https://0-doi-org.brum.beds.ac.uk/10.1186/s12872-021-02225-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s12872-021-02225-w

Keywords