Objectives
Risk scoring models (RSMs) are commonly used for estimation of postoperative-mortality
risk in patients undergoing cardiac surgery, but their prediction accuracy may vary
in different populations and clinical situations. The prognostic accuracies of some
RSMs have not yet been fully evaluated in the Australian population. In this retrospective
observational study, our aims were to assess the performance of four contemporary
RSMs, to identify the best RSMs for prediction of postoperative-mortality in the single-centre
cohort, and to determine a statistical threshold for classification of patients with
increased or “higher” mortality risk.
Methods
The study population included patients who underwent cardiac surgery at Liverpool
Hospital between January 2013 and December 2014. Demographic information was collected,
and mortality risks were estimated with the ES2 (EuroSCORE II), STS (Society of Thoracic
Surgeons Score), AS (AusSCORE total) and ASMR (AusSCORE multi-risk) RSMs. (Additive
EuroSCORE) (AES) and LES (logistic EuroSCORE) were included for historical interest.
Discrimination, the ability to stratify patients between mortality and no mortality
outcomes, and calibration, the comparison of risk score estimated and observed outcome
in the population, were evaluated for each RSM, to determine their predictive accuracy
in the study population. Discrimination was assessed by the AUC (area under the receiver
operating characteristic curve), and acceptable calibration by the p-value greater
than 0.05 for the Hosmer–Lemeshow (H-L) test. The best AUCs in contempory models were
compared using the DeLong test. For ES2 and STS risk scores, cut-off points, or thresholds,
for patients at increased risk of mortality were derived using Youden’s J-statistics, calculated from sensitivity and specificity of models in predicting mortality.
Results
From a total study population of 898 patients, 738 had scores for all six RSMs. The
three EuroSCORE risk models and Youden’s J-statistics analysis included the total population. Of the models in contemporary
use, ES2 had higher discrimination (AUC=0.850) in this population than ASMR (AUC=0.767,
p=0.024) and AS (AUC=0.739) and non-significantly higher discrimination than STS (AUC=0.806,
p=0.19). All contemporary models had acceptable calibration but the older LES (H-L
p=0.024) did not. Estimated mortality was closest to observed mortality with the ES2
model. Both AES and LES over predicted mortality. The RSM with the highest discrimination
in isolated coronary artery bypass graft surgery (CAGs) (AUC=0.847), isolated valves
(AUC=0.830), and females (AUC=0.784) was the ES2 model. STS discrimination was highest
in CAGs plus valve procedures (AUC 0.891), and males (STS AUC=0.891). Cut-off points
for risk scores to define increased risk populations were 3.0% for ES2 and 1.7% for
STS. Similar proportions of patients in each RSM (ES2-26% to STS-32%) were defined
as higher risk by the model threshold score depending on type of procedure.
Conclusion
Among RSMs in contemporary use, ES2 and STS showed the best discrimination and acceptable
calibration. Caution is recommended in specific subgroups. Increased mortality risk
score cut-off points could be identified for these two RSMs in this single-centre
cohort.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Heart, Lung and CirculationAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Reliability of risk algorithms in predicting early and late operative outcomes in high-risk patients undergoing aortic valve replacement.The J Thorac Cardiovasc Surg. 2008; 135: 180-187
- Comparison of the EuroSCORE II and Society of Thoracic Surgeons 2008 risk tools.Eur J Cardiothorac Surg. 2013; 44 (discussion 1005): 999-1005
- EuroSCORE II.Eur J Cardiothorac Surg. 2012; 41 (discussion 44-5): 734-744
- The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 1--coronary artery bypass grafting surgery.Ann Thorac Surg. 2009; 88: S2-S22
- The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 2--isolated valve surgery.Ann Thorac Surg. 2009; 88: S23-S42
- The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 3--valve plus coronary artery bypass grafting surgery.Ann Thorac Surg. 2009; 88: S43-S62
- Comparison of 19 pre-operative risk stratification models in open-heart surgery.Eur Heart J. 2006; 27: 867-874
- An Australian risk prediction model for 30-day mortality after isolated coronary artery bypass: the AusSCORE.J Thorac Cardiovasc Surg. 2009; 138: 904-910
- A preoprative risk prediction model for 30-day mortality following cardiac surgery in an Australian cohort.Eur J Cardiothorac Surg. 2010; 37: 1086-1092
- AusSCORE II in predicting 30-day mortality after isolated coronary artery bypass grafting in Australia and New Zealand.J Thorac Cardiovasc Surg. 2014; 148: 1850-1855 e2
- 2018 ESC/EACTS Guidelines on myocardial revascularization.Eur Heart J. 2019; 40: 87-165
- Validation of EuroSCORE II in a modern cohort of patients undergoing cardiac surgery.Eur J Cardiothorac Surg. 2013; 43: 688-694
- Risk stratification for in-hospital mortality after cardiac surgery: external validation of EuroSCORE II in a prospective regional registry.Eur J Cardiothorac Surg. 2014; 46: 840-848
- Society of Thoracic Surgeons score is superior to the EuroSCORE determining mortality in high risk patients undergoing isolated aortic valve replacement.Ann Thorac Surg. 2009; 88 (discussion 74-5): 468-474
- Performance of EuroSCORE II compared to EuroSCORE I in predicting operative and mid-term mortality of patients from a single center after combined coronary artery bypass grafting and aortic valve replacement.Gen Thorac Cardiovasc Surg. 2014; 62: 103-111
- Comparison of four risk scores for contemporary isolated coronary artery bypass grafting.Heart Lung Circ. 2014; 23: 469-474
- Predicted and observed mortality in adult cardiac surgery in the King Abdulaziz Cardiac Center: application of EuroSCORE I and EuroSCORE II.Eur Heart J Suppl. 2014; 16: B55-B58
- Validation of EuroSCORE II in a single centre Australian coholt.Heart Lung Circ. 2014; 23: e41
- Early and late mortality in patients undergoing transcatheter aortic valve implantation: comparison of the novel EuroScore II with established risk scores.Cardiology. 2013; 126: 15-23
- Outcomes of on-pump versus off-pump coronary artery bypass graft surgery in the high risk (AusSCORE > 5).Heart Lung Circ. 2015; 24: 1216-1224
- Reliability of new scores in predicting perioperative mortality after isolated aortic valve surgery: a comparison with the society of thoracic surgeons score and logistic EuroSCORE.Ann Thorac Surg. 2013; 95: 1539-1544
- Accuracy, calibration, and clinical performance of EuroSCORE II and age creatinine, and ejection fraction score in 53,472 isolated coronary artery bypass surgeries in Australia and New Zealand.Heart Lung Circ. 2018; 27: S532
- Agreement between the new EuroSCORE II, the logistic EuroSCORE and the Society of Thoracic Surgeons score: implications for transcatheter aortic vlave implantation.Arch Cardiovasc Dis. 2014; 107: 353-360
- European system for cardiac operative risk evaluation (EuroSCORE).Eur J Cardiothorac Surg. 1999; 16: 9-13
- A call for an evidence-based approach to the Heart Team for patients with severe aortic stenosis.J Am Coll Card. 2015; 65: 1472-1480
- Risk factors for and economic implications of prolonged ventilation after cardiac surgery.J Thorac and Cardiovasc Surg. 2005; 130: 1270-1277
- Economics of psychosocial factors in patients with cardiovascular disease.Prog Cardiovasc Dis. 2013; 55: 563-573
- Comparing areas under two or more correlated receiver operating characteristics curves: a non parametric approach.Biometrics. 1988; 44: 837-845
- ROC-ing along: evaluation and interpretation of receiver operating characteristic curves.Surgery. 2016; 159: 1638-1645
- An index for rating diagnostic tests.Cancer. 1950; 3: 32-35
- Optimal cut-point and its corresponding Youden index to discriminate individuals using pooled blood samples.Epidemiology. 2005; 16: 73-81
- Youden index and optimal cut-point estimated from observations affected by a lower limit of detection.Biom J. 2008; 50: 419-430
- Relationship between the logistic EuroSCORE and the Society of Thoracic Surgeons Predicted Risk of Mortality score in patients implanted with the CoreValve ReValving system--a Bern-Rotterdam Study.Am Heart J. 2010; 159: 323-329
- Validation of the EuroSCORE model in Australia.Eur J Cardiothorac Surg. 2006; 29 (discussion 6): 441-446
- When is urgent really urgent and does it matter? Misclassification of procedural status and implication for risk assessment in cardiac surgery.Heart Lung Circ. 2016; 25: 196-203
- Reply to Letter to the Editor - The Definition of 'urgent' in Australia and New Zealand Society of Cardiac and Thoracic Surgeons database.Heart Lung Circ. 2016; 25: 1141-1142
- Role of EuroSCORE II in predicting long-term outcome after percutaneous catheter intervention for coronary triple vessel disease or left main stenosis.Int J Cardiol. 2013; 168: 3273-3279
- Anatomical and clinical characteristics to guide decision making between coronary artery bypass surgery and percutaneous coronary intervention for individual patients: development and validation of SYNTAX score II.Lancet. 2013; 381: 639-650
- Addition of frailty and disability to cardiac surgery risk scores identifies elderly patients at high risk of mortality or major morbidity.Circ Cardiovasc Qual Outcomes. 2012; 5: 222-228
- The implication of frailty on preoperative risk assessment.Curr Opin Anaesthesiol. 2014; 27: 330-335
- Impact of body mass index and albumin on morbidity and mortality after cardiac surgery.J Thorac Cardiovasc Surg. 1999; 118: 866-873
Article info
Publication history
Published online: October 27, 2021
Accepted:
August 26,
2021
Received in revised form:
August 4,
2021
Received:
December 22,
2020
Identification
Copyright
© 2021 Published by Elsevier B.V. on behalf of Australian and New Zealand Society of Cardiac and Thoracic Surgeons (ANZSCTS) and the Cardiac Society of Australia and New Zealand (CSANZ).
