Advertisement
Heart, Lung and Circulation
Original Article| Volume 29, ISSUE 3, P374-383, March 2020

Development and Use of Prediction Models for Classification of Cardiovascular Risk of Remote Indigenous Australians

Open AccessPublished:February 16, 2019DOI:https://doi.org/10.1016/j.hlc.2019.02.005

      Background

      Cardiovascular disease (CVD) is the leading cause of death for Indigenous Australians. There is widespread belief that current tools have deficiencies for assessing CVD risk in this high-risk population. We sought to develop a 5-year CVD risk score using a wide range of known risk factors to further improve CVD risk prediction in this population.

      Methods

      We used clinical and demographic information on Indigenous people aged between 30 and 74 years without a history of CVD events who participated in the Well Person’s Health Check (WPHC), a community-based survey. Baseline assessments were conducted between 1998 and 2000, and data were linked to administrative hospitalisation and death records for identification of CVD events. We used Cox proportional hazard models to estimate the 5-year CVD risk, and the Harrell’s c-statistic and the modified Hosmer-Lemeshow (mH-L) χ2 statistic to assess the model discrimination and calibration, respectively.

      Results

      The study sample consisted of 1,583 individuals (48.1% male; mean age 45.0 year). The risk score consisted of sex, age, systolic blood pressure, diabetes mellitus, waist circumference, triglycerides, and albumin creatinine ratio. The bias-corrected c-statistic was 0.72 and the bias-corrected mH-L χ2 statistic was 12.01 (p-value, 0.212), indicating good discrimination and calibration, respectively. Using our risk score, the CVD risk of the Indigenous Australians could be stratified to a greater degree compared to a recalibrated Framingham risk score.

      Conclusions

      A seven-factor risk score could satisfactorily stratify 5-year risk of CVD in an Indigenous Australian cohort. These findings inform future research targeting CVD risk in Indigenous Australians.

      Keywords

      Introduction

      Early prevention of cardiovascular disease (CVD) is recognised to be crucial for indigenous populations given their substantially higher CVD risk compared to non-indigenous people [
      • International Diabetes Federation
      Expert meeting on indigenous peoples, diabetes and development report.
      ]. The excess CVD risk among Indigenous Australians is largely due to the very high prevalence of traditional CVD risk factors such as hypertension and diabetes. In addition, other highly prevalent risk factors such as albuminuria have also been shown to be associated with CVD in Indigenous Australians [
      • McDermott R.A.
      • McCulloch B.
      • Li M.
      Glycaemia and albuminuria as predictors of coronary heart disease in Aboriginal and Torres Strait Islander adults: a north Queensland cohort.
      ,
      • Barr E.
      • Cunningham J.
      • Tatipata S.
      • Dunbar T.
      • Kangaharan N.
      • Guthridge S.
      • et al.
      Associations of mortality and cardiovascular disease risks with diabetes and albuminuria in urban Indigenous Australians: the DRUID follow-up study.
      ].
      In CVD primary prevention, absolute CVD risk assessment is an important component that has been widely used and recommended [
      • National Vascular Disease Prevention Alliance
      Guidelines for the management of absolute cardiovascular disease risk.
      ,
      • Goff D.C.
      • Lloyd-Jones D.M.
      • Bennett G.
      • Coady S.
      • D’agostino R.B.
      • Gibbons R.
      • et al.
      ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines.
      ]. This helps to ensure that higher-risk individuals are appropriately targeted for initiation of primary prevention strategies [
      • Reath J.
      • Brown N.
      Managing cardiovascular disease in Aboriginal and Torres Strait Islander people.
      ]. While the annual CVD risk assessment using a validated absolute risk algorithm has been recommended for Indigenous Australians [
      • Brown A.
      • O’Shea R.L.
      • Mott K.
      • McBride K.F.
      • Lawson T.
      • Jennings G.L.
      Essential service standards for equitable national cardiovascular care for Aboriginal and Torres Strait Islander people.
      ], there is widespread belief that current tools have deficiencies in assessing CVD risk in this high risk population [
      • Reath J.
      • Brown N.
      Managing cardiovascular disease in Aboriginal and Torres Strait Islander people.
      ,
      • Chan A.
      Making the case for a more accurate cardiovascular disease risk assessment tool for Indigenous Australians.
      ,
      • Vasant B.R.
      • Matthews V.
      • Burgess C.P.
      • Connors C.M.
      • Bailie R.S.
      Wide variation in absolute cardiovascular risk assessment in Aboriginal and Torres strait Islander people with type 2 diabetes.
      ]. Several studies have shown that the equations derived from the Framingham study underestimate the CVD risk of Indigenous Australians [
      • Wang Z.
      • Hoy W.E.
      Is the Framingham coronary heart disease absolute risk function applicable to Aboriginal people?.
      ,
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      ]. One explanation for this underestimation may be that certain CVD risk factors for Indigenous people such as albuminuria and waist circumference are not included in the Framingham CVD equations [
      • McDermott R.A.
      • McCulloch B.
      • Li M.
      Glycaemia and albuminuria as predictors of coronary heart disease in Aboriginal and Torres Strait Islander adults: a north Queensland cohort.
      ,
      • Barr E.
      • Cunningham J.
      • Tatipata S.
      • Dunbar T.
      • Kangaharan N.
      • Guthridge S.
      • et al.
      Associations of mortality and cardiovascular disease risks with diabetes and albuminuria in urban Indigenous Australians: the DRUID follow-up study.
      ,
      • Wang Z.
      • Hoy W.
      Waist circumference, body mass index, hip circumference and waist-to-hip ratio as predictors of cardiovascular disease in Aboriginal people.
      ,
      • Adegbija O.
      • Hoy W.
      • Wang Z.
      Prediction of cardiovascular disease risk using waist circumference among Aboriginals in a remote Australian community.
      ]. While it is possible to recalibrate the Framingham models to correct for this underestimation [
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      ], this does not account for a broader range of risk factors which are potentially useful in further stratifying risk.
      On the other hand, there are also circumstances in which it is useful to stratify CVD risk using a restricted set of available risk factors [
      • Modesti P.A.
      • Agostoni P.
      • Agyemang C.
      • Basu S.
      • Benetos A.
      • Cappuccio F.P.
      • et al.
      Cardiovascular risk assessment in low-resource settings: a consensus document of the European Society of Hypertension Working Group on Hypertension and Cardiovascular Risk in Low Resource Settings.
      ]. For example, sometimes Indigenous participants were excluded from laboratory tests according to different rules [
      • Rowley K.G.
      • Thorpe A.H.
      Research, information and consent for the Australian Health Survey: a separate standard for Indigenous people?.
      ]. When information on traditional risk factors is not available, a standard approach for risk stratification is to produce a model or risk chart based on a reduced set of factors [
      • World Health Organization
      Prevention of cardiovascular disease : guidelines for assessment and management of cardiovascular risk.
      ].
      In the present study, we aimed to develop and validate prediction models for 5-year risk of CVD using data from an Indigenous Australian cohort. We developed two models: a primary model that included any relevant factors and a reduced information model that does not contain laboratory variables. After internally validating these models, we used them to classify the study cohort as groups with high, moderate and low CVD risks alongside classification based on the Australian guidelines for CVD prevention [
      • National Vascular Disease Prevention Alliance
      Guidelines for the management of absolute cardiovascular disease risk.
      ] and the recalibrated 2008 Framingham equation [
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      ], and compared the average risks in each predicted category with the observed risks. Then, we examined the extent to which the Australian guidelines may misclassify Indigenous Australians with and without high 5-year CVD risk.

      Methods

      Study Cohort

      The cohort used in the present study was obtained from the Well Person’s Health Check (WPHC) study [
      • Miller G.
      • McDermott R.
      • McCulloch B.
      • Leonard D.
      • Arabena K.
      • Muller R.
      The Well Person’s Health Check: a population screening program in indigenous communities in north Queensland.
      ] and has been described in detail in the study by Hua et al. [
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      ]. Briefly, the WPHC was conducted between 1998 and 2000 and included 3,508 people in 26 rural and remote Indigenous communities in North Queensland. For our analysis, we included Indigenous men and women aged between 30 and 74 years who could be linked to hospitalisation and death records for identification of the CVD events occurring after the study entry. We excluded people with a history of CVD events. In contrast to the study by Hua et al. [
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      ], we did not exclude people with missing risk factors at baseline. Instead, we used imputations to replace the missing values with the estimates. As a result, the study cohort consisted of 1,583 individuals (see supplementary document - Figure S1 for a flow chart of the cohort selection).

      Outcome Events and Follow-Up Time

      We used the definition of CVD from the Framingham Heart Study for our outcome events, which included a composite of coronary heart disease (coronary insufficiency, myocardial infarction, and angina pectoris), cerebrovascular events (ischaemic stroke, haemorrhagic stroke and transient ischaemic attack), congestive heart failure, peripheral vascular disease, and coronary deaths [
      • D’Agostino R.B.
      • Vasan R.S.
      • Pencina M.J.
      • Wolf P.A.
      • Cobain M.
      • Massaro J.M.
      • et al.
      General cardiovascular risk profile for use in primary care — the Framingham Heart Study.
      ]. These events were identified using the International Classification of Diseases and procedure codes (versions ICD-9-CM and ICD-10-AM; for details, see Hua et al.) [
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      ]. The follow-up time was calculated as the duration between baseline measurement and hospitalisation of a CVD event, coronary death, non-coronary death or last date of data collection (1 December 2014), whichever came first. To maintain alignment with the CVD risk calculators recommended by the Australian guidelines for prevention of CVD [
      • National Vascular Disease Prevention Alliance
      Guidelines for the management of absolute cardiovascular disease risk.
      ], a 5-year follow-up period was generated for all individuals. This was done by considering an individual as right censored if there was no recorded CVD event within the 5-year follow-up period.

      Statistical Analysis

      Missing Data and Multiple Imputation

      The percentage of observations with missing data ranged from 0% to 15.8% depending on variables recorded in the data set. We used multiple imputation [
      • Rubin D.
      Multiple imputation for nonresponse in surveys.
      ] to replace missing data with appropriate estimates (see detailed methods in supplementary document - section 2).

      Model Selection

      We used Cox proportional hazard models to estimate the 5-year CVD risk. Two models were developed: (i) the primary model with variables selected from risk factors included in the Framingham CVD models [
      • D’Agostino R.B.
      • Vasan R.S.
      • Pencina M.J.
      • Wolf P.A.
      • Cobain M.
      • Massaro J.M.
      • et al.
      General cardiovascular risk profile for use in primary care — the Framingham Heart Study.
      ] as well as other factors such as waist circumference [
      • Wang Z.
      • Hoy W.
      Waist circumference, body mass index, hip circumference and waist-to-hip ratio as predictors of cardiovascular disease in Aboriginal people.
      ] and albumin-to-creatinine ratio (ACR) [
      • McDermott R.A.
      • McCulloch B.
      • Li M.
      Glycaemia and albuminuria as predictors of coronary heart disease in Aboriginal and Torres Strait Islander adults: a north Queensland cohort.
      ,
      • Barr E.
      • Cunningham J.
      • Tatipata S.
      • Dunbar T.
      • Kangaharan N.
      • Guthridge S.
      • et al.
      Associations of mortality and cardiovascular disease risks with diabetes and albuminuria in urban Indigenous Australians: the DRUID follow-up study.
      ,
      • Wang Z.
      • Hoy W.E.
      Albuminuria and incident coronary heart disease in Australian Aboriginal people.
      ] which have been shown to be associated with CVD risk in Indigenous populations; and (ii) the reduced information model that excluded risk factors requiring laboratory measurements. Initial covariates considered in the primary model included sex, age, total cholesterol, high-density lipoprotein (HDL) cholesterol, low-density lipoprotein (LDL) cholesterol, total cholesterol to HDL ratio, triglycerides, systolic blood pressure (SBP), diastolic blood pressure (DBP), current smoking, diabetes status, fasting blood glucose, body mass index (BMI), waist circumference and ACR (see supplementary document - section 3 for methods of risk factor measurements). Of note, the diabetes condition was identified based on the patient self-reports and confirmed by the primary care medical records. In the reduced information model, only sex, age, SBP, DBP, current smoking, diabetes status, BMI and waist circumference were considered. To select the optimal set of predictors, we performed both stepwise and manual procedures, and included variables with a p-value smaller than 0.15 in the final models (see supplementary document - section 4 for detailed methods). We used the scaled Schoenfeld residuals [
      • Grambsch P.M.
      • Therneau T.M.
      Proportional hazards tests and diagnostics based on weighted residuals.
      ] to test the proportional hazard assumption.

      Risk Equation Development and Performance Assessment

      The equation for 5-year CVD risk based on the Cox proportional hazard model takes the following form [
      • Woodward M.
      Epidemiology: study design and data analysis.
      ]:
      p=1-S(5)expi=1nβˆi(xi-x¯i)


      where p is the 5-year CVD risk, S(5) is the survival rate at 5 years of a person with mean values of risk factors, βˆi is the estimated coefficient for the i-th risk factor, and xi and x¯i are the actual and mean values, respectively, of the i-th risk factor. We estimated S5 and βˆi and their 95% CIs by fitting the final model to each of the imputed data sets and combined the estimates using Rubin’s rule (see supplementary document - section 5 for detailed methods).
      We used the Harrell’s c-statistic [
      • Harrell F.E.
      • Lee K.L.
      • Mark D.B.
      Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors.
      ] to evaluate the model discrimination, i.e. the model’s ability to separate persons with events from those without events. The c-statistic is defined as the proportion of all possible pairs of persons, at least one of whom has an event, in which the predicted survival time is larger for the person who lived longer. A c index of 0.7 or larger indicates good discrimination.
      To quantify the model calibration, i.e. the agreement between the observed and predicted risks, we used the modified Hosmer-Lemeshow (mH-L) χ2 statistic proposed by D’Agostino and Nam for survival models [
      • D’Agostino R.
      • Nam B.-H.
      Evaluation of the performance of survival analysis models: discrimination and calibration measures.
      ]. The participants were divided into deciles based on their predicted CVD risks, and the Kaplan-Meier estimate for failure within each decile was compared with the mean predicted risk in the same decile. A mH-L χ2 statistic larger than 20 indicates a poor calibration [
      • D’Agostino Sr., R.B.
      • Grundy S.
      • Sullivan L.M.
      • Wilson P.
      • CRP Group
      Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation.
      ].
      To internally validate the models, we calculated the apparent c and mH-L χ2 statistics and corrected these for the optimism that might arise from overfitting using the algorithm developed by Harrell [
      • Harrell F.E.
      • Lee K.L.
      • Mark D.B.
      Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors.
      ] (see supplementary document - section 6 for detailed methods). The corrected c and mH-L χ2 statistics were shown to represent nearly unbiased assessment of the model performance [
      • Harrell F.E.
      • Lee K.L.
      • Mark D.B.
      Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors.
      ].

      Risk Classification

      We classified 1,583 individuals as either a high (>15%), moderate (10–15%) or low 5-year CVD risk (<10%) using the algorithm provided by the Australian guidelines for CVD prevention [
      • National Vascular Disease Prevention Alliance
      Guidelines for the management of absolute cardiovascular disease risk.
      ], the recalibrated 2008 Framingham equation [
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      ] and the new primary and reduced information equations, then computed mean risks and their 95% CIs within each category using Rubin’s rule [
      • Rubin D.
      Multiple imputation for nonresponse in surveys.
      ,
      • Rubin D.B.
      • Schenker N.
      Multiple imputation for interval estimation from simple random samples with ignorable nonresponse.
      ]. We used the Kaplan-Meier estimator to compute 5-year CVD risks of people within the same category in different imputed data sets and combined them to estimate the observed risks. The Australian guidelines automatically classify people with one or more of the following conditions as high-risk people: diabetes and age >60 years, diabetes with microalbuminuria, moderate or severe chronic kidney disease, a history of familial hypercholesterolaemia, a SBP ≥ 180 mmHg, serum total cholesterol >7.5 mmol/L, or age >74. For people not automatically classified as high risk, the 1991 Framingham equation [
      • Anderson K.
      • Odell P.
      • Wilson P.
      • Kannel W.
      Cardiovascular disease risk profiles.
      ] is recommended by the Australian guidelines for calculating the absolute risks, which are used to classify these people into high (>15%), moderate (10–15%) and low (<10%) risks. Therefore, people classified as high risk by the guidelines may have or not have any of the above-mentioned conditions. We used the same cut-off points for risk categories as those in the guidelines to classify people using our new equations. Because the Australian guidelines do not recommend any equation for the calculation of the actual risk of those automatically classified as high risk, we used the 1991 Framingham equation [
      • Anderson K.
      • Odell P.
      • Wilson P.
      • Kannel W.
      Cardiovascular disease risk profiles.
      ] to examine their absolute risks in reference to the observed risks and the risks predicted by our new models and the recalibrated 2008 Framingham model.
      To examine the potential misclassification of 5-year CVD risk using the Australian guidelines, we calculated the proportions of people classified as high risk and medium-low risk by the Australian guidelines, and within each of these groups determined the proportions of people with a contrasting risk category based on our new models.

      Results

      Of 1,583 people, 142 developed CVD within 5 years after the first screening. We found that sex, age, SBP, waist circumference, diabetes status, triglycerides and ACR ratio were associated with 5-year CVD risk. Table 1 provides summary statistics of these risk factors and the 5-year survival rate of an average person. Current smoking and cholesterol-related variables did not significantly contribute to the prediction (p-values >0.5; see supplementary document - section 7 for coefficients of all variables included in the full model). Table 2 shows the regression coefficients and hazard ratios (HRs) for CVD events associated with the risk factors in the primary and reduced information models. The proportional hazard assumption was satisfied in fitting the models to any of the imputed data sets.
      Table 1Baseline risk factors and 5-year survival rate of the study cohort (N = 1,583).
      Risk factorValue
      Pooled values using Rubin’s rule [19] from estimates for 20 imputed data sets.
      Men, % (SE)
      Risk factors in the primary model.
      Risk factors in the reduced information model.
      48.1 (1.3)
      Age, mean (SE)
      Risk factors in the primary model.
      Risk factors in the reduced information model.
      45.0 (0.3)
      Systolic BP, mean (SE), mmHg
      Risk factors in the primary model.
      Risk factors in the reduced information model.
      134.5 (0.5)
      Diastolic BP, mean (SE), mmHg75.6 (0.3)
      Current smoking, % (SE)53.4 (1.2)
      BMI, mean (SE), kg/m228.8 (0.18)
      Waist circumference, mean (SE), cm
      Risk factors in the primary model.
      Risk factors in the reduced information model.
      99.2 (0.4)
      Diabetes, % (SE)
      Risk factors in the primary model.
      Risk factors in the reduced information model.
      23.2 (1.1)
      Fasting blood glucose, mean (SE), mg/dL114.4 (1.4)
      Total cholesterol, mean (SE), mg/dL198.9 (1.0)
      HDL cholesterol, mean (SE), mg/dL44.5 (0.3)
      LDL cholesterol, mean (SE), mg/dL121.2 (0.9)
      Total cholesterol:HDL ratio4.7 (0.04)
      Triglycerides, mean (SE), mg/dL
      Risk factors in the primary model.
      179.8 (3.7)
      ACR, mean (SE)
      Risk factors in the primary model.
      19.2 (1.6)
      Underlying 5-year survival rate
      Estimated using the fitted Cox proportional hazard models for a person with mean risk factors.
       Primary equation0.9298
       Reduced information equation0.9344
      Abbreviations: ACR, Albumin creatinine ratio; BMI, body mass index; BP, blood pressure; HDL, high-density lipoprotein; LDL, low-density lipoprotein; SE, standard deviation of the mean.
      * Pooled values using Rubin’s rule
      • Rubin D.
      Multiple imputation for nonresponse in surveys.
      from estimates for 20 imputed data sets.
      Risk factors in the primary model.
      Risk factors in the reduced information model.
      § Estimated using the fitted Cox proportional hazard models for a person with mean risk factors.
      Table 2Estimated coefficients and hazard ratios from Cox proportional hazard models.
      VariablePrimary modelReduced information model
      Coefficient (p-value)Hazard ratioCoefficient (p-value)Hazard ratio
      [95% CI][95% CI][95% CI][95% CI]
      Men0.272 (0.074)1.3280.261 (0.076)1.312
      [−0.029, 0.573][0.925, 1.731][−0.031, 0.552][0.927, 1.697]
      Age, years0.050 (<0.001)1.0520.048 (<0.001)1.050
      [0.038, 0.063][1.039, 1.065][0.036, 0.061][1.037, 1.062]
      Systolic BP, mmHg0.010 (0.012)1.0100.014 (<0.001)1.014
      [0.002, 0.018][1.002, 1.018][0.006, 0.022][1.006, 1.022]
      Waist circumference, cm0.009 (0.110)1.0100.009 (0.09)1.010
      [-0.002, 0.020][0.999, 1.020][−0.001, 0.020][0.998, 1.020]
      Diabetes
      Self-reported diabetes confirmed by the primary care medical records, used in both primary and reduced information models.
      0.268 (0.104)1.3250.405 (0.009)1.517
      [−0.015, 0.592][0.974, 1.755][0.101, 0.709][1.055, 1.980]
      Triglycerides, mg/dL0.001 (0.142)1.001
      [0.000, 0.001][1.000, 1.001]
      ACR0.003 (0.001)1.003
      [0.001, 0.005][1.001, 1.004]
      Abbreviations: ACR, albumin creatinine ratio; BP, blood pressure; CI, confidence interval.
      * Self-reported diabetes confirmed by the primary care medical records, used in both primary and reduced information models.
      Table 3 provides performance indices of our new models and the recalibrated 2008 Framingham model [
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      ]. The bias-corrected c-statistics of the primary and reduced information models were 0.722 and 0.719, respectively, indicating good discrimination. The bias-corrected mH-L χ2 statistic (p-values) of the primary and reduced information models were 12.01 (0.212) and 12.49 (0.188), respectively, indicating good calibration. Compared to the recalibrated 2008 Framingham model [
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      ], which had a bias-corrected c-statistic of 0.727 and a bias-corrected mH-L χ2 statistic (p-value) of 29.47 (0.0005), our new models perform similarly in terms of discrimination, but far better in terms of calibration. The observed and predicted risks in deciles of the primary and reduced information models were similar (Figure 1).
      Table 3Performance of the new models and the recalibrated 2008 Framingham model for 5-year CVD risk prediction.
      New modelRecalibrated 2008 Framingham model
      PrimaryReduced information
      Discrimination
       Apparent c-statistic0.7350.7310.728
       [95% CI][0.698, 0.782][0.701, 0.783][0.681–0.776]
       Optimism0.0130.0120.001
       Bias-corrected c-statistic0.7220.7190.727
      These values are slightly different from those in the study by Hua et al. [11] because in the present study we used multiple imputation to replace missing values with the estimates and evaluated performance of the recalibrated 2008 Framingham model on the 20 imputed data sets, while in the previous study [11] the recalibrated 2008 Framingham model was evaluated on the cohort with complete cases.
      Calibration
       Apparent mH-L χ2 statistic (p-value)5.70 (0.770)7.46 (0.589)19.17 (0.024)
      These values are slightly different from those in the study by Hua et al. [11] because in the present study we used multiple imputation to replace missing values with the estimates and evaluated performance of the recalibrated 2008 Framingham model on the 20 imputed data sets, while in the previous study [11] the recalibrated 2008 Framingham model was evaluated on the cohort with complete cases.
       Optimism6.315.0310.30
       Biased-corrected mH-L χ2 statistic (p-value)12.01 (0.212)12.49 (0.188)29.47 (<0.001)
      * These values are slightly different from those in the study by Hua et al.
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      because in the present study we used multiple imputation to replace missing values with the estimates and evaluated performance of the recalibrated 2008 Framingham model on the 20 imputed data sets, while in the previous study
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      the recalibrated 2008 Framingham model was evaluated on the cohort with complete cases.
      Figure 1
      Figure 1Mean observed and predicted 5-year cardiovascular risks in deciles of the predicted risks using (A) the primary model and (B) the reduced information model. The error bars represent 95% confidence intervals.
      Table 4 shows mean observed and predicted absolute risks for the whole cohort and different groups stratified by sex, age and predicted risk levels. The predicted mean 5-year risks for the whole cohort, women and men were 9.4%, 8.8% and 10.1%, respectively, using the primary model, and were 9.4%, 8.6% and 10.2%, respectively, using the reduced information model. These predicted values were almost identical to the observed 5-years risks (9.4%, 8.7% and 10.1% for the whole cohort, women and men, respectively).
      Table 4Mean 5-year cardiovascular risk estimated using Kaplan-Meier method, 1991 Framingham equation
      • Anderson K.
      • Odell P.
      • Wilson P.
      • Kannel W.
      Cardiovascular disease risk profiles.
      , recalibrated 2008 Framingham model
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      and the new models for the whole cohort and groups classified by sex, age and predicted risk levels.
      Sample sizeObserved risk based on Kaplan-Meier method, % [95% CI]
      Pooled values using Rubin’s rule [19] from estimates for 20 imputed data sets.
      Predicted risk, % [95% CI]
      Pooled values using Rubin’s rule [19] from estimates for 20 imputed data sets.
      1991 Framingham equation
      • Anderson K.
      • Odell P.
      • Wilson P.
      • Kannel W.
      Cardiovascular disease risk profiles.
      Recalibrated 2008 Framingham model
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      New model
      PrimaryReduced information
      Total1583
      Number of persons in each of 20 imputed data sets.
      9.4 [7.9–10.8]6.7 [6.4–7.1]
      Estimated for all people including persons automatically classified as those with a high cardiovascular risk by the guidelines. See criteria in the previous note.
      10.4 [9.8–10.9]9.4 [8.9–9.8]9.4 [8.9–9.8]
      Sex
       Women822
      Number of persons in each of 20 imputed data sets.
      8.7 [6.7–10.7]5.7 [5.2–6.2]
      Estimated for all people including persons automatically classified as those with a high cardiovascular risk by the guidelines. See criteria in the previous note.
      9.6 [8.9–10.3]8.8 [8.1–9.4]8.6 [8.1–9.3]
       Men761
      Number of persons in each of 20 imputed data sets.
      10.1 [7.9–12.2]7.9 [7.3–8.4]
      Estimated for all people including persons automatically classified as those with a high cardiovascular risk by the guidelines. See criteria in the previous note.
      11.2 [10.4–11.9]10.1 [9.4–10.6]10.2 [9.5–10.8]
      Age groups (years)
       30–34327
      Number of persons in each of 20 imputed data sets.
      3.2 [1.2–5.2]1.3 [1.2–1.5]
      Estimated for all people including persons automatically classified as those with a high cardiovascular risk by the guidelines. See criteria in the previous note.
      2.9 [2.6–3.1]3.3 [3.1–3.5]2.8 [2.7–3.0]
       35–44541
      Number of persons in each of 20 imputed data sets.
      3.7 [2.1–5.3]3.7 [3.4–4.0]
      Estimated for all people including persons automatically classified as those with a high cardiovascular risk by the guidelines. See criteria in the previous note.
      6.1 [5.7–6.5]5.6 [5.4–5.9]5.1 [4.9–5.3]
       45–54377
      Number of persons in each of 20 imputed data sets.
      13.0 [9.5–16.4]7.9 [7.3–8.5]
      Estimated for all people including persons automatically classified as those with a high cardiovascular risk by the guidelines. See criteria in the previous note.
      11.8 [11.0–12.5]9.9 [9.2–10.5]9.3 [8.9–9.8]
       55–74338
      Number of persons in each of 20 imputed data sets.
      20.3 [15.8–24.6]15.5 [14.5–16.5]
      Estimated for all people including persons automatically classified as those with a high cardiovascular risk by the guidelines. See criteria in the previous note.
      22.8 [21.4–24.3]20.6 [19.5–21.7]22.7 [21.623.8]
      Risk categories based on the guidelines
      • National Vascular Disease Prevention Alliance
      Guidelines for the management of absolute cardiovascular disease risk.
       High (>15%)450
      Average number of persons across imputed data sets.
      17.1 [11.0–23.2]20.9 [19.6–22.2]
      Estimated for persons not automatically classified as those with high CVD risk by the guidelines [4], i.e. persons do not have any of the following conditions: (1) Diabetes and age>60 years, (2) Diabetes with albumin creatinine ratio >2.5 mg/mmol for males or >3.5 mg/mmol for females, (3) moderate or severe chronic kidney disease, (4) a previous diagnosis of familial hypercholesterolaemia, (5) Systolic blood pressure (BP)≥180 mmHg or diastolic BP > 110 mmHg, and (6) total cholesterol >7.5 mmol/L.
      /13.8 [12.8–14.8]
      Estimated for all people including persons automatically classified as those with a high cardiovascular risk by the guidelines. See criteria in the previous note.
      20.4 [18.9–21.9]16.5 [15.4–17.6]16.3 [15.2–17.4]
       Moderate (10–15%)108
      Average number of persons across imputed data sets.
      11.7 [1.9–22.8]12.4 [11.5–13.2]17.1 [16.1–18.1]14.1 [13.0–15.2]15.8 [14.3–17.3]
       Low (<10%)1025
      Average number of persons across imputed data sets.
      5.7 [3.0–8.5]3.1 [2.9–3.2]5.3 [5.0–5.5]5.7 [5.0–6.0]5.7 [5.4–6.0]
      Risk categories based on the recalibrated 2008 Framingham equation
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
       High (>15%)348
      Average number of persons across imputed data sets.
      19.5 [13.5–25.5]15.8 [14.9–16.7]
      Estimated for persons not automatically classified as those with high CVD risk by the guidelines [4], i.e. persons do not have any of the following conditions: (1) Diabetes and age>60 years, (2) Diabetes with albumin creatinine ratio >2.5 mg/mmol for males or >3.5 mg/mmol for females, (3) moderate or severe chronic kidney disease, (4) a previous diagnosis of familial hypercholesterolaemia, (5) Systolic blood pressure (BP)≥180 mmHg or diastolic BP > 110 mmHg, and (6) total cholesterol >7.5 mmol/L.
      / 18.1 [17.1–19.1]
      Estimated for all people including persons automatically classified as those with a high cardiovascular risk by the guidelines. See criteria in the previous note.
      26.4 [24.9–27.9]20.4 [19.2–21.6]21.4 [20.2–22.5]
       Moderate (10–15%)212
      Average number of persons across imputed data sets.
      13.7 [1.1–25.3]8.4 [7.6–9.2]12.3 [11.3–13.3]11.6 [11.0–12.3]11.8 [11.0k12.6]
       Low (< 10%)1023
      Average number of persons across imputed data sets.
      5.1 [1.4–9.3]2.4 [2.2–2.6]4.5 [4.2–4.8]5.2 [4.7–5.3]4.8 [4.6–5.0]
      Risk categories based on the primary equation
       High (>15%)271
      Average number of persons across imputed data sets.
      25.4 [15.8–35.1]14.7 [13.3–16.1]
      Estimated for persons not automatically classified as those with high CVD risk by the guidelines [4], i.e. persons do not have any of the following conditions: (1) Diabetes and age>60 years, (2) Diabetes with albumin creatinine ratio >2.5 mg/mmol for males or >3.5 mg/mmol for females, (3) moderate or severe chronic kidney disease, (4) a previous diagnosis of familial hypercholesterolaemia, (5) Systolic blood pressure (BP)≥180 mmHg or diastolic BP > 110 mmHg, and (6) total cholesterol >7.5 mmol/L.
      / 18.0 [17.1-19.2]
      Estimated for all people including persons automatically classified as those with a high cardiovascular risk by the guidelines. See criteria in the previous note.
      26.6 [25.0–28.2]25.0 [23.1–25.7]25.9 [24.7–27.1]
       Moderate (10–15%)218
      Average number of persons across imputed data sets.
      12.6 [6.6–19.9]10.3 [9.7–10.9]14.9 [14.1–15.8]12.3 [12.1–12.5]12.5 [12.1–12.8]
       Low (<10%)1094
      Average number of persons across imputed data sets.
      4.6 [2.3–7.0]3.2 [3.0–3.4]5.4 [5.2–5.6]4.9 [4.9–5.1]4.7 [4.5–4.8]
      Risk categories based on the reduced information equation
       High (>15%)289
      Average number of persons across imputed data sets.
      25.2 [14.9–32.9]13.5 [12.3–14.7]
      Estimated for persons not automatically classified as those with high CVD risk by the guidelines [4], i.e. persons do not have any of the following conditions: (1) Diabetes and age>60 years, (2) Diabetes with albumin creatinine ratio >2.5 mg/mmol for males or >3.5 mg/mmol for females, (3) moderate or severe chronic kidney disease, (4) a previous diagnosis of familial hypercholesterolaemia, (5) Systolic blood pressure (BP)≥180 mmHg or diastolic BP > 110 mmHg, and (6) total cholesterol >7.5 mmol/L.
      / 17.0 [16.4–18.5]
      Estimated for all people including persons automatically classified as those with a high cardiovascular risk by the guidelines. See criteria in the previous note.
      25.7 [24.1–27.2]23.4 [22.1–24.6]25.6 [24.5–26.7]
       Moderate (10–15%)183
      Average number of persons across imputed data sets.
      12.9 [5.2–20.6]10.4 [9.7–11.1]15.0 [14.0–15.9]12.3 [11.8–12.8]12.3 [12.1k12.5]
       Low (<10%)1111
      Average number of persons across imputed data sets.
      5.0 [3.0–7.1]3.3 [3.1–3.5]5.6 [5.4–5.9]5.2 [5.0–5.4]4.7 [4.5–4.8]
      * Number of persons in each of 20 imputed data sets.
      Pooled values using Rubin’s rule
      • Rubin D.
      Multiple imputation for nonresponse in surveys.
      from estimates for 20 imputed data sets.
      Average number of persons across imputed data sets.
      § Estimated for persons not automatically classified as those with high CVD risk by the guidelines
      • National Vascular Disease Prevention Alliance
      Guidelines for the management of absolute cardiovascular disease risk.
      , i.e. persons do not have any of the following conditions: (1) Diabetes and age>60 years, (2) Diabetes with albumin creatinine ratio >2.5 mg/mmol for males or >3.5 mg/mmol for females, (3) moderate or severe chronic kidney disease, (4) a previous diagnosis of familial hypercholesterolaemia, (5) Systolic blood pressure (BP)≥180 mmHg or diastolic BP > 110 mmHg, and (6) total cholesterol >7.5 mmol/L.
      Estimated for all people including persons automatically classified as those with a high cardiovascular risk by the guidelines. See criteria in the previous note.
      When the cohort was stratified into high, moderate and low risks based on the predicted values, the 1991 Framingham equation [
      • Anderson K.
      • Odell P.
      • Wilson P.
      • Kannel W.
      Cardiovascular disease risk profiles.
      ] slightly overestimated the mean risk of the moderate-risk group (12.4% against 11.7%) and underestimated the mean risk of the low-risk group (3.1% against 5.7%). In the high-risk group classified by the guidelines’ algorithm, the 1991 Framingham equation [
      • Anderson K.
      • Odell P.
      • Wilson P.
      • Kannel W.
      Cardiovascular disease risk profiles.
      ] underestimated the mean risk of the whole group (13.8% against 17.1%), and overestimated the mean risk of persons who did not have any of the conditions pre-defined in the guidelines (20.9% against 17.1%). The recalibrated 2008 Framingham model [
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      ] overestimated the mean risk in the high-risk group (26.4% against 19.5%), but slightly underestimated the mean risk in the moderate- (12.3% against 13.2%) and low-risk (4.5% against 5.1%) groups. In contrast, mean risks predicted by the primary model were relatively close to the observed risks in the high- (25.0% vs 25.4%), moderate- (12.3% vs 12.6%) and low-risk (4.9% vs 4.6%) groups. Similar mean predicted and observed risks in three risk categories were observed using the reduced information model.
      When risk was calculated separately for women and men (see Tables S5 and S6 in the supplementary document), we found that, in women, the CVD risk predicted by the 1991 Framingham model [
      • Anderson K.
      • Odell P.
      • Wilson P.
      • Kannel W.
      Cardiovascular disease risk profiles.
      ] was only one third of the observed CVD risk in the youngest group (30–34 years old), while this ratio was much larger in other age groups of women (0.53–0.86) and in any age groups of men (0.6–1.1). When the cohort was classified into major CVD risk categories, the 1991 Framingham equation [
      • Anderson K.
      • Odell P.
      • Wilson P.
      • Kannel W.
      Cardiovascular disease risk profiles.
      ] underestimated CVD risk in all categories in women, while this was not the case in men with moderate risk. In both women and men, the agreement between the observed and predicted CVD risk using the recalibrated 2008 Framingham model [
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      ] and our new models was much better compared to the 1991 Framingham model [
      • Anderson K.
      • Odell P.
      • Wilson P.
      • Kannel W.
      Cardiovascular disease risk profiles.
      ] in every age group and major risk category.
      Using the primary model, the reduced information model, the recalibrated 2008 Framingham model [
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      ] and the guidelines for risk classification, the proportions of cohort (i) with high risk were 17.1%, 18.3%, 22.0% and 28.4%, respectively, (ii) with moderate risk were 13.8%, 11.6%, 13.4% and 6.8%, respectively, and (iii) with low risk were 69.1%, 70.2%, 64.7% and 64.8%, respectively (Figure 2). Given the good calibration and the predicted mean risks almost identical to the observed mean risks in any risk categories of the new models, these figures suggest that the guidelines overestimate the proportion of people with high risk, underestimate the proportion of people with moderate and low risk, and potentially misclassify a small proportion of people with actual high, moderate or low risk. The agreement between the guidelines and the new models in classification of proportions with different risk categories was higher in the low-risk category than in the moderate- and high-risk categories.
      Figure 2
      Figure 2Percentage of cohort with high (>15%), moderate (10–15%) and low 5-year cardiovascular risk (<10%) classified by the new models, the recalibrated 2008 Framingham model and the guidelines.
      Using our new models as a standard tool for identification of a person with a high 5-year CVD risk, we found potential misclassification of the high risk and moderate-low risk people by the Australian guidelines (Figure 3). Sixteen per cent (16%) of the population were false positives, i.e. the individuals classified as high risk by the guidelines but as moderate-low risk by either the primary or reduced information model. Four per cent (4%) or 6% of the population were false negatives on the basis of the primary or reduced information model, respectively, i.e. the individuals classified as moderate-low risk by the guidelines but as high risk by the primary or reduced information model. The estimated sensitivity and specificity of guidelines were 74% and 81%, respectively, based on the primary model, and were 67% and 80%, respectively, based on the reduced information model.
      Figure 3
      Figure 3Potential misclassification of the high risk and moderate-low risk by the Australian guidelines for prevention of CVD based on the (A) primary model and (B) reduced information model for 5-year CVD risk prediction. The bars represent proportions of people classified as high risk (>15%) and moderate-low risk classified by the guidelines, within each of which the upper part represents the proportion of people that are misclassified on the basis of the classification by the new models.

      Discussion

      In this study, we developed the first 5-year CVD risk prediction models based on Indigenous Australian data. Both the primary model and the reduced information model performed well, and had a far better calibration (mH-L χ2 statistic) than the recalibrated 2008 Framingham model [
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      ]. This was expected as in the recalibrated model only the baseline survival rate and means of risk factors were derived from the Indigenous data, while the predictors and their coefficients were obtained from the Framingham equation [
      • D’Agostino R.B.
      • Vasan R.S.
      • Pencina M.J.
      • Wolf P.A.
      • Cobain M.
      • Massaro J.M.
      • et al.
      General cardiovascular risk profile for use in primary care — the Framingham Heart Study.
      ]. As the reduced information model performed similarly to the primary model, it provides a convenient tool for a quick assessment of 5-year CVD risk where no laboratory variables are needed. Using multiple imputation combined with bootstrapping, we confirmed the findings in the study by Hua et al. [
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      ] that the 1991 Framingham model [
      • Anderson K.
      • Odell P.
      • Wilson P.
      • Kannel W.
      Cardiovascular disease risk profiles.
      ] underestimated mean 5-year risk of the Indigenous people, and the recalibrated 2008 Framingham model [
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      ] corrected the under prediction of the original model. We also showed that the 1991 Framingham equation [
      • Anderson K.
      • Odell P.
      • Wilson P.
      • Kannel W.
      Cardiovascular disease risk profiles.
      ] underestimated the CVD risk more severely in younger women (aged 30–34 years) compared to older women and to men. Despite this, the observed CVD risk in this age group is low (mean, 3.2; 95% CI, 1.2–5.2) and, therefore, this underestimation would not have impacted the cardiovascular health of younger women more strongly compared to older women and men.
      In terms of discrimination, the primary model and the reduced information model performed similarly to the recalibrated 2008 Framingham model [
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      ]. This might be because the c-statistic is not always sensitive to model fit [
      • May M.
      • Royston P.
      • Egger M.
      • Justice A.C.
      • Sterne J.A.
      • AC Collaboration
      Development and validation of a prognostic model for survival time data: application to prognosis of HIV positive patients treated with antiretroviral therapy.
      ,
      • Cook N.R.
      Use and misuse of the receiver operating characteristic curve in risk prediction.
      ]. Another explanation could be that a number of risk factors that are associated with CVD such as family history of CVD [
      • Damen J.A.
      • Hooft L.
      • Schuit E.
      • Debray T.P.
      • Collins G.S.
      • Tzoulaki I.
      • et al.
      Prediction models for cardiovascular disease risk in the general population: systematic review.
      ], depression [
      • Goldston K.
      • Baillie A.J.
      Depression and coronary heart disease: a review of the epidemiological evidence, explanatory mechanisms and management approaches.
      ] and history of acute rheumatic fever [
      • Carapetis J.R.
      • McDonald M.
      • Wilson N.J.
      Acute rheumatic fever.
      ,
      • Marijon E.
      • Mirabel M.
      • Celermajer D.S.
      • Jouven X.
      Rheumatic heart disease.
      ] (which is still common in many Australian Indigenous communities [
      • RHDAustralia (ARF/RHD writing group) NHFoAatCSoAaNZ
      The Australian guideline for prevention, diagnosis and management of acute rheumatic fever and rheumatic heart disease.
      ]) were not collected and therefore could not be controlled for in our analysis. Our data set also did not contain information on socioeconomic deprivation, a risk factor for CVD mortality in an Australian cohort [
      • Backholer K.
      • Hirakawa Y.
      • Tonkin A.
      • Giles G.
      • Magliano D.J.
      • Colagiuri S.
      • et al.
      Development of an Australian cardiovascular disease mortality risk score using multiple imputation and recalibration from national statistics.
      ]. Therefore, socioeconomic deprivation might be a potential risk factor for CVD including both fatal and non-fatal cardiovascular events as defined in our study. It is important to note that the distributions of risk factors in our study sample are different from those in the Framingham cohort [
      • D’Agostino R.B.
      • Vasan R.S.
      • Pencina M.J.
      • Wolf P.A.
      • Cobain M.
      • Massaro J.M.
      • et al.
      General cardiovascular risk profile for use in primary care — the Framingham Heart Study.
      ]. For example, participants in our study sample have a lower mean total cholesterol (198.9 vs 213.9 mg/dL), but higher mean SBP (134.5 vs 127.6 mmHg) and substantially higher mean rates of smoking (53.4% vs 34.7%) and diabetes (23.2% vs 5.0%) compared to those in the Framingham study [
      • D’Agostino R.B.
      • Vasan R.S.
      • Pencina M.J.
      • Wolf P.A.
      • Cobain M.
      • Massaro J.M.
      • et al.
      General cardiovascular risk profile for use in primary care — the Framingham Heart Study.
      ]. Our models include several factors such as triglycerides and ACR that are not included in the Framingham risk equations [
      • D’Agostino R.B.
      • Vasan R.S.
      • Pencina M.J.
      • Wolf P.A.
      • Cobain M.
      • Massaro J.M.
      • et al.
      General cardiovascular risk profile for use in primary care — the Framingham Heart Study.
      ,
      • Anderson K.
      • Odell P.
      • Wilson P.
      • Kannel W.
      Cardiovascular disease risk profiles.
      ]. The results from our analysis confirmed previous findings that ACR is an important CVD risk factor in Indigenous populations [
      • McDermott R.A.
      • McCulloch B.
      • Li M.
      Glycaemia and albuminuria as predictors of coronary heart disease in Aboriginal and Torres Strait Islander adults: a north Queensland cohort.
      ,
      • Barr E.
      • Cunningham J.
      • Tatipata S.
      • Dunbar T.
      • Kangaharan N.
      • Guthridge S.
      • et al.
      Associations of mortality and cardiovascular disease risks with diabetes and albuminuria in urban Indigenous Australians: the DRUID follow-up study.
      ]. Two studies of Indigenous Australians showed that microalbuminuria (ACR between 3 and 30 mg/mmol) and macroalbuminuria (ACR ≥ 30 mg/mmol) increased the risk of coronary heart disease by two to three times compared to normal albuminuria, after adjusting for other CVD risk factors [
      • Barr E.
      • Cunningham J.
      • Tatipata S.
      • Dunbar T.
      • Kangaharan N.
      • Guthridge S.
      • et al.
      Associations of mortality and cardiovascular disease risks with diabetes and albuminuria in urban Indigenous Australians: the DRUID follow-up study.
      ,
      • Wang Z.
      • Hoy W.E.
      Albuminuria and incident coronary heart disease in Australian Aboriginal people.
      ]. We did not categorise ACR because this reduced the model calibration.
      The outcomes from our model fitting showed that, in the presence of waist circumference, BMI was not significantly associated with CVD risk. This is in line with the results from the study by Wang and Hoy [
      • Wang Z.
      • Hoy W.
      Waist circumference, body mass index, hip circumference and waist-to-hip ratio as predictors of cardiovascular disease in Aboriginal people.
      ], which found that waist circumference was a better predictor for CVD risk than BMI in Indigenous Australians. This may be explained by the fact that Indigenous Australians have a greater amount of abdominal fat compared to the European Australians and the use of BMI for weight status classification has been found to be inappropriate in Indigenous populations [
      • Piers L.
      • Rowley K.
      • Soares M.
      • O’Dea K.
      Relation of adiposity and body fat distribution to body mass index in Australians of Aboriginal and European ancestry.
      ,
      • World Health Organization
      Obesity: preventing and managing the global epidemic. WHO Technical Report Series 894.
      ].
      Surprisingly, smoking was not a significant predictor of 5-year CVD risk in our study. An explanation for this might be that smoking was masked by SBP and triglycerides, as smoking was strongly correlated with the latter. In the study by Luke et al. [
      • Luke J.N.
      • Brown A.D.
      • Brazionis L.
      • O’Dea K.
      • Best J.D.
      • McDermott R.A.
      • et al.
      Exploring clinical predictors of cardiovascular disease in a central Australian Aboriginal cohort.
      ], smoking was also found to be significantly associated with dyslipidaemia and SBP. Although smoking did not significantly contribute to the prediction of 5-year CVD risk, we found it significant when fitting a model without the constraint on the 5-year follow-up time. Hence, the fact the smoking is not included in our models for 5-year CVD risk prediction does not mean that reducing smoking is not an important component of the CVD prevention strategy.
      The agreement between the Australian guidelines’ algorithm [
      • National Vascular Disease Prevention Alliance
      Guidelines for the management of absolute cardiovascular disease risk.
      ] and our new models in risk classification were only moderate. The false positives represented the largest disagreement, which can lead to unnecessary treatment and increased health care costs. The consequences of false negatives are potentially severe with individuals who would otherwise be medicated not receiving treatment to reduce CVD risk. Although the proportion of people with false negatives was relatively small (about 4–6% of the population), this is roughly equivalent to a large number of 31,000–47,000 Indigenous people with high CVD risk but could potentially be untreated, assuming that the distributions of the risk factors in the WPHC was similar to those in the Indigenous Australian population [
      • Australian Bureau of Statistics
      2940.0 — Census of population and housing: details of overcount and undercount.
      ].
      As our models were able to better classify the CVD risk of Indigenous people compared to the algorithm recommended by the Australian guidelines [
      • National Vascular Disease Prevention Alliance
      Guidelines for the management of absolute cardiovascular disease risk.
      ], they should be considered as a contemporarily evidence-based tool for CVD risk assessment in remote Indigenous Australians. Since CVD is the leading cause of death in Indigenous Australians [], translation of our findings to clinical practice is an important step towards addressing the target set by the Commonwealth of Australia, that is to close the gap in life expectancy between Indigenous and non-Indigenous Australians within a generation [
      • Australian Institute of Health and Welfare
      Chapter 5: Health of population groups.
      ,
      • Commonwealth of Australia-Department of the Prime Minister and Cabinet
      ]. Detailing strategies for this translation is beyond the scope of the present study, but it is worth noting that developing a tool for risk assessment that is readily available and acceptable to health professionals and Indigenous communities is crucial. Communication of this tool and our findings with physicians can be done in collaboration with the Heart Foundation and other relevant organisations via the internet, scientific meetings and media.

      Strengths and Limitations

      In the present study, we used a relatively large community-based cohort and high-quality data for model development, multiple imputation to deal with missing data, and rigorous methods to select the predictors and assess the model performance. However, our study has some limitations. The cohort used for model development might not represent the Indigenous population in North Queensland because the participants in the screening program were volunteers. Because our models were developed for remote Indigenous people and have not been tested in other populations, their generalisability is currently inconclusive. However, they provide an additional evidence-based tool for risk assessment that can be used alongside existing risk scores such as the recalibrated Framingham model [
      • Hua X.
      • McDermott R.
      • Lung T.
      • Wenitong M.
      • Tran-Duy A.
      • Li M.
      • et al.
      Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
      ]. As data on treatment for hypertension were not ascertained in our study, our models did not include treatment as a covariate, which is aligned with the 1991 Framingham equation [
      • Anderson K.
      • Odell P.
      • Wilson P.
      • Kannel W.
      Cardiovascular disease risk profiles.
      ]. However, we note that the 2008 Framingham equation [
      • D’Agostino R.B.
      • Vasan R.S.
      • Pencina M.J.
      • Wolf P.A.
      • Cobain M.
      • Massaro J.M.
      • et al.
      General cardiovascular risk profile for use in primary care — the Framingham Heart Study.
      ] accounted for the effect of anti-hypertensive treatment and therefore it is important to explore this treatment effect in future studies for risk stratification in Indigenous populations. Other potential risk factors such as family history of CVD [
      • Damen J.A.
      • Hooft L.
      • Schuit E.
      • Debray T.P.
      • Collins G.S.
      • Tzoulaki I.
      • et al.
      Prediction models for cardiovascular disease risk in the general population: systematic review.
      ], socioeconomic deprivation [
      • Backholer K.
      • Hirakawa Y.
      • Tonkin A.
      • Giles G.
      • Magliano D.J.
      • Colagiuri S.
      • et al.
      Development of an Australian cardiovascular disease mortality risk score using multiple imputation and recalibration from national statistics.
      ] and rheumatic heart disease [
      • Carapetis J.R.
      • McDonald M.
      • Wilson N.J.
      Acute rheumatic fever.
      ,
      • Marijon E.
      • Mirabel M.
      • Celermajer D.S.
      • Jouven X.
      Rheumatic heart disease.
      ] were not collected, and thus additional relevant predictors might be missed to be included in the models. Our equations were designed as a risk prediction tool and thus they may not represent all key risk factors associated with the aetiology of CVD. An example of this is the inclusion of triglycerides as the sole lipid fraction in our primary model while there is a well-established strong correlation between CVD and other lipid fractions such as total and HDL cholesterol [
      • Damen J.A.
      • Hooft L.
      • Schuit E.
      • Debray T.P.
      • Collins G.S.
      • Tzoulaki I.
      • et al.
      Prediction models for cardiovascular disease risk in the general population: systematic review.
      ]. In this respect the role of triglycerides in our equation represents both the direct impact as well as the indirect impact on CVD via its correlation with other lipid fractions in Indigenous Australians [
      • Australian Bureau of Statistics
      4727.0.55.003 — Australian Aboriginal and Torres Strait Islander Health Survey: biomedical results, 2012–13.
      ,
      • Li M.
      • McCulloch B.
      • McDermott R.
      Metabolic syndrome and incident coronary heart disease in Australian indigenous populations.
      ]. Although we rigorously assess the model performance internally, external validation of the models using other data sources is needed and this warrants further study in the future.

      Conclusions

      The algorithm for classification of people as high, moderate and low 5-year CVD risk recommended by the Australian guidelines for prevention of CVD [
      • National Vascular Disease Prevention Alliance
      Guidelines for the management of absolute cardiovascular disease risk.
      ] is subject to misclassification of Indigenous people in these major risk categories, and is not accurate in estimating mean absolute risk in each of these categories. The prediction models we developed improved these. The reduced information model, with good discrimination and calibration, provides a convenient and reasonable tool for a quick assessment of 5-year CVD risk where no laboratory variables are required.

      Funding Source

      This study is supported by the Australia National Health and Medical Research Council (project number NHMRC/1107140). The sponsor has no role in study design; or collection, analysis and interpretation of data; or writing of the manuscript; or the decision to submit the article for publication.

      Conflicts of Interest

      The authors declared that there is no potential conflict of interest.

      Appendix A. Supplementary data

      The following is Supplementary data to this article:

      References

        • International Diabetes Federation
        Expert meeting on indigenous peoples, diabetes and development report.
        International Diabetes Federation, Copenhagen2012 (. Accessed 9 August 2018)
        • McDermott R.A.
        • McCulloch B.
        • Li M.
        Glycaemia and albuminuria as predictors of coronary heart disease in Aboriginal and Torres Strait Islander adults: a north Queensland cohort.
        Med J Aust. 2011; 194: 514-518
        • Barr E.
        • Cunningham J.
        • Tatipata S.
        • Dunbar T.
        • Kangaharan N.
        • Guthridge S.
        • et al.
        Associations of mortality and cardiovascular disease risks with diabetes and albuminuria in urban Indigenous Australians: the DRUID follow-up study.
        Diabet Med. 2017; 34: 946-957
        • National Vascular Disease Prevention Alliance
        Guidelines for the management of absolute cardiovascular disease risk.
        2012 (. Accessed 9 August 2018)
        • Goff D.C.
        • Lloyd-Jones D.M.
        • Bennett G.
        • Coady S.
        • D’agostino R.B.
        • Gibbons R.
        • et al.
        ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines.
        J Am Coll Cardiol. 2013; 2014: 2935-2939
        • Reath J.
        • Brown N.
        Managing cardiovascular disease in Aboriginal and Torres Strait Islander people.
        Aust Prescr. 2010; 33: 72-75
        • Brown A.
        • O’Shea R.L.
        • Mott K.
        • McBride K.F.
        • Lawson T.
        • Jennings G.L.
        Essential service standards for equitable national cardiovascular care for Aboriginal and Torres Strait Islander people.
        Heart Lung Circ. 2015; 24: 126-141
        • Chan A.
        Making the case for a more accurate cardiovascular disease risk assessment tool for Indigenous Australians.
        Contemp Nurse. 2015; 50: 92-93
        • Vasant B.R.
        • Matthews V.
        • Burgess C.P.
        • Connors C.M.
        • Bailie R.S.
        Wide variation in absolute cardiovascular risk assessment in Aboriginal and Torres strait Islander people with type 2 diabetes.
        Front Public Health. 2016; 4: 37
        • Wang Z.
        • Hoy W.E.
        Is the Framingham coronary heart disease absolute risk function applicable to Aboriginal people?.
        Med J Aust. 2005; 182: 66-69
        • Hua X.
        • McDermott R.
        • Lung T.
        • Wenitong M.
        • Tran-Duy A.
        • Li M.
        • et al.
        Validation and recalibration of the Framingham cardiovascular disease risk models in an Australian Indigenous cohort.
        Eur J Prev Cardiol. 2017; 24: 1660-1669
        • Wang Z.
        • Hoy W.
        Waist circumference, body mass index, hip circumference and waist-to-hip ratio as predictors of cardiovascular disease in Aboriginal people.
        Eur J Clin Nutr. 2004; 58: 888-893
        • Adegbija O.
        • Hoy W.
        • Wang Z.
        Prediction of cardiovascular disease risk using waist circumference among Aboriginals in a remote Australian community.
        BMC Public Health. 2015; 15: 57
        • Modesti P.A.
        • Agostoni P.
        • Agyemang C.
        • Basu S.
        • Benetos A.
        • Cappuccio F.P.
        • et al.
        Cardiovascular risk assessment in low-resource settings: a consensus document of the European Society of Hypertension Working Group on Hypertension and Cardiovascular Risk in Low Resource Settings.
        J Hypertens. 2014; 32: 951-960
        • Rowley K.G.
        • Thorpe A.H.
        Research, information and consent for the Australian Health Survey: a separate standard for Indigenous people?.
        Med J Aust. 2011; 195: 158-159
        • World Health Organization
        Prevention of cardiovascular disease : guidelines for assessment and management of cardiovascular risk.
        World Health Organization, Geneva2007 (Accessed 9 August 2018)
        • Miller G.
        • McDermott R.
        • McCulloch B.
        • Leonard D.
        • Arabena K.
        • Muller R.
        The Well Person’s Health Check: a population screening program in indigenous communities in north Queensland.
        Aust Health Rev. 2002; 25: 136-147
        • D’Agostino R.B.
        • Vasan R.S.
        • Pencina M.J.
        • Wolf P.A.
        • Cobain M.
        • Massaro J.M.
        • et al.
        General cardiovascular risk profile for use in primary care — the Framingham Heart Study.
        Circulation. 2008; 117: 743-753
        • Rubin D.
        Multiple imputation for nonresponse in surveys.
        John Wiley & Son, New York2004
        • Wang Z.
        • Hoy W.E.
        Albuminuria and incident coronary heart disease in Australian Aboriginal people.
        Kidney Int. 2005; 68: 1289-1293
        • Grambsch P.M.
        • Therneau T.M.
        Proportional hazards tests and diagnostics based on weighted residuals.
        Biometrika. 1994; : 515-526
        • Woodward M.
        Epidemiology: study design and data analysis.
        3rd ed. CRC Press, Boca Raton2014
        • Harrell F.E.
        • Lee K.L.
        • Mark D.B.
        Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors.
        Stat Med. 1996; 15: 361-387
        • D’Agostino R.
        • Nam B.-H.
        Evaluation of the performance of survival analysis models: discrimination and calibration measures.
        Handbook of statistics. 23. 2003: 1-25
        • D’Agostino Sr., R.B.
        • Grundy S.
        • Sullivan L.M.
        • Wilson P.
        • CRP Group
        Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation.
        JAMA. 2001; 286: 180-187
        • Rubin D.B.
        • Schenker N.
        Multiple imputation for interval estimation from simple random samples with ignorable nonresponse.
        J Am Stat Assoc. 1986; 81: 366-374
        • Anderson K.
        • Odell P.
        • Wilson P.
        • Kannel W.
        Cardiovascular disease risk profiles.
        Am Heart J. 1991; 121: 293-298
        • May M.
        • Royston P.
        • Egger M.
        • Justice A.C.
        • Sterne J.A.
        • AC Collaboration
        Development and validation of a prognostic model for survival time data: application to prognosis of HIV positive patients treated with antiretroviral therapy.
        Stat Med. 2004; 23: 2375-2398
        • Cook N.R.
        Use and misuse of the receiver operating characteristic curve in risk prediction.
        Circulation. 2007; 115: 928-935
        • Damen J.A.
        • Hooft L.
        • Schuit E.
        • Debray T.P.
        • Collins G.S.
        • Tzoulaki I.
        • et al.
        Prediction models for cardiovascular disease risk in the general population: systematic review.
        BMJ. 2016; 353: i2416
        • Goldston K.
        • Baillie A.J.
        Depression and coronary heart disease: a review of the epidemiological evidence, explanatory mechanisms and management approaches.
        Clin Psychol Rev. 2008; 28: 288-306
        • Carapetis J.R.
        • McDonald M.
        • Wilson N.J.
        Acute rheumatic fever.
        Lancet. 2005; 366: 155-168
        • Marijon E.
        • Mirabel M.
        • Celermajer D.S.
        • Jouven X.
        Rheumatic heart disease.
        Lancet. 2012; 379: 953-964
        • RHDAustralia (ARF/RHD writing group) NHFoAatCSoAaNZ
        The Australian guideline for prevention, diagnosis and management of acute rheumatic fever and rheumatic heart disease.
        2nd ed. National Heart Foundation of Australia and the Cardiac Society of Australia and New Zealand, 2012
        • Backholer K.
        • Hirakawa Y.
        • Tonkin A.
        • Giles G.
        • Magliano D.J.
        • Colagiuri S.
        • et al.
        Development of an Australian cardiovascular disease mortality risk score using multiple imputation and recalibration from national statistics.
        BMC Cardiovasc Disord. 2017; 17: 17
        • Piers L.
        • Rowley K.
        • Soares M.
        • O’Dea K.
        Relation of adiposity and body fat distribution to body mass index in Australians of Aboriginal and European ancestry.
        Eur J Clin Nutr. 2003; 57: 956-963
        • World Health Organization
        Obesity: preventing and managing the global epidemic. WHO Technical Report Series 894.
        World Health Organization, Geneva2000
        • Luke J.N.
        • Brown A.D.
        • Brazionis L.
        • O’Dea K.
        • Best J.D.
        • McDermott R.A.
        • et al.
        Exploring clinical predictors of cardiovascular disease in a central Australian Aboriginal cohort.
        Eur J Prev Cardiol. 2013; 20: 246-253
        • Australian Bureau of Statistics
        2940.0 — Census of population and housing: details of overcount and undercount.
        Australian Bureau of Statistics, Australia2016 (Accessed 9 August 2018)
        • Australian Bureau of Statistics
        3303.0 — Causes of death.
        Australian Bureau of Statistics, Australia2015
        • Australian Institute of Health and Welfare
        Chapter 5: Health of population groups.
        Australia’s health 2016. Australia’s health series no. 15. Cat. no. AUS 199. AIHW, Canberra2016 (Accesseds November 20, 2018)
        • Commonwealth of Australia-Department of the Prime Minister and Cabinet
        Closing the gap Prime Minister’s report.
        2018 (Accessed July 31, 2018)
        • Australian Bureau of Statistics
        4727.0.55.003 — Australian Aboriginal and Torres Strait Islander Health Survey: biomedical results, 2012–13.
        2014 (Accessed 30 November 2018)
        • Li M.
        • McCulloch B.
        • McDermott R.
        Metabolic syndrome and incident coronary heart disease in Australian indigenous populations.
        Obesity. 2012; 20: 1308-1312