*CANCELLED* Predictive Accuracy Measures for Binary Outcomes: Relationships and Impact of Incidence Rate
Description
Evaluating the performance of models predicting a binary outcome can be done using a variety of measures. While some measures intend to describe the model's overall fit, others more accurately describe the model's ability to discriminate between the two outcomes. A desirable model would be one that both fit the data and could discriminate between the two outcomes well. In this presentation, the relationships among the measures of discrimination and overall fit will be examined under general conditions and also controlling for the incidence rate in the data used to build the model. The measures of interest include the area under the ROC curve (auc), Brier score, discrimination slope, log loss and r-squared. Presented analysis includes the use of real data from common medical research studies and simulated data controlling for the incidence rate.
Date of Event
April 21, 2016
*CANCELLED* Predictive Accuracy Measures for Binary Outcomes: Relationships and Impact of Incidence Rate
Evaluating the performance of models predicting a binary outcome can be done using a variety of measures. While some measures intend to describe the model's overall fit, others more accurately describe the model's ability to discriminate between the two outcomes. A desirable model would be one that both fit the data and could discriminate between the two outcomes well. In this presentation, the relationships among the measures of discrimination and overall fit will be examined under general conditions and also controlling for the incidence rate in the data used to build the model. The measures of interest include the area under the ROC curve (auc), Brier score, discrimination slope, log loss and r-squared. Presented analysis includes the use of real data from common medical research studies and simulated data controlling for the incidence rate.