Event Title

*CANCELLED* Predictive Accuracy Measures for Binary Outcomes: Relationships and Impact of Incidence Rate

Description

Evaluating the performance of models predicting a binary outcome can be done using a variety of measures. While some measures intend to describe the model's overall fit, others more accurately describe the model's ability to discriminate between the two outcomes. A desirable model would be one that both fit the data and could discriminate between the two outcomes well. In this presentation, the relationships among the measures of discrimination and overall fit will be examined under general conditions and also controlling for the incidence rate in the data used to build the model. The measures of interest include the area under the ROC curve (auc), Brier score, discrimination slope, log loss and r-squared. Presented analysis includes the use of real data from common medical research studies and simulated data controlling for the incidence rate.

Presenter Profile Page(s)

http://stat.fsu.edu/people/students.php?id=731

Date of Event

April 21, 2016

Share

COinS
 
Apr 21st, 12:00 PM Apr 21st, 1:00 PM

*CANCELLED* Predictive Accuracy Measures for Binary Outcomes: Relationships and Impact of Incidence Rate

Evaluating the performance of models predicting a binary outcome can be done using a variety of measures. While some measures intend to describe the model's overall fit, others more accurately describe the model's ability to discriminate between the two outcomes. A desirable model would be one that both fit the data and could discriminate between the two outcomes well. In this presentation, the relationships among the measures of discrimination and overall fit will be examined under general conditions and also controlling for the incidence rate in the data used to build the model. The measures of interest include the area under the ROC curve (auc), Brier score, discrimination slope, log loss and r-squared. Presented analysis includes the use of real data from common medical research studies and simulated data controlling for the incidence rate.