Evaluting Practical Limitations of Signal Detection Theory in Clinical Behavioral Research

Evaluting Practical Limitations of Signal Detection Theory in Clinical Behavioral Research

Alvin Sherman Library

Deficits in inhibitory control - the ability to suppress automatic, impulsive responses - has been widely considered as a potential biomarker for mental health disorders. Signal Detection Theory (SDT) has provided a systematic measurement method for inhibitory control in clinical behavioral assessment. To evaluate the feasibility of inhibitory control as a biomarker, researchers have used meta-analysis to examine its effect size (Hedge's g) in differentiating between individuals with mental health disorders and healthy controls. Values of g>0.8 indicate that inhibitory control may serve as a clinically useful biomarker for distinguishing patients from healthy individuals, whereas g<0.8 suggests limited clinical utility. However, our study found that a limitation of SDT - its lack of reliability when participants show perfect performance - would lead to distorted and misleading results when evaluating the efficacy of inhibitory control as a biomarker. In our computational simulation, we found that when applying SDT to a clinically useful biomarker (for example, true effect size g = 0.95), SDT's practical limitation could lower the estimated effect size (for example, estimated effect size g = 0.77) and mistook useful biomarkers as low-utility. Based on the results, SDT shows a substantial problem in clinical behavioral measurement, and we propose to identify practical solutions for this problem in future research.