•  
  •  
 

Archives of Assessment Psychology

Abstract

Purpose: The current study was conducted to examine the interrater reliability of the Raphael Projective System (RPS), a standardized system for the administration, scoring, and interpretation of projective drawings. In anticipation of completion of the comprehensive manual of the RPS, the authors examined the inter-rater reliability of raters who were trained on scoring House, Tree, and Person drawings. Methods: Three of the authors refined the scoring system and reached consensus on scoring for 30 projective drawing test protocols (ten House drawings, ten Tree drawings, and ten Person drawings), which they had independently scored. Five graduate psychology students were trained on the system and independently scored the same 30 drawings scored by the authors. The percentages of agreement and inter-rater reliability (IRR) between the students and the authors and between all rater pairs were calculated. Due to the high prevalence of negative responses, agreement was measured using both kappa and adjusted kappa statistic for skewed prevalence. Results: The average percentages of agreement between the student raters and the authors were as follows: 93.2% for House drawings, 85.4% for Tree drawings, 87.8% for Person drawings, and 89.7% for all drawings. Inter-rater reliability between the student raters and the authors’ consensus was substantial overall, with an average k=0.88 for House drawings, k =0.72 for Tree drawings, and k=0.76 for Person drawings, with an overall average k = 0.79. An average prevalence index was measured at 0.57, and positive agreement was averaged at 0.94. Discussion: The findings suggest strong inter-rater reliability for the RPS, even for novice raters with minimal instruction on the system. The results are similar in strength to previous studies examining the reliability of projective drawing scoring systems.

Download

Share

COinS