Title

Debunking the Evaluation “Gotcha” Myth: Developing Rapport and Trust in Qualitative Education Evaluation Research

Location

DeSantis Room 1047

Format Type

Plenary

Format Type

Paper

Start Date

15-1-2020 10:15 AM

End Date

15-1-2020 10:35 AM

Abstract

Many federal grants require an evaluation component used to judge program efficacy, scalability, and future funding. Qualitative researchers are poised to make significant contributions to the design and conduct of such evaluations, but face the common challenge of evaluations being thought of as a “gotcha.” In-depth interviews often produce a tension where stakeholders feel “tested” on their progress as well as pressured to place the project in the best possible light. In particular, using in-depth interviews in evaluation research tests fundamental premises of qualitative research methodology in at least three ways. First, a primary promise made to the respondents involved in interview-based qualitative research is anonymity and confidentiality. Yet, using interview methodology in evaluation research may sometimes require identifying stakeholders, often by name or institution. Second, qualitative researchers normally take great care in diminishing hierarchies between themselves and the people they are interviewing. However, during interviews occurring in evaluation research, creating non-hierarchical relationships may prove challenging as the evaluator is perceived as the one having power over the assessment of the project. Finally, interviewees may choose to answer questions in as few words as possible, for fear of contradicting themselves or any aspect of the project. This creates a challenge for collecting the thick descriptions that are a hallmark of qualitative research. Taking these concerns together, using interview methodology in evaluation research poses issues for trust and rapport, which are integral to the success of qualitative data collection. We conclude with strategies developed from the field to address these concerns.

Keywords

education evaluation, evaluation methods, stakeholder relationships

Comments

Thank you for this opportunity to submit our work for your review!

This document is currently not available here.

COinS
 
Jan 15th, 10:15 AM Jan 15th, 10:35 AM

Debunking the Evaluation “Gotcha” Myth: Developing Rapport and Trust in Qualitative Education Evaluation Research

DeSantis Room 1047

Many federal grants require an evaluation component used to judge program efficacy, scalability, and future funding. Qualitative researchers are poised to make significant contributions to the design and conduct of such evaluations, but face the common challenge of evaluations being thought of as a “gotcha.” In-depth interviews often produce a tension where stakeholders feel “tested” on their progress as well as pressured to place the project in the best possible light. In particular, using in-depth interviews in evaluation research tests fundamental premises of qualitative research methodology in at least three ways. First, a primary promise made to the respondents involved in interview-based qualitative research is anonymity and confidentiality. Yet, using interview methodology in evaluation research may sometimes require identifying stakeholders, often by name or institution. Second, qualitative researchers normally take great care in diminishing hierarchies between themselves and the people they are interviewing. However, during interviews occurring in evaluation research, creating non-hierarchical relationships may prove challenging as the evaluator is perceived as the one having power over the assessment of the project. Finally, interviewees may choose to answer questions in as few words as possible, for fear of contradicting themselves or any aspect of the project. This creates a challenge for collecting the thick descriptions that are a hallmark of qualitative research. Taking these concerns together, using interview methodology in evaluation research poses issues for trust and rapport, which are integral to the success of qualitative data collection. We conclude with strategies developed from the field to address these concerns.