Debunking the Evaluation “Gotcha” Myth: Developing Rapport and Trust in Qualitative Education Evaluation Research
Location
DeSantis Room 1047
Format Type
Plenary
Format Type
Paper
Start Date
15-1-2020 10:15 AM
End Date
15-1-2020 10:35 AM
Abstract
Many federal grants require an evaluation component used to judge program efficacy, scalability, and future funding. Qualitative researchers are poised to make significant contributions to the design and conduct of such evaluations, but face the common challenge of evaluations being thought of as a “gotcha.” In-depth interviews often produce a tension where stakeholders feel “tested” on their progress as well as pressured to place the project in the best possible light. In particular, using in-depth interviews in evaluation research tests fundamental premises of qualitative research methodology in at least three ways. First, a primary promise made to the respondents involved in interview-based qualitative research is anonymity and confidentiality. Yet, using interview methodology in evaluation research may sometimes require identifying stakeholders, often by name or institution. Second, qualitative researchers normally take great care in diminishing hierarchies between themselves and the people they are interviewing. However, during interviews occurring in evaluation research, creating non-hierarchical relationships may prove challenging as the evaluator is perceived as the one having power over the assessment of the project. Finally, interviewees may choose to answer questions in as few words as possible, for fear of contradicting themselves or any aspect of the project. This creates a challenge for collecting the thick descriptions that are a hallmark of qualitative research. Taking these concerns together, using interview methodology in evaluation research poses issues for trust and rapport, which are integral to the success of qualitative data collection. We conclude with strategies developed from the field to address these concerns.
Keywords
education evaluation, evaluation methods, stakeholder relationships
Debunking the Evaluation “Gotcha” Myth: Developing Rapport and Trust in Qualitative Education Evaluation Research
DeSantis Room 1047
Many federal grants require an evaluation component used to judge program efficacy, scalability, and future funding. Qualitative researchers are poised to make significant contributions to the design and conduct of such evaluations, but face the common challenge of evaluations being thought of as a “gotcha.” In-depth interviews often produce a tension where stakeholders feel “tested” on their progress as well as pressured to place the project in the best possible light. In particular, using in-depth interviews in evaluation research tests fundamental premises of qualitative research methodology in at least three ways. First, a primary promise made to the respondents involved in interview-based qualitative research is anonymity and confidentiality. Yet, using interview methodology in evaluation research may sometimes require identifying stakeholders, often by name or institution. Second, qualitative researchers normally take great care in diminishing hierarchies between themselves and the people they are interviewing. However, during interviews occurring in evaluation research, creating non-hierarchical relationships may prove challenging as the evaluator is perceived as the one having power over the assessment of the project. Finally, interviewees may choose to answer questions in as few words as possible, for fear of contradicting themselves or any aspect of the project. This creates a challenge for collecting the thick descriptions that are a hallmark of qualitative research. Taking these concerns together, using interview methodology in evaluation research poses issues for trust and rapport, which are integral to the success of qualitative data collection. We conclude with strategies developed from the field to address these concerns.
Comments
Thank you for this opportunity to submit our work for your review!