CCE Theses and Dissertations

Date of Award

2024

Document Type

Dissertation

Degree Name

Doctor of Philosophy in Information Systems (DISS)

Department

College of Computing and Engineering

Advisor

Laurie Dringus

Committee Member

Ling Wang

Committee Member

Junping Sun

Abstract

As false news can propagate to others rapidly, social media platforms employ multiple methods to combat misinformation. Debunking techniques are warning features embedded into a platform’s interface that alert readers of misinformation. These warnings have two goals: to “debunk” false information and to prevent the further spread of misinformation. Researchers have evaluated the effectiveness of debunking techniques to understand how users increase their awareness of misinformation, and what users do with the information given in warning messages. Message popup warnings are a newer and understudied type of debunking technique.

The overarching research question of this study was: Are message popup warnings effective for deterring the spread of misinformation? The research goal was to determine the effectiveness of message popup warnings in the context of the user’s choice behaviors of liking, sharing, or commenting on misinformation. Four sub-questions explored: (RQ1) to what extent users read the full content of posting, with or without warnings (user engagement), (RQ2) what differences occur in user’s choice responses to the posting, with or without warnings (effectiveness), (RQ3) how users rate the credibility of the posting, with and without warnings, and (RQ4) how users rate the usability and usefulness of message popup warnings as a debunking technique.

The study was conducted as a between-subjects experimental design using an online interactive scenario and survey (ISS). The participants (N = 109) were recruited from the student population at Wentworth Institute of Technology (WIT) and were randomly assigned to either the control or treatment group. The ISS combined interactive Facebook-like scenarios (i.e., postings) that contained links to the full text of an article to be read and an embedded survey that prompted users to take an action or no action with the posting. By adapting questions from the Perceived Usefulness Measurement Scale (PUMS) and the System Usability Scale (SUS), the treatment group also assessed the usefulness and usability of message popup warnings.

Three hypotheses (H1-H3) were created as null hypotheses to test the first three research questions. Regarding H1, no significant differences were found for user engagement between the treatment group (presented with message popup warnings) and the control group (presented without warnings). For H2, there were no significant differences found for effectiveness between the treatment and control groups in user choice behaviors. However, when the data was compared within the treatment posting types (false postings with warnings, false postings without warnings, and true postings) and compared responses from before and after the appearance of the warning, there were significant differences found in user choice actions. Regarding H3, there were significant differences found for credibility between treatment and control groups, suggesting that participants found postings with message popup warnings significantly less credible than postings without a warning. Additionally, data gathered from the SUS found usability unacceptable, while the PUMS indicated that the usefulness was fair with some deficiencies.

The results suggest that message popup warnings may be effective for influencing the credibility of postings and may be effective for deterring the spread of misinformation when warnings appear on some of the postings. Future studies could vary the messaging in the warning, study different participant age groups, and investigate user responses across multiple platforms. Additionally, platforms may want to add training messages to help users distinguish between true and false postings.

Download
Share Feedback

Share

COinS