Research on the Effectiveness of Individualized Feedback on Voting Behavior

Abstract

This study investigates the effectiveness of individualized feedback to voters from the perspective of changes in decision-making. We conducted a two-phase-based study: In the first phase, 163 participates with different backgrounds were invited to do mock voting for two presidential candidates according to the candidates’ performance after three debates, and to write and rank order six reasons for their votes. In the second phase, participants received feedback including the results of a Rasch Model analysis of reasons for voting. Shortly after reading the analysis, voters were asked to respond to a new set of measurment items, and then do another mock vote. A detailed comparative analysis of voters’ protocols before and after feedback indicates that the feedback is useful in helping voters make decisions. That is, they incorporated the feedback into their voting. This study sheds further light on their rating behavior. However, the feedback impacts individuals differently. Some raters adjust their decisions, while the others remain unchanged.

Appendices

Appendix: Example Feedback Report for Rater X

• Report on Individual Feedback Information of Voter X

Your Individual Feedback information is shown as following:

• (1) Observed count

You did not use categories 1 of the rating scale like some respondents did in the group. In this way we almost have to collapsing these categories. In order to improve the measurement quality, we intent to invite you to take part in another mock voting.

• (2) Consistency

Score internal consistency refers to the relative consistency of your review scores, namely whether your score presents a certain pattern. The index is weighted mean square(infit). If the rater displayed an infit, that means that square value is above 1.4 (Infit), the values <0.8 means that the rater is identified as rating with too little variation. That the values is between (0.8, 1.4) belongs to the normal range, a value of 1 shows that the data and the statistical model fitting is good.

Your infit value is 0.79. That means that a further investigation was necessary to ascertain whether the lack of variation was achieved by a central tendency effect. In another word, you only used same categories for both candidates. Do not feel scare to use a large variety

• (3) Bias

Bias refers to any individual biases raters in relation to the rating scale criteria. It means the tendency of a measurement process to over- or under-estimate the value of a population parameter. Raters were considered to have a significant bias if they displayed a z-score above +2 or below −2.

In your case, your rating was slightly harsh. Only remember this feedback when you rate again.

Overall Evaluation

Overall, your ratings were reasonable. When compared to the other raters in the group. You rated consistently and made good use of the rating criteria. However, you were rating slightly severe. Next time you rate, remember this feedback when you have trouble deciding between two rating scale points.