Keywords

1 Introduction

Motivating energy reduction behavior by using technological solutions (such as mobile applications, websites, and in home feedback devices) is an exciting and challenging research and product design endeavor. Hundreds of household energy feedback technology products have been developed in less than a decade [1]. Technology design in the energy reduction field is often characterized by one of two perspectives; “build and they will come” and “begin with human attributes, motivations, and ability.” From the human centered design perspective – human characteristics and perceptions precede and iteratively refine product design. Specific human considerations motivating the use of specific technologies include personality factors, demographics, and perceptions about the focal technology such as trust in content and source, topic involvement, and perceived functionality. These perceptions about a technology are often assessed by the SUS, measures of “stickiness” or usability or other newer indices such as the UPscale [2–4]. While usability perceptions can influence continued technology refinements, judged technology effectiveness cannot stop at user’s perceived usability. Prior to scaled deployment, technologies must be assessed with regard to higher order outcomes such behavioral intention, perceived behavioral barriers and actual behavior. Understanding the potential for impact is key to technology scaling.

In this randomized controlled trial (RCT), we examine the effects of three prototype motivationally framed energy reduction Facebook applications on perceived usability, behavioral intention, and perceived barriers to behavior. Further, we investigate whether three human personality characteristics that have the potential to be drivers of motivation to change behavior are related to technology impact. Specifically we investigate if when human motivations “match” the motivational frames of the technology prototype behavioral outcomes are enhanced. For example, do individuals high in the need for positive emotion perform better with the affectively framed technology?

1.1 Relevant Literature

Rarely do technologies aimed to engage users to reduce energy focus on emotional content, or differentiate the emotional content from other frames such as need for thought or cognitions and need for affiliation or social comparison such as game play with others [1]. In fact, most energy reduction technologies focus on feedback regarding cost and kWh savings, at times combined with incentives and behavioral principles such as goal setting and social comparison [5]. Yet, few technologies and intervention studies compare the effects of different motivationally framed applications or perhaps even more importantly, examine the extent to which the frames are even more (or less) effective when matched to individual personality characteristics and motivations.

Human Personality Characteristics, Persuasion, and Behavior Change. There is a long history of research into human motivations and personality characteristics and their role in persuasive communications; from ads, to narratives to stories. Much of this research literature is steeped in theoretical perspectives regarding dual system processing of information.

One well researched human personality characteristic theorized to play a role in persuasion and behavior is the need for cognition (NFC). NFC is defined by Cacioppo and Petty [6, 7] as the inclination to engage in and like thinking and problem solving, this tendency is well studied in many different areas of inquiry, from attitudes and persuasion, judgment and decision making, interpersonal and group interactions and applied settings [7]. People high in need for cognition are more likely to form their attitudes and actions by paying close attention to relevant arguments (i.e., via the central route to persuasion). People lows in need for cognition are more likely to rely on peripheral cues such as speaker or message attractiveness and credibility or is more likely to use stereotypes to judge others. NFC is correlated with numeracy skill, which is intuitively linked to the commonly used energy feedback interface – graphic displays of energy consumption [8]. We argue that individuals high in NFC would be more likely to prefer and engage in interfaces that require thinking and analysis, such as energy feedback graphs.

Need for affect (NFA) has been proposed as the affective counterpart to the NFC [9]. Need for affect is defined as the “general motivation of people to approach or avoid situations and activities that are emotion inducing for themselves and others.” (p. 586). In particular, NFA may be most persuasive in certain emotion evoking contexts, such as story telling where it has been demonstrated that transportability and NFA moderate the effects of stories on attitudes [10]. Maio and Esses [9] developed a two part need for affect instrument, the seeking out of emotions (approach subscale) and the avoidance of emotional situations. The approach subscale has been shown to predict selection of emotional versus non-emotional movies, and the emotionality of respondents’ favorite television shows. Parallel to the message tailoring results for NFC, that is, those high in NFC are more persuaded by factual information; those high in NFA when exposed to emotional content were more likely to be persuaded [10, 11].

This personal motivation and message type interaction points to the conceptual importance of “motivational message tailoring,” indicating that this conceptual persuasion profiling (e.g. rather than demographic tailoring) shows promise for the outcomes of energy reduction technologies. To date persuasion profiling is not widely used in energy reduction or climate change action and advocacy.

A third potential human motivator is the need to affiliate with others. Hill [11] conceptualized and built a five factor scale of human motivation regarding people. Given recent attention to competition via game play [12] and norm interventions involving comparisons of energy consumers to their “neighbors,” an affiliation need that is social comparison based could future improve the development of “social” energy reduction interventions. That is, individuals high on the social comparison scale may be more receptive to interventions that involve competition, self-monitoring, and comparison to others. Human responses to normative comparison feedback are one of the more widespread behavioral principles in current use [13]. Recent experiments regarding “neighbor” comparison feedback may be potentially related social comparison affiliation, even though in this case the normative feedback is a pseudo non-interactive social engagement. Results typically show 1–3 % energy reduction in response to neighbor feedback comparison reports [5, 13]. In Hill’s [11] work the social comparison index was related to measures of sociability, self-monitoring, and public self-consciousness and not related to two empathy scales.

In this paper we argue that personality characteristics can play an independent and interactive role in user engagement with technologies to reduce energy consumption.

Previous research on advertising appeals suggests that persuasive appeals tend to be more effective when the nature of the appeal matches, rather than mismatches, individuals preferred processing style. Ruiz and Sicilia [14] showed that when individuals were exposed to ads congruent with their processing styles, in terms of affect and cognition, higher advertising effectiveness was obtained. Other experimental work on the persuasive effects of story, show that the magnitude of a person’s need for affect determined whether and to what extent the person’s experiences transportation into the story work and is persuaded by the information presented in the narrative.

The three technology prototypes designed for this study have been infused with user feedback beginning with ethnography [15], investigation of application images and graphic feedback of electricity use [8]. The results of the ethnography stimulated the creation of motivationally framed energy reduction applications that are affective, cognitive, and engage sociability.

2 Research Questions and Hypotheses

We posited one broad research question (RQ) and four primary research hypotheses (HYP). We used research questions to examine relationships among variables and assess questions that we had little or no theoretical reason to posit directional outcomes.

RQ: What are the relationships among demographic, outcome, and mediating variables?

a. What are the relationships among the three personality measures?

b. What are the relationships among the current behavior and future behavioral intentions?

c. What is the role of gender in current behavior, change in behavior change intentions and perceived barriers to behavior change?

HYP1: Motivationally framed technologies will not differ in user perceptions of application function, involvement and satisfaction from a control energy reduction technology but will differ from the control technology.

HYP2: Motivationally framed technologies will improve energy reduction behavioral intentions and number of behaviors changed more than a control technology.

HYP3: Personality characteristics, NFA, NFC, and NFS, will interact with their matched motivationally framed application, Kidogo, Powerbar, and Power tower respectively.

3 Methods

3.1 Research Design

We used a randomized controlled design where volunteer student participants were randomly assigned to one of five conditions (three treatments and two controls). Because of a problem in the second control where post survey measures were only collected for 17 participants, we examine here three energy reduction application conditions and one “traditional” control of a utility website.

3.2 Participants

Participants were 162 community college students who received course credit for were participation. Their average age was 23 years, 69 % were female, and with an average of two years of college courses completed. 39.3 % of students were Asian, 30.6 % were white, 15 % Hispanic, 4.6 % African American, and 9.8 % other demographic groups.

3.3 Procedures

Community college student study participants were part of a research experience program whereby they received credit for participating in university approved studies.

Prior to participation in this study, all student participants, as part of their credit for participating in research program answered a one hour long survey of a diverse array of questions, for this study NFA, NFC, and NFS questions. Participation in the current study after completion of the course required survey, took approximately 1.hour with ranges from 45 min to 1.5 h. Participants were sent a link to the Qualtrics platform which hosted the study. Once in Qualtrics, participants were randomly assigned to one of the five conditions. Upon assignment, they consented via clicking a link to a university approved IRB form, rated their current performance on and current intentions to perform 22 energy saving behaviors. They then saw a three minute video describing the energy reduction application (condition videos ranged from 2.8–3.2 min). Immediately after viewing the video, participants answered a series of questions on application stickiness, and intention to perform the same behaviors shown in the pretest questionnaire.

3.4 Study Treatment Materials

There were three motivationally framed energy reduction technologies; the affective application, Kidogo, used a stealth intervention strategy [16] whereby human engagement in emotions such as the altruistic desire to help global social entrepreneurs to succeed framed the behavior changes necessary to reduce energy. Kidogo turned the low cost of electricity and thus low financial return into more engaging values. The cognitively framed application, Powerbar, attempted to evoke the need of self-monitoring, interest in numeracy using graphs and journaling to engage users. The sociability framed application, Powertower, used a game structure, to engage users with one another in teamwork and competition, with behavior changes and electricity reduction yielding more “blocks” to build a stable and high tower. Embedded in all applications was the ability to collect real electricity data, to set goals, report behavior commitments, receive feedback, and receive reminders, and to “live” in the ambient Facebook environment (see Table 1 below).

Table 1. Images of each energy reduction application indicating key components of Kidogo, Powerbar and Powertower.
Full size table

There were three “App Treatment conditions,” a prototype – video produced for each of three “treatment” applications: affective, cognitive, and social motivationally framed energy reduction applications (Table 1 below). The applications are titled: Kidogo (Swalhi meaning a little bit), Powerbar, and Powertower.

To create the stimulus materials: first, we created screen shots of the current development status of the three energy applications, Table 1 below shows screen shots used in video and indicates the behavioral components embedded in the applications. Next, three minute videos with a voice over were created. Each voice over told an energy story relevant to each application; for example, in Kidogo, the narrator, states that she has a relative in one of the target social entrepreneur countries and will donate to the social entrepreneur from that country. The same person generated all of the voice overs for the videos. Previously we have published research that determined the most motivational images for Kidogo, and the most comprehensible graphs for Powerbar [8, 15]. In addition there was a control condition; a typical treatment; images of a typical utility graphing of electricity information. The video of the typical condition paralleled that of the three treatment applications.

3.5 Measures

There were three dependent measures, application stickiness, energy behavior intentions, energy behavior barriers, and based on theory and examination of variables related to the dependent variables there were four mediator variables, need for affect, cognition, sociability, current behavior, and gender.

Dependent measures. The primary dependent measures were application “stickiness” and energy reduction behavior intention change from pre to post-test.

The application stickiness questions were asked only at posttest and adapted from twenty two questions previously used to measure website stickiness [4]. The questions originally factored into seven factors, we used six of the factor questions which were modified from website to application (a total of thirteen questions were used): (1) five content questions (such as; The content of this application is; useful, complete, clear, accurate), (2) two context questions (This application looks organized, uses fonts properly), (3) two positive attitude questions (such as; This application has personal meaning to me, I like using this application), (4) two trust questions, (such as, I trust this application), (5) two infrastructure questions (such as, This energy application looks easy to navigate through), and (6) four sticky questions (such as, I intend to visit this application as often as I can, I intend to stay a longer time on this website than other websites). In addition we supplemented the measure with five additional questions: This energy application would motivate me to reduce my electricity consumption; I would invite others to visit this site, I would like to share personal experiences with energy reduction with users on this site, this energy application looks professional, I like this energy application. All questions were measured on a one to five scale with one being strongly disagree and five being strongly agree. Because the function, involvement and satisfaction questions all factored as a single scale, we combined them into a single index – overall app usability scale (Cronbach’s α = .95).

Energy Reduction Behavior Performance: Energy reduction behaviors were a set of 22 low cost, moderate to low kWh energy saving impact and easy to perform behaviors such as limit showers to less than 8 min, use “task” lighting instead of overhead, line dry clothes, set hot water temperature to 120 degrees F., shut windows/blinds in AM to capture cool air. At pretest only, participants also were asked on a 1–5 scale (never to all of the time) how often they currently performed each behavior. The 22 behaviors factored into a single index with a Cronbach’s α = .85.

Energy Reduction Behavioral Intention: At pre and posttest, participants were asked to rate each behavior on a 1–10 scale (not at all – absolutely will) which of the behaviors they intended to performed next week. The single factor behavioral intention measure had a Cronbach’s α = 92. at pretest and .94 at post-test.

Energy Reduction Behavior Barriers: At posttest, participants were also asked for each behavior, Which of the following would prevent you from performing each of the following actions? Participants could check all that apply of seven barriers; it is too hard, it costs money, I will not remember, it will make me uncomfortable, it will interfere with my lifestyle, I do not think it is necessary, and there are no barriers. The order of all questions within an index; stickiness, performance, and intention was randomized continuously at pretest and posttest.

Mediator measures. There were three hypothesized personality mediator variables; NFA, NFC and NFS.

Need for Affect: Need for Affect (NFA) questions were selected from a previously validated measure [9]. In the original studies there were two indices, approach affect and avoidance (negative affective). We used eight of the fourteen affect approach questions. We selected all questions that have a previously assessed factor score above .51. Individual predisposition to approach emotions questions included; It is important to me to be in touch with my feelings, I approach situations in which I expect to experience strong emotions, Emotions help people get along in life. Questions were measured on a −3 to a +3 (strongly disagree to strongly agree) scale. In our sample of participants, the NFA measure had a Cronbach’s α = 0.79.

Need for cognition: NFC was measured exactly as assessed in the original short form (18 questions) of the NFC Scale (NFC) by Caccippo and colleagues [6]. Questions such as, I would prefer complex to simple problems, Thinking is not my idea of fun, I find satisfaction in deliberating hard and for long hours were measured on a one-five Likert scale; 1 is extremely uncharacteristic of you (very much like you) to 5 extremely characteristic of you.

Need for Social Comparison: Social comparisons questions were from a five factor Affiliation Motivation Scale [10] In Hill’s original research social comparison (a five item subscale) had a Cronbach’s α = .71. Questions included I find that I often look to certain other people to see how I compare to others; I prefer to participate in activities alongside other people rather than by myself because I like to see how I am doing on the activity. Participants rate themselves on a 1–5 Likert scale, with 5 being not at all true to 1 being completely true.

Because the study took place over 3 quarters of a college school year, the personality measures did not get asked over summer quarter, thus our sample size on those measures was reduced by one third.

4 Results

We examined the distributions of gender, ethnicity, political ideology, age, and percent who reporting paying their own utility bill. There were no differences between the groups (Kidogo view group N = 47; PowerBar group, N = 38, PowerTower, N = 41, Utility website control, N = 36). There were also no significant differences between groups in NFA, NFC, NFS, mean current behavior performance, mean pre-test behavioral intention, indicating that randomization worked.

In RQ1a, we asked about the relationships among the three personality measures, NFA, NFC, and NFS. NFA and NFS were significantly correlated (r = .21; p < .01). In RQ1b we examined the relationships between the current behavior, pre and post behavioral intentions, and increase from pre-to-post behavioral intentions. Current behavioral performance and pre next week behavioral intention and post behavioral intentions, were significantly correlated (r = .56; p < .01; r = .62; p < .01, respectively). Pre and post intention were also significantly related (r = .86; p < .01) and pre-post intention increase was correlated with pre and post intentions (r = .20; p < .05; r = .27; p < .05, respectively).

RQ1c asked about the relationship between the personality measures and behavior. NFA was not significantly related to any behavior measures. NFC was significantly related to current behavioral performance, pre behavioral intention and number of no barriers (r = .17, p < .05; r = .16, p < .05, respectively). NFS was significantly negatively correlated to pretest behavior intentions and number of behaviors with no barriers (r = −.14, p < .05; r = −.16, p < .05 respectively.

RQ1d: What is the role of gender in current behavior, change in behavior change intentions and perceived barriers to behavior change? We found that women reported a higher mean of no barriers to behaviors (women X = 9.86, SD 5.58 versus men X = 7.84 SD 6.75) but that men made greater behavior change than did women.

We also note the usability index is unrelated to all other measures.

To test HYP 1, we used a one-way analysis of variance and found an overall significance (F = 3.325; p < .02). Contrast tests revealed significant differences in the overall application usability perception scale, with Kidogo (p < .03) and Powertower (p < .01) being significantly different from the control app. HYP1 was partially confirmed.

HYP 2 was not confirmed, there were no significant differences between applications in behavioral intention, number of behaviors with no barriers, and number of behaviors changed.

HYP 3 was also not confirmed none of the personality characteristics, NFA, NFC, and NFS, significantly interacted with their matched application. However, in an analysis of covariance controlling for pre-test behavioral intention, there was an independent effect for NFA on post behavioral intention (p < .05). In addition, in the Kidogo condition, participants with higher NFA increased more behaviors in the post test (r = .24, p < .10).

There was also an independent effect of gender on the mean number of behaviors increased from pre to post, with men changing more behaviors. The interaction of NFC and gender in the post measure of mean number of behaviors with no barriers; women and those with high NFC perceived more no barriers to behavior (p < .02 and p < .009 respectively).

5 Discussion and Future Research

In this randomized controlled test of motivationally framed energy reduction applications, we find differences among applications on a multi-faceted measure of application usability, with both the affective and sociability technology being significantly different from the control condition. However, there were no differences between conditions in behavioral intention and number of behaviors increased.

Notably, the usage perception measures were unrelated to change in behavior intentions, raising the question of the utility of usability measures such as application function and involvement as primary outcomes of tests of online energy reduction applications. More research is needed to examine the role of usability measures as interim assessments of technologies and their ability to assess engagement and impact.

Personality measure results did not show the predicted interactive effect with their “matched” technology. However, there are indications that both NFA and NFC need further research to determine and clarify their role in profiling and matching human motivations to technological frames. NFA did have an independent effect on post behavioral intention. NFC was correlated significantly with current behavior performance and pre-test behavioral intentions. NFS was negatively correlated with pre-behavior intentions. Previously, we were able to see marginal interactive effects of NFA and Kidogo and here within the Kidogo condition, NFA was significantly correlated with the number of behaviors increased from pre-posttest.

Gender also played an independent role in application behavioral intention, with males showing more positive increase in behavioral intentions.

Individual motivational personality measures assessed here intriguingly point to the need to assess individual personality factors related to energy reduction technology effectiveness.

The lack of effects of the applications to differentially influence behavioral intentions is noteworthy. While we used an RCT design, touted as a means of developing theory within the HCI field, the measurement, time span, selection of controls, and audience are obviously critical features of a robust design [17]. Our results do not allow us to tease out the role of these additional considerations of technology prototyping. However, the loss of already small samples per condition (loss from no personality variables from the summer sample) deterred out ability to detect significant interactions. Further, a no or other topic control may be a better option in the future. Finally, this online study would have benefited from a delayed post assessment. While logistics, loss of sample, and costs are considerations, a one-two week delay of the posttest would have allowed participants time to think and consider what they are being asked and how they want to respond.

We conclude by pointing out that while there few predicted significant results, there are results that are in predicted directions and small and consistent results that indicate the potential for fruitful investigation of personality based motivations to spur investigating tailored technology interventions.