Abstract
Managers must constantly scan a sea of new information and judge (1) if this information is relevant to evaluating the firm’s strategy (i.e. information relevance judgments), and (2) if this information suggests the firm’s strategy is appropriate or inappropriate (i.e. strategy appropriateness judgments). I examine these judgments in a behavioral experiment with Amazon Mechanical Turk workers. Replicating prior research, I test how these judgments are affected by formatting the firm’s strategy to include cause–effect linkages, a defining feature of strategic performance measurement systems. I also add a manipulation of causal relatedness, i.e. the subjective probability of a given cause–effect linkage, especially as it relates to the inferred bridging mediators that can logically connect a cause–effect linkage. Like prior research, I find that adding cause–effect linkages improves managers’ information relevance judgments, specifically improving managers’ ability to filter out irrelevant information. Extending prior research, I also find that cause–effect linkages only improve strategy appropriateness judgments when those linkages have high causal relatedness. Experience also moderates this latter improvement such that it is limited to those with relatively low experience.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
Data available upon request.
Notes
My instrument is framed using balanced scorecard/strategy map language because of its popularity. However, my experimental design manipulates features that I expect to generalize to most if not all SMPSs.
As discussed in Sect. 3, I manipulate causal relatedness by presenting strategic goals in goal-pairs that either come from adjacent (high causal relatedness) or non-adjacent (low causal relatedness) balanced scorecard categories. The defining nature of this manipulation is that non-adjacent goal-pairs are more distant and thus a bridging mediator cannot easily be inferred, making the linkage seems unlikely, while adjacent goal-pairs are less distant and thus a bridging mediator can easily be inferred, making the linkage seems likely.
Or, also important, this new information can increase management’s confidence in the firm’s current strategy.
The brain converts current cognitions into constructs that are encoded in memory by neurally categorizing and connecting them to other memories (e.g. Nickerson and Adams 1979; Anderson 1974; Shiffrin and Atkinson 1969; see also Otmakhova et al. 2013; Bliss and Collingridge 1993). Before examining the effect of an SPMS on memory, it is important to distinguish between working memory and long-term memory. Working memory is a temporary cache of recent cognitions that allows one to easily recall recent thoughts. Long-term memory, in contrast, is the collective body of cognitions that have been neurally categorized and connected with other memories. These categories and connections with other memories serve as cues to allow the encoded cognitions to be recalled in the future. Encoding a cognition in memory effectively means encoding it in long-term memory.
To focus their experiment on long-term memory, Cheng and Humphreys (2012) give participants a distractor task that clears their working memory before participants make information relevance judgments and strategy appropriateness judgments. Cheng and Humphreys then present participants with summaries of six fictional articles, which are construed as equivalent to articles from a reputable business journal. Participants first determine if these article summaries are relevant to the task of evaluating the firm’s strategy. Then, for articles that participants consider relevant, they determine whether the article summaries suggest the firm’s strategy is appropriate or inappropriate for the current business environment.
For my purposes, I treat causal relatedness as being effectively synonymous with cause−effect linkage strength (see Webb 2004); I use the term causal relatedness instead of the term cause−effect linkage strength because the former term has an associated literature with rich predictions to help in examining boundary conditions for Cheng and Humphreys’ (2012) results. Prior literature’s description of cause−effect linkage strength does clearly point to it also being synonymous with the subjective probability of a given cause−effect relationship. Webb (2004) says it is how much the cause goal “should lead to” the effect goal (Webb 2004, p. 941; see also King 2007 who reviews a related idea: causal ambiguity). Strong cause−effect linkages “are perceived as being plausible and informative” (p. 929) and weak cause−effect linkages are “questionable” (p. 939). Luft (2004) considers strong linkages “highly probable” and weak linkages “dubious” (p. 959).
Background knowledge refers to knowledge already encoded in memory that can be relevant to the business environment. It is not necessarily technical knowledge or knowledge that is only acquired from specialized training or education. Some background knowledge is gained from common life experiences over time, so background knowledge should be correlated with general experience.
Or, as an example from the earlier sentences about Joey, what if one knew that Joey and his neighbor are close friends who often play rough and tumble games of football after school? If so, one would likely be able to narrow down the possible bridging mediators for Sentence Pair 2 and read it as a sentence pair with high causal relatedness.
My instrument is original, but I designed it to be comparable to Cheng and Humphreys’ (2012). I create an original scenario, original strategic goals, an original distractor task, and original article summaries. Key similarities that I have held constant include the general pattern of the experiment (i.e. learn goals, complete distractor task, and then make judgments about new information), instructions about the key between-subject manipulation, and nearly identical phrasing of my dependent variable questions.
Adding these instructions to the linkages condition is consistent with similar explicit instructions provided by Cheng and Humphreys (2012).
I only create goal-pairs using goals from the same strategic theme, and no strategic goal or goal-pair is presented more than once.
I randomize the order of goal-pairs as a pair. Individual strategic goals within a goal-pair are always presented in order based on the typical balanced scorecard/strategy map order: the earlier category is shown first and the latter category is shown second. Strategic themes play no role in randomizing goal-pair presentation order: the two goal-pairs created by a single strategic theme could just as easily be presented completely separate, with one presented at the beginning of the firm’s strategy and the other presented at the end of the firm’s strategy.
When I add an indicator variable that reflects this counterbalancing to the main hypothesis tests for my six hypotheses, it does not materially change the direction or significance of the coefficients that test those hypotheses. The indicator variable itself is not significant when added to my test of H1a, H2a, or H3a (see regressions in Table 2 Panels A and B; Table 3 Panels A and B; and Table 4 Panel A; p > 0.40 two-tailed). However, the indicator variable is significant when added to my test of H1b (see regression in Table 2 Panel C; coefficient = 0.14, p = 0.02 two-tailed), H2b (see regression in Table 3 Panel C; coefficient = 0.14, p = 0.02 two-tailed), and H3b (see regression in Table 4 Panel B; coefficient = 0.13, p = 0.03 two-tailed).
These attention checks involve no deception. I am deliberately straightforward about the correct answer participants should choose. Thus, if a participant misses an attention check, he or she is very unlikely to be paying sufficient attention to provide meaningful responses.
To test these screening protocols, I repeat my hypothesis tests (see hypothesis tests in Tables 2, 3, and 4) without these screening protocols. First, I remove IP address screening (n = 352) and find results that are almost entirely unchanged in terms of direction and significance (see Sect. 4). The only change is that my test for H1a is now significant at the 95% confidence level (linkages: p = 0.046 one-tailed). Second, I remove all screening criteria (n = 371). My H1a result is again marginally significant (linkages: p = 0.0503 one-tailed), while the other hypothesis tests are unchanged in direction and significance.
In their experimental instrument, Cheng and Humphreys (2012) explicitly describe strategy maps as portraying cause−effect relationships between strategic goals (to participants who receive a strategy map). They also give strategy map participants check questions that further emphasize how a strategy map suggests cause−effect relationships between strategic goals.
I present both questions to all participants, but I instruct them to select “N/A” for the strategy appropriateness judgment if they rate an article summary’s relevance as zero. Participants cannot advance to the next article summary if they indicate the article summary is irrelevant and select anything other than “N/A.” They also cannot advance if they select “N/A” while indicating the article summary is anything other than irrelevant. Cheng and Humphreys (2012) similarly instruct participants who rate the information as irrelevant to move onto the next question without rendering an appropriateness judgment. This is reasonable, since appropriateness judgments about information that one perceives to be irrelevant would be difficult to interpret.
For example, assume there is an irrelevant article summary, and the most correct answer is zero, meaning irrelevant. If a participant answers this item with a four, meaning highly relevant, I reverse code that participant’s response as a zero (i.e. 4 − 4 = 0). This reflects how the response is as distant from the correct answer as possible.
I test these fixed effects’ effectiveness at removing noise by repeating my hypothesis tests from Table 2 through Table 4without this set of variables included in the model. All results reported in Tables 2 through 4 remain the same in terms of direction and significance except that the coefficient that tests H2b is only marginally significant (linkages × causal relatedness: p = 0.09 one-tailed, compare with Table 3 Panel C).
I do not expect this manipulation check to be influenced by the fact that some goal-pairs have high causal relatedness and others have low causal relatedness. First, because the mixture of causal relatedness is consistent for all participants: half of the goal-pairs are high and half of the goal-pairs are low. Second, because the question explicitly asks whether the participant believes the firm expects a cause−effect relationship, not whether the participants themselves could interpret a cause−effect relationship between the goal-pairs previously presented to them.
I also record the number of business classes participant have taken . I do not include this in my analysis because I already use a more informative control variable: whether participants have a business degree.
With one exception, none of these variables differs significantly between conditions of the SPMS cause−effect linkage manipulation (p > 0.10 one-tailed in either direction). The exception is participants’ familiarity with strategy maps, which differs in a marginally significant fashion (p = 0.053, one-tailed), suggesting that participants in the linkages condition are more familiar with strategy maps (mean = 1.15) than participants in the no linkages condition (mean = 0.95). This makes sense because I describe the firm’s SPMS as a “strategy map” in the linkages condition.
Interestingly, the simple effects tests for this interaction are non-significant. I find non-significant linkages coefficients in both the low causal relatedness subsample (coefficient = −0.09, p = 0.23 two-tailed) and the high causal relatedness subsample (coefficient = 0.09, p = 0.105 one-tailed). However, this may be due to noise from erroneous strategy appropriateness judgments. That is, participants who incorrectly rate an irrelevant article summary as greater than zero relevance likely provide noisy and difficult-to-interpret judgments about the strategy appropriateness implications of that irrelevant information item. When I only consider simple effects of responses related to relevant article summaries (n = 1,152), I find a non-significant linkages coefficient in the low causal relatedness subsample (coefficient = −0.11, p = 0.25 two-tailed) and a significant linkages coefficient in the high causal relatedness subsample (coefficient = 0.15, p = 0.04 one-tailed). This latter simple effects test supports H2b as reasoned in the hypothesis development section. This noise-reduction simplification (of only considering strategy appropriateness judgments of relevant article summaries) has no effect on the direction or non-significance of my results for H1b, but it makes the three-way interaction from my later H3b results significant at the 95% confidence level (p < 0.01 one-tailed).
Choi et al. (2012, 2013) even find that participants substitute performance measures for strategic goals, interpreting these measures of progress toward strategic goals as substitutes for the strategic goals themselves.
The phrase Chesterton’s fence refers to a quotation from G. K. Chesterton where he compares societal institutions, norms, or laws to a fence that one encounters along a road—a fence that one does not know the purpose of (Chesterton 1930). Chesterton cautions against tearing down such fences without understanding why it was there in the first place: it likely serves a function significant enough to justify its construction. Or, by way of another analogy, one should hardly try renovating a home by simply removing walls he or she does not like. Some of those walls are likely to be load bearing. Likewise, practitioners might look at experimental results—which are constrained by the figurative fences of the experiment’s context—and think they should easily translate to real world results. They are unlikely to be aware of which contextual features were preconditions for the experimental results unless replications examine these preconditions directly.
References
Anderson, J. R. (1974). Retrieval of propositional information from long-term memory. Cognitive Psychology, 6(4), 451–474.
Banker, R. D., Chang, H., & Pizzini, M. J. (2004). The balanced scorecard: Judgmental effects of performance measures linked to strategy. The Accounting Review, 79(1), 1–23.
Bisbe, J., & Malagueño, R. (2012). Using strategic performance measurement systems for strategy formulation: Does it work in dynamic environments? Management Accounting Research, 23(4), 296–311.
Bliss, T. V., & Collingridge, G. L. (1993). A synaptic model of memory: Long-term potentiation in the hippocampus. Nature, 361(6407), 31.
Capelo, C., & Dias, J. F. (2009). A system dynamics-based simulation experiment for testing mental model and performance effects of using the balanced scorecard. System Dynamics Review, 25(1), 1–34.
Chen, Y., Jermias, J., & Panggabean, T. (2016). The role of visual attention in the managerial judgment of balanced-scorecard performance evaluation: Insights from using an eye-tracking device. Journal of Accounting Research, 54(1), 113–146.
Cheng, M. M., & Humphreys, K. A. (2012). The Differential improvement effects of the strategy map and scorecard perspectives on managers’ strategic judgments. The Accounting Review, 87(3), 899–924.
Chesterton, G. K. (1930). The Thing: Why I am a Catholic. Dodd, Mead & Company.
Corneille, O., Goldstone, R. L., Queller, S., & Potter, T. (2006). Asymmetries in categorization, cerceptual discrimination, and visual search for reference and nonreference exemplars. Memory and Cognition, 34(3), 556–567.
Day, G. S., & Schoemaker, P. J. (2005). Scanning the periphery. Harvard Business Review, 83(11), 135.
Demidenko, E. (2004). Mixed models: Theory and applications. Wiley Series in probability and statistics. NewYork: Wiley.
Duffy, S. A., Shinjo, M., & Myers, J. L. (1990). The effect of encoding task on memory for sentence pairs varying in causal relatedness. Journal of Memory and Language, 29(1), 27–42.
Farrell, A. M., Grenier, J. H., & Leiby, J. (2017). Scoundrels or stars? Theory and evidence on the quality of workers in online labor markets. The Accounting Review, 92(1), 93–114.
Frank, M. C., Bergelson, E., Bergmann, C., Cristia, A., Floccia, C., Gervain, J., et al. (2017). A collaborative approach to infant research: Promoting reproducibility, best practices, and theory-building. Infancy, 22(4), 421–435.
Gimbert, X., Bisbe, J., & Mendoza, X. (2010). The role of performance measurement systems in strategy formulation processes. Long Range Planning, 43(4), 477–497.
Jensen, M. C., & Meckling, W. H. (1992). Specific and general knowledge and organizational structure. Knowledge Management and Organizational Design, 8(2), 17–18.
Jensen, M. C., & Meckling, W. H. (1999). Specific knowledge and divisional performance measurement. Journal of Applied Corporate Finance, 12(2), 8–17.
Kaplan, R. S., & Norton, D. P. (2008). The execution premium: linking strategy to operations for competitive advantage. Harvard: Harvard Business Press.
King, A. W. (2007). Disentangling interfirm and intrafirm causal ambiguity: A conceptual model of causal ambiguity and sustainable competitive advantage. Academy of Management Review, 32(1), 156–178.
King, A. W., & Zeithaml, C. P. (2001). Competencies and firm performance: Examining the causal ambiguity paradox. Strategic Management Journal, 22(1), 75–99.
Leichsenring, F., Abbass, A., Hilsenroth, M. J., Leweke, F., Luyten, P., Keefe, J. R., et al. (2017). Biases in research: Risk factors for non-replicability in psychotherapy and pharmacotherapy research. Psychological Medicine, 47(6), 1000–1011.
Libby, R., Bloomfield, R., & Nelson, M. W. (2002). Experimental research in financial accounting. Accounting, Organizations and Society, 27(8), 775–810.
Luft, J. (2004). Discussion of: Managers’ commitment to the goals contained in a strategic performance measurement system. Contemporary Accounting Research, 21(4), 959–964.
Markman, A. B., & Wisniewski, E. J. (1997). Similar and different: The differentiation of basic-level categories. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23(1), 54.
Mason, R. A., & Just, M. A. (2004). How the brain processes causal inferences in text a theoretical account of generation and integration component processes utilizing both cerebral hemispheres. Psychological Science, 15(1), 1–7.
Mason, W., & Suri, S. (2012). Conducting behavioral research on Amazon’s mechanical turk. Behavior Research Methods, 44(1), 1–23.
Maxwell, S. E., Delaney, H. D., & Kelley, K. (2017). Designing experiments and analyzing data: A model comparison perspective. London: Routledge.
Mintzberg, H., & Waters, J. A. (1985). Of strategies, deliberate and emergent. Strategic Management Journal, 6(3), 257–272.
Myers, J. L., Shinjo, M., & Duffy, S. A. (1987). Degree of causal relatedness and memory. Journal of Memory and Language, 26(4), 453–465.
Nickerson, R. S., & Adams, M. J. (1979). Long-term memory for a common object. Cognitive Psychology, 11(3), 287–307.
Niven, P. R. (2006). Balanced scorecard step-by-step: Maximizing performance and maintaining results (2nd ed.). New York: Wiley.
Otmakhova, N., Duzel, E., Deutch, A. Y., & Lisman, J. (2013). The hippocampal-VTA loop: The role of novelty and motivation in controlling the entry of information into long-term memory. Intrinsically motivated learning in natural and artificial systems (pp. 235–254). Berlin Heidelberg: Springer.
Pennington, N., & Hastie, R. (1992). Explaining the evidence: Tests of the story model for juror decision making. Journal of Personality and Social Psychology, 62(2), 189.
Shiffrin, R. M., & Atkinson, R. C. (1969). Storage and retrieval processes in long-term memory. Psychological Review, 76(2), 179.
Stock, W. A., Robinson, D. H., & Pridemore, D. R. (1991). The effects of evidence statements in a passage on the acceptability of bridging inferences. Contemporary Educational Psychology, 16(3), 254–264.
Towry, K. L. (2012). Discussion of: Subordinates as the first line of defense against biased financial reporting. Journal of Management Accounting Research, 24(1), 25–30.
Tsang, E. W., & Kwan, K. M. (1999). Replication and theory development in organizational science: A critical realist perspective. Academy of Management Review, 24(4), 759–780.
Virtue, S., Haberman, J., Clancy, Z., Parrish, T., & Beeman, M. J. (2006). Neural activity of inferences during story comprehension. Brain Research, 1084(1), 104–114.
Webb, R. (2004). Managers’ commitment to the goals contained in a strategic performance measurement system. Contemporary Accounting Research, 21(4), 925–958.
Zwaan, R. A., & Radvansky, G. A. (1998). Situation models in language comprehension and memory. Psychological Bulletin, 123(2), 162.
Acknowledgements
This study is based on my dissertation. I acknowledge and thank my dissertation committee for their guidance and support: Don Moser (chair), Willie Choi, Marc Coutanche, Harry Evans, and Dhinu Srinivasan. I also thank Mandy Cheng and Kerry Humphreys, who provided their experimental instrument for reference. This study also benefits from comments by Kristy Towry, Paul Fischer, Chaoping (Dylan) Li, Weiming Liu, and James Wilhelm at the 2016 AAA/Deloitte Foundation/J. Michael Cook Doctoral Consortium and from comments by discussants (Todd Thornock and Rachel Martin) and participants at the 2017 Brigham Young University Accounting Research Symposium. I thank the editors and anonymous reviewers for their time and effort.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Knox, B.D. A replication about cause–effect linkage benefits and managers’ strategic judgments. J Manag Control 32, 225–251 (2021). https://doi.org/10.1007/s00187-021-00315-6
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00187-021-00315-6