Skip to main content
Log in

Assessment of the cross-national validity of an End-anchored 9-point hedonic product liking scale

  • Published:
Quality & Quantity Aims and scope Submit manuscript

Abstract

An end-anchored 9-point hedonic product liking (PL) scale is an easy-to-apply instrument to examine consumers’ PL. Because 9-point hedonic PL scales are also popular in cross-national research, strong demands are put on its cross-national equivalence, that is, the absence of cross-national scoring bias. The present study provides a procedure to identify the presence of cross-national scoring bias in the use of the end-anchored 9-point hedonic PL scale or any other rating scale to measure PL. The procedure is illustrated on experimental (illustrative) data from students in four European nations (i.e., Belgium, Germany, the Netherlands, and Spain). It explores cross-national equivalence in terms of (1) mean PL scores, (2) variation in PL scores, and (3) the impact of cross-cultural scoring bias in statistical inference making. Data analyses revealed that cross-national scoring bias only affected the variability in participants’ PL scores, but not the level of their PL scores. However, cross-national scoring bias in variation in PL scores did exert a substantial influence on a statistical inference making of mean PL scores. In sum, this study (1) provides preliminary evidence that cross-national scoring bias may seriously hamper the validity of cross-national comparisons of PL scores; and (2) offers a new methodology allowing food researchers to assess the extent to which the amount of cross-national scoring bias in their PL data will result in invalid cross-national comparisons.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. The remainder of this paper refers to ‘nations’ and ‘cross-national’ differences in PL but the reader may also interpret these terms as ‘cultures’ (e.g., within or across nations) and ‘cross-cultural’ differences in PL.

  2. Although six studies might seem a small number, these are only studies that appeared in scientific journals, and do not include “classified studies” undertaken by food companies.

  3. The validity of these data depends on random distribution of (forced) preference in cases in which two products are liked equally. Obviously, in the (occasional) event that forced choice would systematically favor one product in a subset of products that are actually not differing in the degree of product liking, the validity of forced choice comes under scrutiny.

  4. Instead of the range (i.e., maximum minus the minimum), one may prefer to work with the standard deviation of the number of times a product (A, B, C, and D) is preferred across all pairwise comparisons. Additional analyses (not shown to save journal space) have shown that, after switching from the range to the standard deviation, significance of effects as shown in Tables 2 and 3 did not change. This is not surprising given that the mutual correlation between these alternative measures of variability amounted to .994.

    Table 2 Two-level hierarchical regression model predicting the (hedonic) liking score: Unstandardized regression results (768 product evaluations nested within 192 participants)
    Table 3 Two-level hierarchical regression model predicting within-participant variation in (hedonic) liking scores: Unstandardized regression results (768 product evaluations nested within 192 participants)
  5. We acknowledge that some cross-cultural researchers (e.g., Smith 2004) defend the view that cross-national differences in scoring bias represent cross-national differences in communication styles and, as such, should be considered “real” (i.e., wanted) cultural differences. We do not contest that cross-national differences in scoring behavior may represent cross-national differences in communication styles, but, as far as consumer research on product quality is concerned, the effect of cross-national differences in communication styles on key product quality indicators such as product liking are to be neutralized. Actually, differences in communication styles do not make a positive contribution to obtaining “objective” information on the quality of alternative products.

References

  • Cheung, G.W., Rensvold, R.B.: Assessing extreme and acquiescence response sets in cross-cultural research using structural equations modeling. J. Cross Cult. Psychol. 31, 187–212 (2000)

    Article  Google Scholar 

  • Cordonnier, S.M., Delwiche, J.F.: An alternative method for assessing liking: positional relative rating versus the 9-point hedonic scale. J. Sens. Stud. 23, 284–292 (2008)

    Article  Google Scholar 

  • Cox, D.N., Clark, M.R., Mialon, V.S.: A cross-cultural methodological study of the uses of two common hedonic response scales. Food Qual. Prefer. 12, 119–131 (2001)

    Article  Google Scholar 

  • Davidov, E., De Beuckelaer, A.: How harmful are survey translations? A test with Schwartz’s human values instrument. Int. J. Public Opin. Res. 22, 485–510 (2010)

    Article  Google Scholar 

  • De Beuckelaer, A.: A closer examination on some parametric alternatives to the ANOVA F-test. Stat. Pap. 37, 291–305 (1996)

    Article  Google Scholar 

  • De Beuckelaer, A., Kampen, J.K., Van Trijp, J.C.M.: An empirical assessment of the cross-national measurement validity of graded paired comparisons. Qual. Quant. 47, 1063–1076 (2013)

    Article  Google Scholar 

  • De Jong, M.G., Steenkamp, J.-B.E.M., Fox, J.-P., Baumgartner, H.: Using item response theory to measure extreme response style in marketing research: a global investigation. J. Mark. Res. 45, 104–115 (2008)

    Article  Google Scholar 

  • De Maeyer, S., Coertjens, L., Ardies, J.: Bivariate en Multivariate Statistiek met R. Een Open Leerpakket. [Bivariate and Multivariate Statistics with R: An Open Learning Package]. Academia Press, Ghent, Belgium (2012)

  • De Mooij, M.: The future is predictable for international marketers: converging incomes lead to diverging consumer behavior. Int. Mark. Rev. 17, 103–113 (2000)

    Article  Google Scholar 

  • Druz, L.L., Baldwin, R.E.: Taste thresholds and hedonic responses of panels representing three nationalities. J. Food Sci. 47, 561–563 (1982)

    Article  Google Scholar 

  • Harzing, A.: Response styles in cross-national survey research: a 26-country study. Int. J. Cross Cult. Manag. 6, 243–266 (2006)

    Article  Google Scholar 

  • Ingenbleek, P.T.M., Tessema, W.K., Van Trijp, H.C.M.: Conducting field research in subsistence markets, with an application to market orientation in the context of Ethiopian pastoralists. Int. J. Res. Mark. 30, 83–97 (2013)

    Article  Google Scholar 

  • Jaeger, S.R., Cardello, A.V.: Direct and indirect hedonic scaling methods: a comparison of the labeled affective magnitude (LAM) scale and best-worst scaling. Food Qual. Prefer. 20, 249–258 (2009)

    Article  Google Scholar 

  • Johnson, T., Kulesa, P., Cho, Y.I., Shavitt, S.: The relation between culture and response styles: evidence from 19 countries. J. Cross Cult. Psychol. 36, 264–277 (2005)

    Article  Google Scholar 

  • Kreft, I., De Leeuw, J.: Introducing Multilevel Modeling. Sage, London (1998)

    Book  Google Scholar 

  • Lawness, H.T., Malone, G.J.: The discriminative efficiency of common scaling methods. J. Sens. Stud. 1, 85–98 (1986)

    Article  Google Scholar 

  • Lawness, H.T., Popper, R., Kroll, B.J.: A comparison of the labeled magnitude (LAM) scale, an 11-point category scale and the traditional 9-point hedonic scale. Food Qual. Prefer. 21, 2–12 (2010)

    Google Scholar 

  • Muthén, L.K., Muthén, B.O.: Mplus User’s Guide, 6th edn. Muthén Muthén, Los Angeles (1998–2010)

  • Nicolas, L., Marquilly, C., O’Mahony, M.: The 9-point hedonic scale: are words and numbers compatible? Food Qual. Prefer. 21, 1008–1015 (2010)

    Article  Google Scholar 

  • O’Donnell, S., Jeong, I.: Marketing standardization within global industries: an empirical study of performance implications. Int. Mark. Rev. 17, 19–33 (2000)

    Article  Google Scholar 

  • Peryam, D.R., Pilgrim, F.J.: Hedonic scale method of measuring food preferences. Food Technol. 11, 9–14 (1957)

    Google Scholar 

  • Prescott, J., Bell, G.A.: Cross-cultural determinants of food acceptability: recent research on sensory perceptions and preferences. Trends Food Sci. Technol. 6, 201–205 (1995)

    Article  Google Scholar 

  • Prescott, J., Bell, G.A., Gillmore, R., Yoshida, M., O’Sullivan, M., Korac, S., Allen, S., Yamazaki, K.: Cross-cultural comparisons of Japanese and Australian responses to manipulations of sweetness in foods. Food Qual. Prefer. 8, 45–55 (1997)

    Article  Google Scholar 

  • Reynolds, N.L., Simintiras, A.C., Diamantopoulos, A.: Theoretical justification of sampling choices in international marketing research: key issues and guidelines for researchers. J. Int. Bus. Stud. 34, 80–89 (2003)

    Article  Google Scholar 

  • Samiee, S., Roth, K.: The influence of global marketing standardization on performance. J. Mark. 56, 1–17 (1992)

    Article  Google Scholar 

  • Smith, P.B.: Acquiescence response bias as an aspect of cultural communication style. J. Cross Cult. Psychol. 35, 50–61 (2004)

    Article  Google Scholar 

  • Suknark, K., McWatters, K.H., Phillips, R.D.: Acceptance by American and Asian consumers of extruded fish and peanut snack products. J. Food Sci. 63, 721–725 (1998)

    Article  Google Scholar 

  • Van De Vijver, F.J.R., Leung, K.: Methods and Data Analysis for Cross-Cultural Research. Sage, London (1997)

    Google Scholar 

  • Van Herk, H., Poortinga, Y.H., Verhallen, T.M.M.: Response styles in rating scales: evidence of method bias in data from six EU countries. J. Cross Cult. Psychol. 35, 346–360 (2004)

    Article  Google Scholar 

  • Yao, E., Lim, J., Tamaki, K., Ishii, R., Kim, K.O., O’Mahony, M.: Structured and unstructured 9-point hedonic scales: a cross cultural study with American, Japanese, and Korean consumers. J. Sens. Stud. 18, 115–139 (2003)

    Article  Google Scholar 

  • Yeh, L.L., Kim, K.O., Chompreeda, P., Rimkeeree, H., Yau, N.J.N., Lundahl, D.S.: Comparison in use of the 9-point hedonic scale between Americans, Chinese, Koreans, and Thai. Food Qual. Prefer. 9, 413–419 (1998)

    Article  Google Scholar 

Download references

Acknowledgments

The authors thank Lisa Trierweiler for the English proofreading of this paper.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Alain De Beuckelaer.

Appendices

Appendix 1

See Table 5.

Table 5 Forty-eight unique questionnaires

Appendix 2

Formal specifications of two-level hierarchical regression models

1.1 Exhibit 1

Model 1: Prediction of PL scores

Level 1 model:

$$\begin{aligned} \hbox {PL}_{\mathrm{ij}} = \hbox {PL}_{\mathrm{j} }+ {\ss }_{\mathrm{1j}} \hbox {ProductB}_{\mathrm{ij} }+ {\ss }_{\mathrm{2j}} \hbox {ProductC}_{\mathrm{ij} }+ {\ss }_{\mathrm{3j}} \hbox {ProductD}_{\mathrm{ij}} + {\ss }_{\mathrm{4j}} \hbox {ProductDominance}_{\mathrm{ij}}+{\upvarepsilon }_{\mathrm{ij}} \end{aligned}$$

with

\(\hbox {PL}_{\mathrm{ij}}\) :

The expected PL score for a given participant j (\(\hbox {j}=1,2,3,\ldots ,192\)) for product i

\(\hbox {PL}_{\mathrm{j}}\) :

A baseline PL score for the Level 1 model: expresses the expected PL score for a given participant (with particular characteristics; see Level 2 model) who: (1) evaluates product A; and (2) never selects the product (A) as the preferred one in a pairwise comparison

\({\ss }_{\mathrm{1j}}\) :

An adjustment for evaluating product B

\({\ss }_{\mathrm{2j}}\) :

An adjustment for evaluating product C

\({\ss }_{\mathrm{3j}}\) :

An adjustment for evaluating product D

\({\ss }_{\mathrm{4j}}\) :

An adjustment to be made each time the product is indicated as the preferred one in a pairwise comparison

\({\upvarepsilon }_{\mathrm{ij}}\) :

The residual variance for this evaluation i

Level 2 model:

$$\begin{aligned} \hbox {PL}_{\mathrm{j}}={\upgamma }_{00}+{\upgamma }_{01} \hbox {Belgium}_{\mathrm{j}}+{\upgamma }_{02} \hbox {Germany}_{\mathrm{j}}+{\upgamma }_{03} \hbox {Spain}_{\mathrm{j}} + {\upgamma }_{04} \hbox {Age}_{\mathrm{j}}+{\upgamma }_{05} \hbox {Female}_{\mathrm{j}}+ \hbox {u}_{\mathrm{j}} \end{aligned}$$

with

\(\hbox {PL}_{\mathrm{j}}\) :

The expected PL score for a given participant who evaluates product A

\({\upgamma }_{00}\) :

A baseline PL score for the Level 2 model: expresses the expected PL score for product A being evaluated by: (1) a participant (j) from the Netherlands; (2) who is male; and (3) who has not reached the age of one year (i.e., a zero baseline)

\({\upgamma }_{01}\) :

An adjustment for Belgian participants

\({\upgamma }_{02}\) :

An adjustment for German participants

\({\upgamma }_{03}\) :

An adjustment for Spanish participants

\({\upgamma }_{04}\) :

An adjustment for every year added to one’s baseline age (zero)

\({\upgamma }_{05}\) :

An adjustment for being female

\(\hbox {u}_{\mathrm{j}}\) :

The residual variance for this participant j

1.2 Exhibit 2

Model 2: Prediction of within-participant variation in PL scores

Level 1 model:

$$\begin{aligned} \hbox {Var}(\hbox {PL}_{\mathrm{ij}}) = \hbox {BaseVar}(\hbox {PL}_{\mathrm{j}})+ {\ss }_{\mathrm{1j}} \hbox {ProductB}_{\mathrm{ij}} + {\ss }_{\mathrm{2j}} \hbox {ProductC}_{\mathrm{ij}} + {\ss }_{\mathrm{3j}} \hbox {ProductD}_{\mathrm{ij}}+{\upvarepsilon }_{\mathrm{ij}} \end{aligned}$$

with

\(\hbox {Var}(\hbox {PL}_{\mathrm{ij}})\) :

Within-individual variation as determined for participant j (\(\hbox {j}=1,2,3,\ldots ,192\)) when evaluating product i (i.e., the absolute deviation between that product’s PL score and the mean PL score for all four products in the test)

\(\hbox {BaseVar}(\hbox {PL}_{\mathrm{j}})\) :

A baseline score for the Level 1 model: expresses the expected within-participant variation of the PL scores given by a certain participant (with particular characteristics; see Level 2 model) who evaluates product A.

\({\ss }_{\mathrm{1j}}\) :

An adjustment for evaluating product B

\({\ss }_{\mathrm{2j}}\) :

An adjustment for evaluating product C

\({\ss }_{\mathrm{3j}}\) :

An adjustment for evaluating product D \({\upvarepsilon }_{\mathrm{ij}}\)] The residual variance for this evaluation i

Level 2 model:

$$\begin{aligned} \hbox {BaseVar}(\hbox {PL}_{\mathrm{j}})&= {\upgamma }_{00}+{\upgamma }_{01}\hbox {Belgium}_{\mathrm{j}}+{\upgamma }_{02} \hbox {Germany}_{\mathrm{j}}+{\upgamma }_{03}\hbox {Spain}_{\mathrm{j}}+{\upgamma }_{04} \hbox {Age}_{\mathrm{j}}\\&+{\upgamma }_{05}\hbox {Female}_{\mathrm{j}}+{\upgamma }_{06} \hbox {VariationinProductDominance}_{\mathrm{j}} + \hbox {u}_{\mathrm{j}} \end{aligned}$$

with

\(\hbox {BaseVar}(\hbox {PL}_{\mathrm{j}})\) :

The expected within-participant variation in the PL score for a given participant who evaluates product A

\({\upgamma }_{00}\) :

A baseline PL score for the Level 2 model expresses the expected within-participant variation in PL scores for the product being evaluated by: (1) a participant (j) from the Netherlands; (2) who is male; (3) who has not reached the age of 1 year (i.e., a zero baseline); and (4) who has provided equal dominance scores for all four products (the last two conditions represent cases with no variability, i.e., completely hypothetical cases which are not found in the data

\({\upgamma }_{01}\) :

An adjustment for Belgian participants

\({\upgamma }_{02}\) :

An adjustment for German participants

\({\upgamma }_{03}\) :

An adjustment for Spanish participants

\({\upgamma }_{04}\) :

An adjustment for every year added to one’s baseline age (zero)

\({\upgamma }_{05}\) :

An adjustment for being female

\({\upgamma }_{06}\) :

An adjustment for \({\upgamma }_{00 }\) for a one-unit increase in the variation in product dominance scores

\(\hbox {u}_{\mathrm{j}}\) :

The residual variance for this participant

Appendix 3

R programming code for Monte Carlo simulation analysis.

1.1 Exhibit 3

See Table 6.

Table 6 Illustrative R code for Product A (see Table 4; BE: \(M\)=5.81 [SD=1.66]; SP: \(M\)=4.92 [SD=2.38])

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

De Beuckelaer, A., Zeeman, M. & Van Trijp, H. Assessment of the cross-national validity of an End-anchored 9-point hedonic product liking scale. Qual Quant 49, 1267–1286 (2015). https://doi.org/10.1007/s11135-014-0049-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11135-014-0049-0

Keywords

Navigation