Skip to main content
SpringerLink
Log in

Interaction design guidelines on critiquing-based recommender systems

  • Original Paper
  • Published:
User Modeling and User-Adapted Interaction Aims and scope Submit manuscript

Abstract

A critiquing-based recommender system acts like an artificial salesperson. It engages users in a conversational dialog where users can provide feedback in the form of critiques to the sample items that were shown to them. The feedback, in turn, enables the system to refine its understanding of the user’s preferences and prediction of what the user truly wants. The system is then able to recommend products that may better stimulate the user’s interest in the next interaction cycle. In this paper, we report our extensive investigation of comparing various approaches in devising critiquing opportunities designed in these recommender systems. More specifically, we have investigated two major design elements which are necessary for a critiquing-based recommender system: critiquing coverage—one vs. multiple items that are returned during each recommendation cycle to be critiqued; and critiquing aid—system-suggested critiques (i.e., a set of critique suggestions for users to select) vs. user-initiated critiquing facility (i.e., facilitating users to create critiques on their own). Through a series of three user trials, we have measured how real-users reacted to systems with varied setups of the two elements. In particular, it was found that giving users the choice of critiquing one of multiple items (as opposed to just one) has significantly positive impacts on increasing users’ decision accuracy (particularly in the first recommendation cycle) and saving their objective effort (in the later critiquing cycles). As for critiquing aids, the hybrid design with both system-suggested critiques and user-initiated critiquing support exhibits the best performance in inspiring users’ decision confidence and increasing their intention to return, in comparison with the uncombined exclusive approaches. Therefore, the results from our studies shed light on the design guidelines for determining the sweetspot balancing user initiative and system support in the development of an effective and user-centric critiquing-based recommender system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Agrawal, R., Imielinski, T., Swami, A.: Mining association rules between sets of items in large databases. In: 1993 ACM SIGMOD International Conference on Management of Data, Washington, DC, USA, pp. 207–216. (1993)

  • Ajzen I.: The theory of planned behavior. Organ. Behav. Hum. Decis. Process. 50, 179–211 (1991)

    Article  Google Scholar 

  • Ariely D.: Controlling the information flow: effects on consumers. Decision making and preferences. J. Consum. Res. 27, 233–248 (2000)

    Article  Google Scholar 

  • Benbasat I., Nault B.R.: An evaluation of empirical research in managerial support systems. Decis. Support Syst. 6(2), 203–226 (1990)

    Article  Google Scholar 

  • Bettman J.R., Johnson E.J., Payne J.W.: A componential analysis of cognitive effort in choice. Organ. Behav. Hum. Decis. Process. 45, 111–139 (1990)

    Article  Google Scholar 

  • Burke, R.: Knowledge-based recommender systems. Encyclopedia Library Inform. Syst. 69, Supplement 32 (2000)

  • Burke, R., Hammond, K., Cooper, E.: Knowledge-based navigation of complex information spaces. In: Thirteenth National Conference on Artificial Intelligence, Portland, Oregon, pp. 462–468 (1996)

  • Burke R., Hammond K., Young B.: The FindMe approach to assisted browsing. IEEE Expert: Intell. Syst. Appl. 12, 32–40 (1997)

    Google Scholar 

  • Carenini G., Poole D.: Constructed preferences and value-focused thinking: implications for AI research on preference elicitation. In: AAAI-02 Workshop on Preferences in AI and CP:. Symbolic Approaches. Edmonton, Canada (2002)

  • Chen, L., Pu, P.: Trust building in recommender agents. In: Workshop on Web Personalization, Recommender Systems and Intelligent User Interfaces at the Second International Conference on E-Business and Telecommunication Networks, Reading, UK, pp. 135–145 (2005)

  • Chen, L., Pu, P.: Evaluating critiquing-based recommender agents. In: Twenty-first National Conference on Artificial Intelligence, Boston, USA, pp. 157–162 (2006)

  • Einhorn H., Hogarth R.: Confidence in judgment: persistence of the illusion of validity. Psychol. Rev. 85, 395–416 (1978)

    Article  Google Scholar 

  • Falk R.F., Miller N.B.: A Primer for Soft Modeling, 1st edn. The University of Akron Press, Akron Ohio (1992)

    Google Scholar 

  • Faltings B., Torrens M., Pu P.: Solution generation with qualitative models of preferences. Int. J. Comput. Intell. Appl. 20, 246–264 (2004)

    MathSciNet  Google Scholar 

  • Grabner-Kräuter S., Kaluscha E.A.: Empirical research in online trust: a review and critical assessment. Int J Hum–Comput Stud 58, 783–812 (2003)

    Article  Google Scholar 

  • Grabner-Kräuter, S., Kaluscha, E.A., Fladnitzer, M.: Perspectives of online trust and similar constructs: a conceptual clarification. In: Eighth International Conference on Electronic Commerce, Fredericton, New Brunswick, Canada, pp. 235–243 (2006)

  • Häubl G., Trifts V.: Consumer decision making in online shopping environments: the effects of interactive decision aids. Mark Sci 19, 4–21 (2000)

    Article  Google Scholar 

  • Hopkins, W.: A New View of Statistics. http://www.sportsci.org/resource/stats/index.html (1997)

  • Koufaris, M., Hampton-Sosa, W.: Customer trust online: examining the role of the experience with the web-site. CIS Working Paper Series, Zicklin School of Business, Baruch College, New York, NY (2002)

  • Linden, G., Hanks, S., Lesh, N.: Interactive assessment of user preference models: the automated travel assistant. In: International Conference on User Modeling, Chia Laguna, Sardinia, Italy, pp. 67–78(1997)

  • McCarthy, K., Reilly, J., McGinty, L., Smyth, B.: On the dynamic generation of compound critiques in conversational recommender systems. In: Third International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems, Eindhoven, Netherlands, pp. 176–184 (2004a)

  • McCarthy, K., Reilly, J., McGinty, L.,Smyth, B.: Thinking positively \({\frac{1}{\mu}}\) explanatory feedback for conversational recommender systems. In: Workshop on Explanation in CBR at the Seventh European Conference on Case-Based Reasoning, Madrid, Spain, pp. 115–124 (2004b)

  • McCarthy, K., McGinty, L., Smyth, B., Reilly, J.: A live-user evaluation of incremental dynamic critiquing. In: Sixth International Conference on Case-based Reasoning, Chicago, IL, USA, pp. 339–352 (2005a)

  • McCarthy, K., McGinty, L., Smyth,B., Reilly J.: On the evaluation of dynamic critiquing: a large-scale user study. In: Twentieth National Conference on Artificial Intelligence and the Seventeenth Innovative Applications of Artificial Intelligence Conference, Pittsburgh, Pennsylvania, USA, pp. 535–540 (2005b)

  • McCarthy, K., Reilly, J., McGinty, L., Smyth, B.: Experiments in dynamic critiquing. In: Tenth International Conference on Intelligent User Interfaces, San Diego, California, USA, pp. 175–182 (2005c)

  • McKnight D.H., Chervany N.L.: What trust means in E-commerce customer relationships: conceptual typology. Int J Electron Commer 6(2), 35–59 (2002)

    Google Scholar 

  • McNee, S.M., Lam, S.K., Konstan, J.A., Riedl, J.: Interfaces for eliciting new user preferences in recommender systems. In: Ninth International Conference on User Modeling, Johnstown, Pennsylvania, USA, pp. 178–188 (2003)

  • McSherry, D.: Explanation in recommender systems. In: Workshop Proceedings of the Seventh European Conference on Case-Based Reasoning, Madrid, Spain, pp. 125–134 (2004)

  • Nielsen, J.: Enhancing the explanatory power of usability heuristics. In: SIGCHI Conference on Human factors in Computing Systems, Boston, USA, pp. 152–158 (1994)

  • Nguyen, Q.N., Ricci, F., Cavada, D.: Critique-based recommendations for mobile users: GUI design and evaluation. In: Third Workshop on “HCI in Mobile Guides” in Conjunction with Sixth International Conference on Human Computer Interaction with Mobile Devices and Services, Glasgow, Scotland (2004)

  • Novak T.P., Hoffman D.L., Yung Y.-F.: Measuring the customer experience in online environments: a structural modelling approach. Mark Sci 19(1), 22–42 (2000)

    Article  Google Scholar 

  • Payne, J.W., Bettman, J.R., Johnson, E.J.: The Adaptive Decision Maker. Cambridge University Press (1993)

  • Payne J.W., Bettman J.R., Schkade D.A.: Measuring constructed preference: towards a building code. J Risk Uncertainty 19(1–3), 243–270 (1999)

    Article  MATH  Google Scholar 

  • Pu, P., Chen, L.: Integrating tradeoff support in product search tools for E-commerce sites. In: Sixth ACM Conference on Electronic Commerce, Vancouver, BC, Canada, pp. 269–278 (2005)

  • Pu, P., Faltings, B.: Enriching buyers’ experiences: the SmartClient approach. In: SIGCHI Conference on Human Factors in Computing Systems, Hague, Netherlands, pp. 289–296 (2000)

  • Pu P., Faltings B.: Decision tradeoff using example critiquing and constraint programming. Special Issue User-Interact Constraint Satisfaction, CONSTRAINT: an Int. J. 9(4), 289–310 (2004)

    Google Scholar 

  • Pu, P., Kumar, P.: Evaluating example-based search tools. In: Fifth ACM Conference on Electronic Commerce, New York, NY, USA, pp. 208–217 (2004)

  • Reilly, J., McCarthy, K., McGinty, L., Smyth, B.: Dynamic critiquing. In: Seventh European Conference on Case-based Reasoning, Madrid, Spain, pp. 763–777 (2004)

  • Reilly J., McCarthy K., McGinty L., Smyth B.: Explaining compound critiques. Artif. Intell. Rev. 24(2), 199–220 (2005)

    Article  Google Scholar 

  • Shimazu, H.: ExpertClerk: navigating shoppers’ buying process with the combination of asking and proposing. In: Seventeenth International Joint Conference on Artificial Intelligence, Seattle, Washington, USA, pp. 1443–1450 (2001)

  • Smyth, B., McGinty L.: An analysis of feedback strategies in conversational recommenders. In: Fourteenth Irish Artificial Intelligence and Cognitive Science Conference, Dublin, Ireland, pp. 211–216 (2003)

  • Shneiderman, B.: In: Designing the User Interface: Strategies for Effective Human–Computer Interaction, 3rd edn. Addison-Wesley, Reading, MA (1997)

  • Spiekermann S., Parachiv C.: Motivating human–agent interaction: transferring insights from behavioral marketing to interface design. J Electron Commer Res 2(3), 255–285 (2002)

    Article  MATH  Google Scholar 

  • Torrens M., Faltings B., Pu P.: SmartClients: constraint satisfaction as a paradigm for scaleable intelligent information systems. Int J Constraints 7(1), 49–69 (2002)

    Article  MATH  Google Scholar 

  • Thompson C.A., Goker M.H., Langley P.: A personalized system for conversational recommendations. J. Artif. Intell. Res. 21, 393–428 (2004)

    Google Scholar 

  • Tversky A., Simonson I.: Context-dependent preferences. Manage. Sci. 39(10), 1179–1189 (1993)

    Article  MATH  Google Scholar 

  • Viappiani P., Faltings B., Pu P.: Preference-based search using example-critiquing with suggestions. J. Artif. Intell. Res. 27, 465–503 (2007)

    Google Scholar 

  • Williams, M.D., Tou, F.N.: RABBIT: an interface for database access. In: ACM ’82 Conference, pp. 83–87 (1982)

Download references

Author information

Authors and Affiliations

  1. Human Computer Interaction Group, School of Computer and Communication Sciences, Swiss Federal Institute of Technology in Lausanne (EPFL), 1015, Lausanne, Switzerland

    Li Chen & Pearl Pu

Authors
  1. Li Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pearl Pu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li Chen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chen, L., Pu, P. Interaction design guidelines on critiquing-based recommender systems. User Model User-Adap Inter 19, 167–206 (2009). https://doi.org/10.1007/s11257-008-9057-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11257-008-9057-x

Keywords

Search

Navigation