Skip to main content
Log in

LinkLive: discovering Web learning resources for developers from Q&A discussions

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Software developers need access to correlated information (e.g., API documentation, Wikipedia pages, Stack Overflow questions and answers) which are often dispersed among different Web resources. This paper is concerned with the situation where a developer is visiting a Web page, but at the same time is willing to explore correlated Web resources to extend his/her knowledge or to satisfy his/her curiosity. Specifically, we present an item-based collaborative filtering technique, named LinkLive, for automatically recommending a list of correlated Web resources for a particular Web page. The recommendation is done by exploiting hyperlink associations from the crowdsourced knowledge on Stack Overflow. We motivate our research using an exploratory study of hyperlink dissemination patterns on Stack Overflow. We then present our LinkLive technique that uses multiple features, including hyperlink co-occurrences in Q&A discussions, locations (e.g., question, answer, or comment) in which hyperlinks are referenced, and votes for posts/comments in which hyperlinks are referenced. Experiments using 7 years of Stack Overflow data show that, our technique recommends correlated Web resources with promising accuracy in an open setting. A user study of 6 participants suggests that practitioners find the recommended Web resources useful for Web discovery.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10

Similar content being viewed by others

Notes

  1. https://d3js.org/

  2. https://gephi.org/

  3. http://dmitrybaranovskiy.github.io/raphael/

  4. https://www.highcharts.com/

  5. https://en.wikipedia.org/wiki/Singleton_pattern

  6. https://en.wikipedia.org/wiki/Abstract_factory_pattern

  7. https://en.wikipedia.org/wiki/Double-checked_locking

  8. http://stackoverflow.com/questions/15425282/singleton-pattern-interview

  9. The LinkLive tool is implemented as a Web browser add-on, based on GreaseMonkey/TamperMonkey technique. It can be downloaded at http://128.199.241.136:9000/download/.

  10. http://stackoverflow.com/help/how-to-answer

  11. Video demonstration at https://youtu.be/PvgzJ-fslGs The tool is available for downloading at http://128.199.241.136:9000/download/

  12. http://stackoverflow.com/questions/23360052

  13. https://www.similartech.com/

  14. http://alternativeto.net/

  15. http://www.similarweb.com/

References

  1. Adomavicius, G., Tuzhilin, A.: Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 17(6), 734–749 (2005)

    Article  Google Scholar 

  2. Ahn, J.-W., Brusilovsky, P., Grady, J., He, D., Syn, S.Y.: Open user profiles for adaptive news systems: help or harm?. In: Proceedings of WWW, pp. 11–20. ACM (2007)

  3. Anderson, A., Huttenlocher, D., Kleinberg, J., Leskovec, J.: Discovering value from community activity on focused question answering sites: a case study of stack overflow. In: Proceedings of KDD, pp. 850–858 (2012)

  4. Ar, Y., Bostanci, E.: A genetic algorithm solution to the collaborative filtering problem. Expert Syst. Appl. 61, 122–128 (2016)

    Article  Google Scholar 

  5. Baeza-Yates, R., Boldi, P., Chierichetti, F.: Essential Web pages are easy to find. In: Proceedings of WWW, pp. 97–107 (2015)

  6. Bagheri, E., Ensan, F.: Semantic tagging and linking of software engineering social content. Autom. Softw. Eng. 23(2), 147–190 (2016)

  7. Balabanović, M., Shoham, Y.: Fab: content-based, collaborative recommendation. Commun. ACM 40(3), 66–72 (1997)

    Article  Google Scholar 

  8. Barragáns-Martínez, A.B., Costa-Montenegro, E., Burguillo, J.C., Rey-López, M., Mikic-Fonte, F.A., Peleteiro, A.: A hybrid content-based and item-based collaborative filtering approach to recommend tv programs enhanced with singular value decomposition. Inform. Sci. 180(22), 4290–4311 (2010)

    Article  Google Scholar 

  9. Bernstein, M.S., Teevan, J., Dumais, S., Liebling, D., Horvitz, E.: Direct answers for search queries in the long tail. In: Proceedings of SIGCHI, pp. 237–246 (2012)

  10. Billsus, D., Pazzani, M.J.: User modeling for adaptive news access. User Model. User-Adap. Inter. 10(2-3), 147–180 (2000)

    Article  Google Scholar 

  11. Blanco, R., Cambazoglu, B.B., Mika, P., Torzec, N.: Entity recommendations in Web search. In: Proceedings of ISWC, pp. 33–48 (2013)

  12. Bobadilla, J., Ortega, F., Hernando, A., Gutiérrez, A.: Recommender systems survey. Knowl.-Based Syst. 46, 109–132 (2013)

    Article  Google Scholar 

  13. Brandt, J., Guo, P.J., Lewenstein, J., Dontcheva, M., Klemmer, S.R.: Two studies of opportunistic programming: interleaving Web foraging, learning, and writing code. In: Proceedings of CHI, pp. 1589–1598 (2009)

  14. Breese, J.S., Heckerman, D., Kadie, C.: Empirical analysis of predictive algorithms for collaborative filtering. In: Proceedings of UAI, pp. 43–52 (1998)

  15. Bretherton, I.: Attachment theory: retrospect and prospect. Monographs Soc. Res. Child Develop. 50(1/2), 3–35 (1985)

  16. Broder, A.: A taxonomy of Web search. In: Proceedings of SIGIR, vol. 36, pp. 3–10 (2002)

  17. Celma, Ò., Serra, X.: Foafing the music: bridging the semantic gap in music recommendation. Web Semant. Sci. Serv. Agents World Wide Web 6(4), 250–256 (2008)

    Article  Google Scholar 

  18. Chowdhury, G.G.: Introduction to Modern Information Retrieval. Facet Publishing, London (2010)

    Google Scholar 

  19. Cooley, R., Mobasher, B., Srivastava, J.: Web mining: information and pattern discovery on the World Wide Web. In: Proceedings of ICTAI, pp. 558–567 (1997)

  20. Das, A.S., Datar, M., Garg, A., Rajaram, S.: Google news personalization: scalable online collaborative filtering. In: Proceedings of WWW, pp. 271–280 (2007)

  21. Degemmis, M., Lops, P., Semeraro, G.: A content-collaborative recommender that exploits wordnet-based user profiles for neighborhood formation. Proceedings of UMAP 17(3), 217–255 (2007)

    Google Scholar 

  22. Dou, Z., Song, R., Nie, J.-Y., Wen, J.-R.: Using anchor texts with their hyperlink structure for Web search. In: Proceedings of SIGIR, pp. 227–234 (2009)

  23. Gomez, C., Cleary, B., Singer, L.: A study of innovation diffusion through link sharing on stack overflow. In: Proceedings of MSR, pp. 81–84 (2013)

  24. Gong, Y., Zhang, Q.: Hashtag recommendation using attention-based convolutional neural network. In: Proceedings of IJCAI, pp. 2782–2788. AAAI Press (2016)

  25. Grbovic, M., Radosavljevic, V., Djuric, N., Bhamidipati, N., Savla, J., Bhagwan, V., Sharp, D.: E-commerce in your inbox: product recommendations at scale. In: Proceedings of KDD, pp. 1809–1818 (2015)

  26. Guo, H., Tang, R., Ye, Y., Li, Z., Deepfm, X.H.E.: A factorization-machine based neural network for ctr prediction. In: Proceedings of IJCAI (2017)

  27. He, X., Liao, L., Zhang, H., Nie, L., Hu, X., Chua, T.-S.: Neural collaborative filtering. In: Proceedings of WWW, pp. 173–182. International World Wide Web Conferences Steering Committee (2017)

  28. Hidasi, B., Karatzoglou, A., Baltrunas, L., Tikk, D.: Session-based recommendations with recurrent neural networks. In: Proceedings of ICLR (2015)

  29. Holmes, R., Walker, R.J., Murphy, G.C.: Strathcona example recommendation tool. In: Proceedings of Software Engineering Notes, vol. 30, pp. 237–240 (2005)

  30. Karypis, G.: Evaluation of item-based top-N recommendation algorithms. In: Proceedings of CIKM, pp. 247–254. ACM, New York (2001)

  31. Klemmer, S.R., Sinha, A.K., Chen, J., Landay, J.A., Aboobaker, N., Wang, A.: Suede: a wizard of oz prototyping tool for speech user interfaces. In: Proceedings of UIST, pp. 1–10. ACM (2000)

  32. Krompaß, D., Baier, S., Tresp, V.: Type-constrained representation learning in knowledge graphs. In: Proceedings of ISWC, pp. 640–655. Springer (2015)

  33. Levandoski, J.J., Sarwat, M., Eldawy, A., Mokbel, M.F.: Lars: a location-aware recommender system. In: Proceedings of ICDE, pp. 450–461 (2012)

  34. Li, Y., Lu, L., Xuefeng, L.: A hybrid collaborative filtering method for multiple-interests and multiple-content recommendation in e-commerce. Expert Syst. Appl. 28(1), 67–77 (2005)

    Article  Google Scholar 

  35. Li, H., Zhao, X., Xing, Z., Bao, L., Peng, X., Gao, D., Zhao, W.: amassist: in-ide ambient search of online programming resources. In: Proceedings of SANER, pp. 390–398 (2015)

  36. Li, J., Bao, L., Xing, Z., Wang, X., Zhou, B.: Bpminer: mining developers’ behavior patterns from screen-captured task videos. In: Proceedings of SAC, pp. 1371–1377. ACM (2016)

  37. Li, J., Xing, Z., Ye, D., Zhao, X.: From discussion to wisdom: Web resource recommendation for hyperlinks in stack overflow. In: Proceedings of SAC, pp. 1127–1133. ACM (2016)

  38. Li, J., Sun, A., Xing, Z.: Learning to answer programming questions with software documentation through social context embedding. Inform. Sci. 448-449, 36–52 (2018)

    Article  Google Scholar 

  39. Li, J., Sun, A., Xing, Z., Han, L.: Api caveat explorer: surfacing negative usages from practice. In: Proceedings of SIGIR, pp. 1293–1296. https://doi.org/10.1145/3209978.3210170 (2018)

  40. Li, J., Xing, Z., Kabir, A.: Leveraging official content and social context to recommend software documentation. IEEE Transactions on Services Computing. IEEE Early Access (2018), https://doi.org/10.1109/TSC.2018.2812729

  41. Lilliefors, H.W.: On the kolmogorov-smirnov test for normality with mean and variance unknown. J. Am. Stat. Assoc. 62(318), 399–402 (1967)

    Article  Google Scholar 

  42. Lin, T., Pantel, P., Gamon, M., Kannan, A., Fuxman, A.: Active objects: actions for entity-centric search. In: Proceedings of WWW, pp. 589–598 (2012)

  43. Linden, G., Smith, B., York, J.: Amazon.com recommendations: item-to-item collaborative filtering. IEEE Internet Comput. 7(1), 76–80 (2003)

    Article  Google Scholar 

  44. Liu, T.-Y.: Learning to rank for information retrieval. Found. Trend Inform. Retrieval 3(3), 225–331 (2009)

    Article  Google Scholar 

  45. Luo, X., Xia, Y., Zhu, Q.: Incremental collaborative filtering recommender based on regularized matrix factorization. Know.-Based Syst. 27, 271–280 (2012)

    Article  Google Scholar 

  46. Magnini, B., Strapparava, C.: Improving user modelling with content-based techniques. In: Proceedings of the International Conference on User Modeling, pp. 74–83. Springer (2001)

  47. Marchionini, G.: Exploratory search: from finding to understanding. Commun. ACM 49(4), 41–46 (2006)

    Article  Google Scholar 

  48. McLaughlin, M.R., Herlocker, J.L.: A collaborative filtering algorithm and evaluation metric that accurately model the user experience. In: Proceedings of SIGIR, pp. 329–336. ACM (2004)

  49. Melville, P., Mooney, R.J., Nagarajan, R.: Content-boosted collaborative filtering for improved recommendations. In: Eighteenth National Conference on Artificial Intelligence, pp. 187–192. American Association for Artificial Intelligence, Menlo Park (2002)

  50. Miliaraki, I., Blanco, R., Lalmas, M.: From Selena Gomez to Marlon Brando: understanding explorative entity search. In: Proceedings of WWW, pp. 765–775 (2015)

  51. Mooney, R.J., Roy, L.: Content-based book recommending using learning for text categorization. In: Proceedings of the Fifth ACM Conference on Digital Libraries, pp. 195–204. ACM (2000)

  52. Nguyen, P.T., Tomeo, P., Di Noia, T., Di Sciascio, E.: Content-based recommendations via dbpedia and freebase: a case study in the music domain. In: Proceedings of ISWC, pp. 605–621. Springer (2015)

  53. Nicholas, I.S.C., Nicholas, C.K.: Combining content and collaboration in text filtering. In: Proceedings of IJCAI, pp. 86–91 (1999)

  54. Nilashi, M., Ibrahim, O.B., Ithnin, N., Zakaria, R.: A multi-criteria recommendation system using dimensionality reduction and neuro-fuzzy techniques. Soft. Comput. 19(11), 3173–3207 (2015)

    Article  Google Scholar 

  55. Nilashi, M., Ibrahim, O., Bagherifard, K.: A recommender system based on collaborative filtering using ontology and dimensionality reduction techniques. Expert Syst. Appl. 92, 507–520 (2018)

    Article  Google Scholar 

  56. Park, M.-H., Hong, J.-H., Cho, S.-B.: Location-based recommendation system using Bayesian user’s preference model in mobile devices. In: Ubiquitous Intelligence and Computing, pp. 1130–1139. Springer (2007)

  57. Park, S.-T., Pennock, D.M.: Applying collaborative filtering techniques to movie search for better ranking and browsing. In: Proceedings of KDD, pp. 550–559 (2007)

  58. Parnin, C., Treude, C., Grammel, L., Storey, M.-A.: Crowd documentation: exploring the Coverage and the Dynamics of Api Discussions on Stack Overflow. Georgia Institute of Technology, Tech. Rep (2012)

  59. Pazzani, M., Billsus, D.: Learning and revising user profiles: the identification of interesting Web sites. Mach. Learn. 27(3), 313–331 (1997)

    Article  Google Scholar 

  60. Ponzanelli, L., Bacchelli, A., Lanza. M.: Seahawk: stack overflow in the ide. In: Proceedings of ICSE, pp. 1295–1298 (2013)

  61. Ponzanelli, L., Bavota, G., Di Penta, M., Oliveto, R., Lanza, M.: Prompter: a self-confident recommender system. In: Proceedings of ICSME, pp. 577–580 (2014)

  62. Sakai, T., Nogami, K.: Serendipitous search via wikipedia: a query log analysis. In: Proceedings of SIGIR, pp. 780–781 (2009)

  63. Salakhutdinov, R., Mnih, A., Hinton, G.: Restricted Boltzmann machines for collaborative filtering. In: Proceedings of ICML, pp. 791–798. ACM, New York (2007)

  64. San Pedro, J., Karatzoglou, A.: Question recommendation for collaborative question answering systems with rankslda. In: Proceedings of RecSys, pp. 193–200 (2014)

  65. Sarwar, B., Karypis, G., Konstan, J., Riedl, J.: Item-based collaborative filtering recommendation algorithms. In: Proceedings of WWW, pp. 285–295 (2001)

  66. Schein, A.I., Popescul, A., Ungar, L.H., Pennock, D.M.: Methods and metrics for cold-start recommendations. In: Proceedings of SIGIR, pp. 253–260. ACM, New York (2002)

  67. Sedhain, S., Menon, A.K., Sanner, S., Xie, L.: Autorec: autoencoders meet collaborative filtering. In: Proceedings of WWW, pp. 111–112. ACM (2015)

  68. Seebach, C.: Searching for answers–knowledge exchange through social media in organizations. In: Proceedings of HICSS, pp. 3908–3917 (2012)

  69. Sheth, B., Maes, P.: Evolving agents for personalized information filtering. In: Proceedings of the Ninth Conference on Artificial Intelligence for Applications, pp. 345–352. IEEE (1993)

  70. Shinde, S.K., Kulkarni, U.: Hybrid personalized recommender system using centering-bunching based clustering algorithm. Expert Syst. Appl. 39(1), 1381–1387 (2012)

    Article  Google Scholar 

  71. Singhal, A., et al.: Modern information retrieval: a brief overview. IEEE Data Eng. Bull. 24(4), 35–43 (2001)

    Google Scholar 

  72. Su, X., Khoshgoftaar, T.M.: A survey of collaborative filtering techniques. Advan. Artif. Intell. 2009, 4 (2009)

    Google Scholar 

  73. Subramanian, S., Inozemtseva, L., Holmes, R.: Live api documentation. In: Proceedings of ICSE, pp. 643–652 (2014)

  74. Tang, D., Wei, F., Yang, N., Zhou, M., Liu, T., Qin, B.: Learning sentiment-specific word embedding for twitter sentiment classification. In: Proceedings of ACL, vol. 1, pp. 1555–1565 (2014)

  75. Tenopir, C., King, D.W.: Communication Patterns of Engineers. Wiley, New York (2004)

    Google Scholar 

  76. Tran, T., Phung, D., Venkatesh, S.: Collaborative filtering via sparse markov random fields. Inform. Sci. 369, 221–237 (2016)

    Article  Google Scholar 

  77. Treude, C., Barzilay, O., Storey, M.-A.: How do programmers ask and answer questions on the Web?: Nier track. In: Proceedings of ICSE, pp. 804–807 (2011)

  78. Treude, C., Robillard, M.: Augmenting api documentation with insights from stack overflow. In: Proceedings of ICSE

  79. Wang, S., Lo, D., Jiang, L.: An empirical study on developer interactions in stackoverflow. In: Proceedings of SAC, pp. 1019–1024 (2013)

  80. Wang, S., Lo, D., Vasilescu, B., Serebrenik, A.: Entagrec: an enhanced tag recommendation system for software information sites. In: Proceedings of ICSME, pp. 291–300 (2014)

  81. Wang, F.-H., Jian, S.-Y.: An effective content-based recommendation method for Web browsing based on keyword context matching. J. Inform. Electron. 1(2), 49–59 (2006)

    MathSciNet  Google Scholar 

  82. Wang, J., De Vries, A.P., Reinders, M.J.: Unifying user-based and item-based collaborative filtering approaches by similarity fusion. In: Proceedings of SIGIR, pp. 501–508 (2006)

  83. Wang, J., Yu, L., Zhang, W., Gong, Y., Xu, Y., Wang, B., Zhang, P., Zhang, D.: Irgan: a minimax game for unifying generative and discriminative information retrieval models. In: Proceedings of SIGIR, pp. 515–524. ACM (2017)

  84. White, R.W., Roth, R.A.: Exploratory search: beyond the query-response paradigm. Synth. Lect. Inform. Concepts Retr. Serv. 1(1), 1–98 (2009)

    Google Scholar 

  85. Yager, R.R.: Fuzzy logic methods in recommender systems. Fuzzy Sets Syst. 136 (2), 133–149 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  86. Ye, M., Yin, P., Lee, W.-C., Lee, D.-L.: Exploiting geographical influence for collaborative point-of-interest recommendation. In: Proceedings of SIGIR, pp. 325–334. ACM, New York (2011)

  87. Ye, D., Xing, Z., Foo, C.Y., Ang, Z.Q., Li, J., Kapre, N.: Software-specific named entity recognition in software engineering social content. In: Proceedings of SANER, vol. 1, pp. 90–101. IEEE (2016)

  88. Ye, D., Xing, Z., Li, J., Kapre, N.: Software-specific part-of-speech tagging: an experimental study on stack overflow. In: Proceedings of SAC, pp. 1378–1385. ACM (2016)

  89. Yu, X., Ma, H., Hsu, B.-J.P., Han, J.: On building entity recommender systems using user click log and freebase knowledge. In: Proceedings of WSDM, pp. 263–272. ACM (2014)

  90. Yuan, Q., Cong, G., Sun, A.: Graph-based point-of-interest recommendation with geographical and temporal influences. In: Proceedings of CIKM, pp. 659–668 (2014)

  91. Zagalsky, A., Barzilay, O., Yehudai, A.: Example overflow: using social media for code recommendation. In: Proceedings of the Third International Workshop on Recommendation Systems for Software Engineering, pp. 38–42 (2012)

  92. Zhang, S., Yao, L., Sun, A.: Deep learning based recommender system: a survey and new perspectives. arXiv:1707.07435 (2017)

  93. Zhou, G., He, T., Zhao, J., Hu, P.: Learning continuous word embedding with metadata for question retrieval in community question answering. In: Proceedings of ACL, pp. 250–259 (2015)

  94. Zimmermann, T., Dallmeier, V., Halachev, K., Zeller, A.: erose: guiding programmers in eclipse. In: Proceedings of SPLASH, pp. 186–187 (2005)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jing Li.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, J., Xing, Z. & Sun, A. LinkLive: discovering Web learning resources for developers from Q&A discussions. World Wide Web 22, 1699–1725 (2019). https://doi.org/10.1007/s11280-018-0621-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-018-0621-y

Keywords

Navigation