Parallel Vector Computing Technique for Discovering Communities on the Very Large Scale Web Graph

  • Kikuko Kawase
  • Minoru Kawahara
  • Takeshi Iwashita
  • Hiroyuki Kawano
  • Masanori Kawazawa
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2737)


The study of the authoritative pages and community discovery from an enormous Web contents has attracted many researchers. One of the link-based analysis, the HITS algorithm, calculates authority scores as the eigenvector of a adjacency matrix created from the Web graph. Although it was considered impossible to compute the eigenvector of a very large scale of Web graph using previous techniques, due to this calculation requires enormous memory space. We make it possible using data compression and parallel computation.


Adjacency Matrix Main Memory Average Link Link Structure Vector Calculation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
  2. 2.
    Barrett, R., Chan, T., Donato, J., Berry, M., Demmel, J.: Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods, 2nd edn. SIAM, Philadelphia (1994)Google Scholar
  3. 3.
    Bharat, K., Henzinger, M.: Improved Algorithm for Topic Distillation in a Hyperlinked Environment. In: Proc. of ACM SIGIR 1998, Melbourne, Australia, pp. 104–111 (1998)Google Scholar
  4. 4.
    Broder, A., Kumar, R., Maghoul, F., Raghavan, P., Rajagopalan, S., Stata, R., Tomkins, A., Wiener, J.: Graph Structure in the Web. Computer Networks, 309–320 (2000)Google Scholar
  5. 5.
    Chakrabarti, S., Dom, B., Kumar, S., Raghavan, P., Rajagopalan, S., Tomkins, A., Gibson, D., Kleinberg, J.: Mining the Web’s Link Structure. Computer, 60–67 (1999)Google Scholar
  6. 6.
    Dean, J., Henzinger, M.: Finding Related Pages in the World Wide Web. In: Proc. of the 8th World-Wide Web Conference, Amsterdam, Netherlands (1999)Google Scholar
  7. 7.
    Fuller, L., Bechtel, R.: Introduction to matrix algebra. Dickenson Pub. Co. (1967)Google Scholar
  8. 8.
  9. 9.
    Hirokawa, S., Ikeda, D.: Structural Analysis of Web Graph. Transactions of the Japanses Society for Artificial Intelligence 16(4), 525–529 (2001)Google Scholar
  10. 10.
    Internet Software Consortium,
  11. 11.
    Kazama, K., Harada, M.: Advanced Web Search Engine Technologies. Transactions of the Japanses Society for Artificial Intelligence 16(4), 503–508 (2001)Google Scholar
  12. 12.
    Kleinberg, J.: Authoritative Sources in a Hyperlinked Environment. Journal of the ACM, 604–632 (1999)Google Scholar
  13. 13.
    Kleinberg, J., Kumar, R., Raghavan, P., Rajagopalan, S., Tomkins, A.: The Web as a Graph: Measurements, Models, and Methods. In: Asano, T., Imai, H., Lee, D.T., Nakano, S.-i., Tokuyama, T. (eds.) COCOON 1999. LNCS, vol. 1627, pp. 1–17. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  14. 14.
    Kumar, R., Raghavan, P., Rajagopalan, S., Tomkins, A.: Extracting Large-scale Knowledge Bases from the Web. In: Proceedings of the 25th International Conference on Very Large Databases, Edinburgh, UK, pp. 639–650 (1999)Google Scholar
  15. 15.
    Murata, T.: Discovery of Web Communities Based on Co-occurrence of References. Transactions of the Japanses Society for Artificial Intelligence 16(3), 316–323 (2001)CrossRefGoogle Scholar
  16. 16.
    NII-NACSIS Test Collection for IR Systems Project,
  17. 17.
  18. 18.
    Page, L., Brin, S., Motwani, R., Winograd, T.: The PageRank Citation Ranking: Bringing Order to the Web, Stanford Digital Library Project No. 1999-66 (1999),
  19. 19.
    Yamada, S., Murata, T., Kitamura, Y.: Intelligent Web Information System. Transactions of the Japanses Society for Artificial Intelligence 16(4), 495–502 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Kikuko Kawase
    • 1
  • Minoru Kawahara
    • 2
  • Takeshi Iwashita
    • 2
  • Hiroyuki Kawano
    • 1
  • Masanori Kawazawa
    • 2
  1. 1.Department of Systems ScienceGraduate School of Informatics, Kyoto University 
  2. 2.Academic Center for Computing and Media StudiesKyoto University 

Personalised recommendations