Hyperlink-Aware Mining and Analysis of the Web

The approximately seven billion hyperlinks on the WWW, and the anchortext surronding them, represent a valuable collection of editorial information about web pages. We begin by discussing methods for incorporating this link information into web search. Next, we consider a follow-on question: is it possible to apply data mining techniques to the link structure of the web in order to discover all communities, including those that have only just formed and whose members may not yet be aware of one another.

We also consider modeling and measurement of this hyperlink structure. A recent analysis of the web graph indicates that the macroscopic structure is considerably more intricate than suggested by earlier experiments. We describe these results, and go on to discuss some progress towards defining analytical models for graphs such as the web.

The work described here is joint with Andrei Broder, Ravi Kumar, Farzin Maghoul, Prabhakar Raghavan, Raymie Stata, Sridhar Rajagopalan, Eli Upfal and Janet Wiener.

Authors and Affiliations

1. K53/B1 IBM Almaden Research Center, 650 Harry Rd., San Jose, CA, 95120-6099, USA

   Andrew Tomkins

Editors and Affiliations

1. Graduate School of Systems Management, Universiy of Tsukuba, 3-29-1 Otsuka, Bunkyo-ku, Tokyo, 112-0012, Japan

   Takao Terano

2. Department of Computer Science and Engineering, Arizona State University, P.O. Box 875 406, Tempe, AZ, 85287-5406

   Huan Liu

3. Department of Computer Science, National Tsing Hua University, Hsinchu, 300, Taiwan ROC

   Arbee L. P. Chen

Reprints and permissions

© 2000 Springer-Verlag Berlin Heidelberg

Policies and ethics