TL;DR
This paper proposes using Artificial Immune Systems (AIS) to build a web site recommender system, extending previous work on film recommendations. The core idea is to model user web profiles as "antibodies" and recommend new sites by finding users with similar browsing histories, leveraging the hierarchical structure of the DMOZ ontology for more nuanced similarity matching.
What Problem Does This Paper Address?
Recommending relevant web content to users is a significant challenge, especially given the vastness of the internet. Traditional collaborative filtering methods often struggle with sparse data (users having few items in common) and can be unduly influenced by superficial similarities. The authors aim to address this by developing a recommender system that can uncover "surprising" yet relevant web sites, going beyond simple search queries. They propose an AIS approach, inspired by the biological immune system's ability to recognize and respond to novel entities, to find users with similar web browsing preferences and recommend sites based on this similarity. A key challenge is effectively representing and comparing user web profiles, which are often collections of URLs, in a way that captures nuanced similarities.
Methods
The paper adapts the concept of Artificial Immune Systems, specifically drawing from Jerne's idiotypic network theory. The core algorithm involves:
- Encoding User Profiles: User bookmark collections (URLs) are not used directly. Instead, they are classified using the DMOZ (Open Directory Project) ontology. This creates a hierarchical representation of user preferences. Two encoding schemes are discussed: integer category IDs and tree-structure-based category addresses.
- Artificial Immune System (AIS) Algorithm: The system initializes a pool of users (antibodies) and a target user (antigen). The algorithm iteratively increases the "concentration" of antibodies (users) that are similar to the antigen.
- Idiotypic Network Principle: A key aspect is the "suppression of similar antibodies." This means that while users similar to the target are favored, users who are too similar to each other are down-weighted. This encourages diversity in the recommendations, leading to potentially more surprising and interesting suggestions.
- Similarity Measure: A novel similarity measure is proposed to compare user web profiles. This measure takes into account:
- Hierarchical Depth: Matches at lower levels of the DMOZ tree (more specific categories) contribute more than matches at higher levels.
- Edge Distance: The "distance" between categories in the DMOZ tree influences the similarity score.
- Disparity Correction: The measure accounts for differences in the number of bookmarks between users to avoid skewed similarity scores.
Datasets
- DMOZ (Open Directory Project): This is used as the ontology for classifying URLs. The paper notes that the DMOZ database is approximately 1 GB and contains over 5 million URLs across 428,590 categories, spanning 16 levels.
- User Bookmark Collections: The system uses a collection of over 1000 individuals' web profiles (bookmark files) for training and evaluation. The paper mentions an example of raw bookmark data from the Opera browser.
Key Insights and Ideas
- Leveraging Hierarchical Ontologies for Similarity: The paper highlights the importance of using hierarchical structures like DMOZ to go beyond simple URL matching. By considering the relationships between categories, a more nuanced understanding of user similarity can be achieved.
- Idiotypic Network for Diverse Recommendations: The application of the idiotypic network principle to suppress similar "antibodies" (users) is crucial for generating diverse and potentially surprising recommendations, moving beyond the limitations of purely similarity-based approaches.
- The Challenge of Web Profile Representation: The paper underscores the difficulty of directly using raw URLs and the necessity of a robust classification and encoding scheme (like DMOZ) to make user profiles comparable.
- Balancing Specificity and Generality in Similarity: The proposed similarity measure attempts to balance the need for precise matches (lower tree levels) with the reality of sparse data and the potential for broader, yet still relevant, interests.
Key Results
The paper presents a conceptual framework and a proposed similarity measure. While specific quantitative results are not detailed in this excerpt, the authors state:
- Preliminary results suggest that they can map between 60% and 80% of users' bookmarks to suitable categories within the DMOZ ontology.
- They express confidence that this approach provides a strong basis for an Artificial Immune System recommender, with further results to be published later.
Limitations and Future Work
- Encoding Choice: The paper acknowledges that the choice of encoding (integer IDs vs. tree structure addresses) could critically affect the system's success. The integer ID approach faces challenges with evolving classification systems, while the tree structure approach requires a complex similarity measure.
- Similarity Measure Complexity: Developing a robust similarity measure that accurately reflects hierarchical relationships and handles data sparseness is identified as a significant challenge.
- Data Sparseness: The paper notes that the web site recommendation task faces a considerably sparser search space compared to film recommendation, which could hinder the system's effectiveness.
- Future Work: The authors intend to explore the potential of the first encoding (integer IDs) first, given existing precedents. However, they also recognize the need to develop a tree comparison similarity measure as a future challenge. They plan to conduct further experiments and publish subsequent results.
Summary updated