Abstract
The immune system is a complex biological system with a highly distributed, adaptive and self-organising nature. This paper presents an Artificial Immune System (AIS) that exploits some of these characteristics and is applied to the task of film recommendation by Collaborative Filtering (CF). Natural evolution and in particular the immune system have not been designed for classical optimisation. However, for this problem, we are not interested in finding a single optimum. Rather we intend to identify a sub-set of good matches on which recommendations can be based. It is our hypothesis that an AIS built on two central aspects of the biological immune system will be an ideal candidate to achieve this: Antigen–antibody interaction for matching and idiotypic antibody–antibody interaction for diversity. Computational results are presented in support of this conjecture and compared to those found by other CF techniques.
Introduction
Collaborative Filtering (CF) is the foundational technology for modern recommendation engines, typically functioning by identifying a "neighborhood" of users with similar tastes. Traditional methods, such as k-Nearest-Neighbor (k-NN), focus primarily on finding the best matches based on correlation. However, this approach often leads to redundancy; a neighborhood dominated by nearly identical users provides limited information for diverse recommendations.
The challenge lies in identifying a subset of users who are close matches to the target user but also distinct from one another. This paper proposes an Artificial Immune System (AIS) inspired by biological idiotypic networks to solve this selection problem through distributed control and population diversity.
Methodology
The proposed AIS treats the user seeking a recommendation as an antigen and other users in the database as antibodies. The system operates through three primary mechanisms:
- Matching (Stimulation): A user is "stimulated" if their movie ratings correlate highly with the target user, calculated using the Pearson correlation measure.
- Diversity (Suppression): Utilizing Jerne’s idiotypic network theory, antibodies can recognize and suppress each other. If two potential neighbors are too similar, they compete, and one may be suppressed, preventing the neighborhood from becoming a collection of redundant clones.
- Dynamics: The system's state is governed by differential equations (derived from Farmer et al.) that calculate the "concentration" of each user in the neighborhood over time. Users with low concentration are eventually removed.
The final concentration of each "antibody" serves as a weight to predict the target user's preference for specific films.
Experimental Setup
The system was evaluated using the EachMovie database, which contains over 2.8 million votes from approximately 73,000 users across 1,628 films. The authors sampled test users and withheld a portion of their ratings to evaluate the algorithm's predictive and recommendation capabilities.
Results and Discussion
The study highlights that neighborhood composition significantly impacts recommendation quality. The AIS produces neighborhoods that are typically smaller, have less overlap, and possess lower inter-neighbor correlation than standard Pearson-based methods.
Prediction Accuracy
In terms of Mean Absolute Error (MAE), there was no statistically significant difference between the AIS and the standard Simple Pearson (SP) predictor. As shown in the following analysis, both methods performed similarly when guessing specific numerical scores.
Recommendation Quality
The true strength of the AIS approach lies in recommendation ranking rather than raw score prediction. Using Kendall’s Tau to measure ranking accuracy, the AIS significantly outperformed the standard SP approach.
The results indicate a "sweet spot" for suppression. For instance, at a stimulation rate of 0.3, the inclusion of the idiotypic effect increased the Kendall measure from 0.5 to nearly 0.6, effectively raising the probability of correctly ranking two randomly sampled movies from 60% to 80%.
Conclusion
The AIS model successfully leverages biological principles to maintain diversity in user neighborhoods, leading to superior ranked recommendations. While the model requires more computational iterations to stabilize compared to one-shot k-NN calculations, the improvement in recommendation quality is substantial. Future work will focus on scaling the analysis beyond the initial 100-user sample and exploring applications in ad-hoc community formation and web bookmark prediction.
Summary updated