TL;DR
This paper introduces CANTINA+, a robust machine learning framework for detecting phishing websites. It significantly improves upon existing methods by employing a rich set of novel features and a layered approach that includes efficient filtering mechanisms, achieving high detection rates with low false positives.
What Problem Does This Paper Address?
Phishing attacks, a form of identity theft, pose a significant threat, causing billions of dollars in losses annually. Traditional anti-phishing methods, such as URL blacklists, struggle with new and rapidly evolving phishing attacks. Machine learning approaches, while more adaptable, often suffer from a scarcity of effective features and a high rate of false positives, which can lead to legitimate websites being blocked. This paper aims to bridge this gap by developing a more comprehensive and accurate phishing detection system that addresses both the timeliness and accuracy challenges.
Methods
CANTINA+ employs a layered architecture for phishing detection. The core of the system is a feature extractor that leverages 15 high-level features derived from various sources, including URL analysis, HTML DOM inspection, search engine queries, and third-party services. The novelty lies in the introduction of eight new features and the comprehensive combination of existing ones.
Before the main classification, CANTINA+ incorporates two filtering stages:
- Hash-based Near-Duplicate Page Remover: This filter uses SHA1 hashing to quickly identify and remove pages that are nearly identical to known phishing sites, leveraging the fact that phishing toolkits often generate many similar pages.
- Login Form Detector: This heuristic filter identifies and classifies pages without login forms as legitimate. The rationale is that phishing attacks almost invariably aim to collect sensitive information via login forms.
The extracted features are then fed into a machine learning engine, where various algorithms (including Bayesian Network, SVM, Logistic Regression, etc.) are trained to classify webpages. The paper highlights the Bayesian Network (BN) as a consistently high-performing choice.
Datasets
The authors evaluated CANTINA+ on a diverse corpus comprising:
- Phishing Websites: Collected from PhishTank, including two sets (Phish set 1 and Phish set 2) gathered at different times to assess the system's ability to handle evolving threats.
- Legitimate Websites: Collected from five sources, including popular websites, commonly spammed sites, and typical phishing targets. Two sets of legitimate data were used for randomized and time-based evaluations.
The paper does not introduce a new benchmark dataset but uses existing sources to create a comprehensive evaluation corpus.
Key Insights and Ideas
- Feature Richness is Key: The paper strongly advocates for a broad and diverse set of features, including novel ones derived from search engines and third-party services, as crucial for high detection rates.
- Layered Filtering is Effective: The combination of a hash-based filter for near-duplicates and a login form detector significantly reduces false positives and speeds up the overall process, making the system more practical.
- Login Forms are a Strong Indicator: The observation that almost all phishing attacks use login forms is a powerful heuristic that can effectively filter out a large number of legitimate pages.
- Time-Based Evaluation is Crucial: Evaluating the system on data collected over time (time-based evaluation) provides a more realistic assessment of its performance against evolving phishing tactics compared to purely randomized evaluations.
- Machine Learning Generalizes Well: The consistent performance across different machine learning algorithms suggests that the effectiveness of CANTINA+ stems primarily from its feature set rather than a specific algorithm.
Key Results
- Randomized Evaluation:
- Achieved over 92% True Positive (TP) rate on unique testing phish.
- Achieved over 99% TP rate on near-duplicate testing phish.
- Maintained a low False Positive (FP) rate of approximately 0.4% with 10% phishing data in the training set.
- Time-Based Evaluation:
- Achieved over 92% TP rate on unique testing phish.
- Achieved over 99% TP rate on near-duplicate testing phish.
- Reported an FP rate of approximately 1.4% with 20% phishing data in the training set over a two-week sliding window.
- Login Form Detection: Achieved a 98.06% TP rate in detecting phishing pages with login forms.
- Hash-based Filter: Identified 72.67% of phishing pages as near-duplicates, highlighting the prevalence of phishing toolkits.
Limitations and Future Work
- Limitations:
- The "Embedded domain" feature has a relatively poor AUC, and the authors suggest stricter constraints for regular expressions to improve it.
- The "IP address" feature also shows poor performance, and the authors suggest using it as a filter rather than a feature.
- The system may struggle with phishing attacks that compromise legitimate domains or host attacks on servers with no prior history.
- Phishing pages made entirely of images, without any text, pose a challenge, though the authors suggest training models with image-based features.
- The time-based evaluation showed more variability in FP rates compared to randomized evaluation, potentially due to the fixed nature of training data in sliding windows and the use of default model parameters.
- Future Work:
- Develop more robust brand name detectors for features like "Out-of-position brand name."
- Enhance the "Bad action fields" feature to detect absolute URLs with directory hierarchies.
- Explore image-based features to detect phishing pages composed solely of images.
- Investigate solutions for phishing attacks that compromise legitimate domains.
- Further research into mitigating Cross-Site Scripting (XSS) vulnerabilities, which can be used for phishing.
Summary updated