Abstract
Reference-Based Phishing Detection (RBPD) is a robust method for identifying phishing websites by comparing brand identity on a page with its domain. However, existing RBPDs suffer from limited brand coverage and a reliance on visual logos. This paper introduces KnowPhish, an automated pipeline that constructs a massive multimodal brand knowledge base of over 20,000 brands using Wikidata and Google Image Search. We further develop the KnowPhish Detector (KPD), which leverages Large Language Models (LLMs) to extract brand intention from both text and logos. KPD achieves a 92.05% F1 score and a 30x speedup over dynamic state-of-the-art methods.
Introduction
The current "gold standard" for catching phishing sites is Reference-Based Phishing Detection (RBPD). These systems function as digital bouncers: they identify a brand logo on a suspicious page (e.g., PayPal), verify if the domain is legitimate (e.g., paypal.com), and flag mismatches. Despite their effectiveness, RBPDs face two major hurdles:
- Scalability: Most RBPDs rely on manually curated Brand Knowledge Bases (BKBs), typically containing fewer than 300 brands. This allows attackers to impersonate less common brands successfully.
- Visual Dependency: Existing tools are often "blind" to phishing pages that use text instead of logos to identify themselves (e.g., "Login to your Bank of America account").
Methodology
Automated Knowledge Base Construction
To overcome the scalability wall, we developed KnowPhish, an automated pipeline for building a comprehensive BKB. The process involves category-based searches (targeting industries like banking and postal services) and popularity-based searches using Tranco domain rankings. The system automatically augments this data by pulling logos from Wikidata and Google Image Search, identifying text aliases, and performing domain propagation to link parent companies with subsidiaries.
KnowPhish Detector (KPD)
The KnowPhish Detector (KPD) is a multimodal system that analyzes both visual and textual cues. It utilizes an LLM (GPT-3.5-turbo-instruct) to summarize HTML and URL content, identifying "brand intention" even in the absence of a logo. A specialized XLM-RoBERTa model classifies pages as Credential-Requiring Pages (CRP). If the extracted brand intention does not match the domain recorded in the KnowPhish BKB, the page is flagged as phishing.
Evaluation and Results
We evaluated KPD using historical feeds (OpenPhish, APWG), a manually validated dataset of 10,000 samples (TR-OP), and real-world traffic (SG-SCAN).
Key findings include:
- Performance: KPD achieved a 92.05% F1 score on the TR-OP dataset and improved the F1 scores of existing detectors like Phishpedia by 25%.
- Efficiency: KPD is 30 times faster than dynamic methods like DynaPhish, reducing processing time from ~10.6 seconds to between 0.22 and 2.02 seconds per sample.
- Robustness: The LLM-based approach proved resilient against typosquatting and prompt injection attacks.
Conclusion
KnowPhish addresses the critical limitations of manual BKBs and visual-only detection. By building a massive, automated knowledge graph and leveraging LLMs for textual brand extraction, KPD provides a scalable and efficient solution for modern phishing detection. Future work will focus on improving data completeness for private domains and enhancing visual-textual reasoning for complex "text-to-image" attacks.
Summary updated