Pre-Training HTML Detectors on What the Browser Renders

Phishing pages last minutes to hours. Detectors that must decide before a page reaches a user therefore read the raw page source, its HTML, instead of loading and executing it. That shortcut is also their weakness. Raze to the Ground showed that a black-box optimizer can drive several machine-learning phishing detectors to 0.00% detection in fewer than 30 queries. It uses edits that leave the page looking and behaving the same: hidden links styled display:none, scripts that decode Base64 at load time, and benign footers. Those detectors read features of the source, while the victim sees the rendered result. Attackers can change the source freely as long as the rendered page stays the same.

We propose to teach a compact HTML encoder, during pre-training, to predict what a browser would actually produce from a page. For each element it predicts whether that element is visible, what text the user sees, and where the page's forms send their data once scripts have run. The training targets come for free: a headless browser renders a large unlabeled crawl once, offline. A model that has learned to predict rendered behavior from raw source should be insensitive to manipulations that preserve rendered behavior. That includes manipulation families it never saw during training. And unlike rendering every page at decision time, it would run at the millisecond speeds that browser and proxy filters need.

Core Claim

The claim is that render-grounded pre-training buys robustness to unseen functionality-preserving HTML manipulations. This means continued pre-training whose targets are browser-computed properties of the page, not masked source tokens. We claim this robustness is not delivered by three alternatives at equal compute and data: masked-token pre-training on the same HTML, adversarial fine-tuning on known manipulations, or rule-based sanitization. We also claim the approach keeps clean detection competitive at a strict low false-positive operating point.

A secondary claim addresses a contradiction in the domain-pre-training literature. URLTran found that fine-tuning general BERT beat a URL model trained from scratch with masked-token prediction. Understanding HTML with Large Language Models found general checkpoints beat HTML-only scratch training by over 12% on element classification. The domain-specific line argues the opposite, from MonoCoder, BERTHop, the deep fusion work of Durrani et al., and urlBERT. Our hypothesis is that the divergence comes from the training objective, not from the data. Masked-token prediction on web source teaches surface statistics that general text already supplies. An objective tied to the domain's actual semantics, here what the browser does, is where in-domain training pays off. A factorial design tests this directly.

Positioning Against Prior Work

Three prior lines of work set up the opportunity.

Domain pre-training on web artifacts (supplies the method). urlBERT pre-trains on 3.28 billion URLs with five objectives, including virtual adversarial training against character tricks, but it never reads page content. DOM-LM encodes HTML using tree-position embeddings (node, parent, sibling, depth, tag) and node-level masking, reaching 94.2 F1 on SWDE. It has never been tested on phishing or adversarial edits, and its objective reconstructs source rather than behavior. We reuse DOM-LM's tree positions and subtree windowing, and we replace source reconstruction with render prediction. DoD embeds breadth-first tag sequences with Doc2Vec (89.9% true-positive rate at 0.7% false-positive rate). Injected tags break its traversal order, which is exactly the fragility we target.

Motivating evidence. Raze to the Ground supplies the threat model and the attack. It is also a warning: detectors that combined URL and HTML features were bypassed by HTML edits alone, so adding URL features does not rescue a source-feature detector. Characterizing and Mitigating Phishing Attacks at ccTLD Scale shows most phishing sits on compromised, aged domains. URL-only models such as URLTran and urlBERT therefore cannot carry the load.

Behavior-based and large-model detectors (constrain the design). DynaPhish probes live pages with dummy credentials and watches for redirection. That raises recall by 28–30% at about 0.5 s per page, which shows behavior is the right signal, but obtained at inference time. Large-model systems are accurate but slow and exposed to instructions hidden in the page:

Our proposal moves DynaPhish-style behavioral grounding into pre-training and pays the rendering cost once, offline. It does not reproduce dynamic credential probing; that remains future work.

Research Program

Corpus and targets. We would assemble a large unlabeled crawl of HTML pages paired with their URLs, built from Common Crawl and certificate-transparency streams in the spirit of DynaPhish's CertStream collection. Each page is loaded once in an instrumented headless browser, which records, after a fixed settle time:

  • for every source element, whether it ends up visible and its rough on-screen size;
  • the visible text, aligned to the source nodes that produced it;
  • the set of nodes created or modified by scripts;
  • for each form, the host its submission resolves to after redirects and script execution.

These become per-node and per-page prediction targets.

Model. The model is a compact encoder that reads the URL followed by the HTML in tree-aware subtree windows. Position embeddings follow DOM-LM. URL tokenization uses a subword vocabulary, motivated by URLTran's finding that subwords isolate brand fragments. The pre-training loss is a weighted sum of four terms: a per-node visibility loss, a loss for predicting the visible text attached to each node, a script-mutation loss, and a page-level loss for whether the predicted form-submission host matches the URL's host. That last target directly encodes the brand-and-domain mismatch that Lee et al.'s two-phase reasoning checks for. We would keep a small masked-token term to stabilize training. urlBERT's staged task ordering, structural targets before semantic ones, is borrowed to limit interference between objectives.

Initialization factorial. We train from a general text checkpoint and from scratch, each with render targets and with masked-token prediction only. The four cells are compared at matched tokens and compute; this is the test of the secondary claim.

Downstream use. Fine-tuning for phishing detection uses the labeled sets below. The deployed model never runs a browser; the target inference latency is under 10 ms per page on one GPU.

Baselines

Naive reference. The do-nothing reference is the hand-crafted-feature detector family attacked in Raze to the Ground: link ratios, script counts, and URL lexical features fed to gradient-boosted trees in the style of the feature-driven LightGBM approach, alongside the random forest and logistic regression models Raze evaluated. On DeltaPhish, Raze reports 0.00% detection under attack for its CNN, random forest and logistic regression detectors within under 30 queries. It reports 33% to 90% post-attack detection for the SpacePhish baseline. We rerun these models on our splits with the same labels.

Strongest off-the-shelf competitor. This is a general checkpoint fine-tuned on raw HTML plus URL, following Gur et al. Their result is the main counter-evidence to domain pre-training, and the system is easy to deploy. We give it the same backbone size, the same labeled data and the same hyperparameter search budget as our model. It is the cell of the factorial with no continued pre-training. URLTran is the URL-only reference. Its reported 86.80% true-positive rate at 0.01% false-positive rate is on proprietary Microsoft telemetry that we cannot access, so we retrain its recipe on our data rather than compare against that number. DOM-LM, fine-tuned identically, is the structure-aware competitor. Our only reported phishing numbers on HTML structure come from DoD on its PhishTank/Alexa set.

Defense baselines. We include adversarial fine-tuning, which augments the strongest competitor's labeled data with Raze's manipulations. We also include a rule-based sanitizer that strips hidden nodes and statically decodes Base64 strings before classification.

Upper bound. A classifier that runs the real browser at inference time and reads the rendered page is the ceiling our model tries to approach. We report its latency alongside its accuracy.

Key ablation. Masked-token continued pre-training on exactly the same crawl and compute isolates the render objective from simple exposure to more HTML.

What counts as winning. Beating the strongest competitor means two things. First, statistically higher detection under attack on manipulation families held out from all training. Second, clean true-positive rate at 0.01% false-positive rate no worse than that baseline's.

Accuracy and latency reference. PhishAgent's 96.10% TR-OP accuracy and the 0.93 F1 for ALP are reported for context only. They are measured on balanced sets with multi-second latency, so we do not treat them as the target.

Why not just…?

Why not just adversarially fine-tune on Raze's manipulations? This is a real threat against the 14 known manipulations, and we expect it to match us there. The question is generalization. Augmentation teaches invariance to the listed edits, while render targets teach invariance to anything that preserves rendering. We settle it with a leave-families-out protocol. Both methods see a subset of manipulation types and are attacked with held-out types such as opacity or off-screen hiding, String.fromCharCode and other non-Base64 encodings, and DOM nodes injected late by scripts. If adversarial fine-tuning matches us on held-out families, the render objective is not earning its cost, and we would report that.

Why not just sanitize the HTML first? A sanitizer catches the tricks its author anticipated, and nothing else. It is the cheapest serious competitor and is included as a baseline. The same held-out-family test decides it, plus one further test: an adaptive attacker who knows the sanitizer's rules.

Why not just render every page at decision time, or ask a multimodal LLM? Rendering is the accuracy ceiling. The file's reported costs for behavior- or vision-based checks range from about 0.46 s (VisualPhishNet) to about 2.25 s (PhishAgent). Lee et al. also note that page text can hijack a general LLM's verdict. We measure the accuracy gap to the rendering upper bound. The proposal is worthwhile only if that gap is small at a latency two or more orders of magnitude lower.

Why not just distill a multimodal LLM into a small model? Distillation copies the teacher's labels, including its blind spots to prompt injection, and needs paid API calls per training page. Render targets are cheap, exact and label-free. A distilled student, trained on the same pages and labels, is a worthwhile extra comparison if budget allows.

Proposed Experiments

Data. Labeled data come from DeltaPhish, used with Raze's attack code, and from the Lee et al. multimodal dataset of 4,480 pages across 1,882 brands, using the HTML and URLs only. Benign pages from a large held-out Common Crawl slice are added so that true-positive rate can be measured at 0.01% false-positive rate, the operating point URLTran argues is the one that matters in production. We report this rather than F1 on balanced splits. Splits are by brand and by time to mimic zero-day campaigns. TR-OP accuracy is reported for comparability with PhishAgent.

Decisive check one: held-out manipulation families. The headline experiment is the leave-families-out attack described above, run with Raze's query-efficient black-box optimizer at budgets of 30 and 300 queries. We report detection rate as a function of query budget for every baseline.

Decisive check two: attacking the predictor itself. An adaptive attack searches for edits that change the model's predicted rendering while leaving the true rendering unchanged. We can verify the true rendering by actually loading each edited page. The rate at which such edits are found measures how far the learned predictor can be trusted. If the attack succeeds easily, the claim fails.

Decisive check three: initialization factorial. The four-cell comparison (general versus scratch initialization, render versus masked-token objective) tests whether the objective, rather than the initialization, explains the URLTran and Gur et al. results. The prediction is a clear gain from render targets in both initializations. It also predicts that the gap between general and scratch initialization shrinks once render targets are used.

Diagnostic and cost measures. We separately measure how accurately the model predicts per-node visibility and form-submission hosts on held-out pages. Correlating this accuracy with robustness tells us whether the mechanism is the one claimed. Finally, we report latency per page and offline rendering cost per million pages, so readers can judge the trade against inference-time rendering.