ValQID: Learning the Net Value of Semantic Query Groups for Web Device Identification

Yifan Yang, Zhen Huang, Weijia Shi, Xichen Tan, Zhihuang Liu, Xionglve Li, Zhiping CaiarXiv 2026 · arXiv

TL;DR

ValQID is a new framework that identifies networked devices (like routers and cameras) by intelligently analyzing their web interfaces. By using a cost-aware policy to select only the most informative "groups" of semantic questions, it achieves state-of-the-art accuracy—especially for rare device brands—while running up to 8.5x faster than general-purpose language models.

What Problem Does This Paper Address?

Identifying the specific vendor and type of a device (e.g., a "Hikvision NVR" or a "Cisco Router") just by looking at its web management page is a cornerstone of network security and asset management. However, this is harder than it sounds for two main reasons:

  1. Dispersed Cues: Identity information is rarely in one place. You might find a "Save" button in one corner and a "Channel 01" label in another. To understand that this is a video recorder, a system needs to link these objects together and understand their relationship.
  2. The Efficiency Paradox: While Large Language Models (LLMs) are great at understanding these semantic cues, they are computationally expensive. Running a full LLM analysis on every single web response during a large-scale network scan is prohibitively slow and costly.

Traditional methods either rely on rigid "fingerprints" that break when a device's software updates (see Figure 1a) or use expensive learning-based models that treat every page with the same heavy-handed analysis (see Figure 1b).

Figure 1: Motivation for ValQID. (a) Interface updates can in- validate fingerprints while semantic cues remain, requiring manual rule maintenance. (b) Learning-based identification uses semantic cues, with inference costs accumulating across Web responses.

Methods

The authors introduce ValQID, which treats device identification as a sequential decision-making problem. Instead of analyzing everything, it asks: "Which questions will give me the most 'net value' for my computational buck?"

Key components include:

  • Relation Graphs: ValQID parses a web page into a graph where nodes are objects (text, forms, buttons) and edges represent their structural relationships. This provides the necessary context for semantic analysis.
  • DSEJEV (Device Semantic Evaluation JEV): Instead of a standard "chatty" LLM, the authors developed a non-generative evaluator. It takes a question and context and directly outputs a probability distribution over predefined answers. This avoids the slow process of generating text and parsing it.
  • Complementary Question Groups: The system organizes questions into groups that are more informative when answered together (e.g., asking about a form's purpose and its submission button's function simultaneously).
  • Net Value Estimation: A policy model weighs the expected "identification benefit" (how much closer we get to the right answer) against the "execution cost." It selects the best group to execute next and knows when to stop once additional analysis no longer provides enough value.
Figure 2: ValQID workflow. Web object relations organize complementary semantic question groups. A shared relation encoder supports identity prediction and group value estimation. ValQID selects groups by predicted net value and updates the graph with DSEJEV’s option probabilities.

Datasets

The researchers evaluated ValQID on two primary benchmarks:

  • DevTag: A large public dataset for IoT device fingerprinting containing over 75,000 unique web responses covering 80 vendors and 20 device types.
  • BLMProbe: A dataset of protocol responses collected from active internet probing, featuring 67 vendor classes.

Key Insights and Ideas

  • Context is King: A single text fragment like "Admin" is ambiguous. By using a relation graph, ValQID understands that "Admin" inside a "Login Form" linked to a "Firmware Update" page is a much stronger signal for specific device types.
  • The Power of "No": One of ValQID's biggest strengths is knowing when not to compute. By estimating the net value, it can skip semantic analysis for pages that are either very obvious (explicit brand names) or completely irrelevant (generic error pages).
  • Long-Tail Success: Most identification models struggle with "long-tail" vendors—brands that appear rarely in the training data. ValQID excels here because its semantic questions focus on functional cues (what the device does) rather than just textual patterns (what the device says).

Key Results

ValQID significantly outperforms existing state-of-the-art (SOTA) methods like BLMProbe and RoBERTa-based classifiers:

  • Accuracy Gains: It achieved relative gains of up to 47.1% in joint macro-F1 (correctly identifying both vendor and type).
  • Long-Tail Performance: For rare vendors, it saw a massive improvement of up to 82.1% in macro-F1 compared to the strongest baseline (see Figure 3).
  • Efficiency: DSEJEV provided speedups of 3.27× to 8.58× over general-purpose LLMs like Qwen3.5 or DeepSeek (see Figure 4).
  • Query Reduction: The value-based selection policy reduced the number of semantic queries needed by over 53% while maintaining high accuracy.
Figure 3: Vendor F1 gains by training frequency. Stems show per-vendor F1 differences in percentage points (pp). Panel labels show low-frequency macro-F1 gains as percentages of the corresponding baseline’s macro-F1.
Figure 4: Joint identification accuracy (%) versus semantic execution time (s per 1,000 responses). Circles denote BLM- Probe; triangles denote DevTag.

Limitations and Future Work

While highly effective, ValQID currently relies on a fixed library of question templates and relation types discovered during an offline screening phase. If a completely new type of web interface architecture emerges, the current library might need manual or semi-automated updates.

The authors suggest that future work could involve dynamically adapting question groups and their value estimates in real-time as firmware and web interfaces evolve, further reducing the need for human intervention in the "cat-and-mouse" game of device identification.

Summary updated

Abstract

Web interfaces are widely used to access network services and manage networked devices. Identifying the vendors and types of devices behind Web interfaces is a critical step in asset inventory, security management, and risk assessment. Device identity cues are dispersed across page text, configuration fields, and management functions, and interpreting them often requires linking related Web objects. Responses and objects require different amounts of semantic analysis, so exhaustively processing all candidate content can introduce unnecessary computation. We present ValQID, which uses a relation graph to combine dispersed device identity cues, evaluates semantic questions efficiently with DSEJEV, and accounts for complementarity among semantic questions when estimating computation value. Based on the current identification state, ValQID weighs each question group's expected identification benefit against its execution cost and adaptively allocates computation to identify device vendors and types. Evaluation on real Web response data shows that ValQID achieves relative gains of up to 47.1% in joint macro-F1 and up to 82.1% in macro-F1 for low-frequency vendors over a state-of-the-art method. DSEJEV speeds up semantic processing by up to 8.58 times compared with general-purpose language models.
BibTeX
@misc{yang2026valqid,
  author = {Yifan Yang and Zhen Huang and Weijia Shi and Xichen Tan and Zhihuang Liu and Xionglve Li and Zhiping Cai},
  title = {{ValQID: Learning the Net Value of Semantic Query Groups for Web Device Identification}},
  year = {2026},
  eprint = {2611.04576},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.04576},
}
Figure