Abstract
Traditional Natural Language Interfaces to Databases (NLIDB) focus primarily on the translation of natural language to SQL. However, in high-stakes domains like cybersecurity, simply retrieving data is insufficient. This paper introduces a novel framework and the IoT-SQL dataset, which require models to not only generate SQL queries but also perform domain-specific reasoning to classify network traffic. By employing Multi-Task Learning (MTL), the researchers demonstrate that jointly training a model for both querying and classification significantly enhances SQL generation accuracy, enabling smaller models to outperform larger, single-task counterparts.
Introduction
The proliferation of Internet of Things (IoT) devices has led to an explosion of network data, making manual threat detection increasingly difficult. While text-to-SQL models have improved significantly, they remain "data-blind"—capable of fetching rows but unable to interpret the security implications of the retrieved information. Security analysts require systems that understand the context of the data, such as distinguishing between a routine update and a malicious attack.
Furthermore, existing benchmarks like Spider often lack the temporal complexity required for forensic analysis. This research addresses these gaps by proposing a system that integrates data retrieval with automated threat classification.
Methodology and Dataset
The proposed framework operates in two stages: first, translating natural language prompts into schema-aware SQL; and second, performing binary classification on the returned network traffic to identify malicious activity.
The core innovation is the use of Multi-Task Learning (MTL), fine-tuning a single model (specifically Flan-T5) to handle both tasks simultaneously. To support this, the authors created the IoT-SQL dataset, featuring 10,985 unique text-SQL pairs and nearly 240,000 rows of network traffic from real-world Zeek logs (IoT-23) and smart building sensors. The dataset includes complex temporal queries involving JOINs and HAVING clauses, reflecting the dynamic nature of IoT environments.
Results and Key Insights
The experimental results highlight the benefits of joint training. A multi-task T5-base model achieved an execution accuracy of 95.6% on the test set, surpassing a larger T5-large model (92.8%) that was trained solely on text-to-SQL. This suggests that understanding the underlying meaning of the data (classification) reinforces the model's ability to locate it (SQL generation).
Key findings include:
- Joint Training Synergy: Learning to classify data improves the precision of the retrieval mechanism.
- LLM Limitations: GPT-3.5 struggled with domain-specific tabular reasoning, often failing to identify malicious patterns despite few-shot prompting.
- Temporal Reasoning: The inclusion of over 1,000 temporal queries addresses a critical void in current NLIDB benchmarks, which often treat databases as static snapshots.
Conclusion and Future Work
This work demonstrates that integrating domain-specific reasoning into the text-to-SQL pipeline is essential for practical IoT defense. While the current model is highly effective within the IoT domain, future research will focus on generalizing these capabilities to other sectors like finance and healthcare. Additionally, the researchers plan to expand the classification task from row-level analysis to multi-row session reasoning to detect more sophisticated, long-term attack patterns.
Summary updated