Machine-learned intrusion detectors for Internet-of-Things (IoT) networks routinely report accuracy above 99% on public benchmarks, yet they produce unmanageable false alarms once deployed. A well-documented reason is that these benchmarks are captured in scripted testbeds. Each testbed leaves fingerprints in the traffic, such as fixed attacker addresses, regular packet sizes, and tool-specific timing, and a classifier can learn those fingerprints instead of attack behavior. We propose a detector built to separate these two kinds of signal. A small neural network has one shared branch that sees traffic from every training testbed and several routed branches, one per testbed, that are deliberately allowed to absorb that testbed's quirks. At test time on a new network, the routed branches are thrown away. A gradient-boosted tree classifier is then fit on the shared branch's output, and it can be re-fit cheaply from a handful of labeled flows collected at the new site.
The two-sentence version is this. Give each synthetic testbed its own disposable expert so that its artifacts have somewhere to go other than the shared representation, then classify with a tree model on what remains. Success is measured by how well a detector trained on some testbeds works on a testbed, or a physical network, it never saw, not by in-benchmark accuracy.
Core Claim
A mixture of experts is a network whose layers contain several parallel sub-networks ("experts"), with some rule deciding which experts process each input. Here the rule is simple: during training, a flow from testbed $e$ passes through the shared expert $s$ and through routed expert $r_e$. The claim is that this structure, combined with a penalty that keeps the shared expert from encoding testbed identity, yields shared embeddings that transfer across capture environments better than three alternatives: one network trained on pooled data, manual removal of known shortcut features, or a single network trained with an invariance penalty. We further claim that a tree-ensemble head on the shared embedding transfers better than a softmax head and recalibrates from few labels. Under this claim, the hybrid neural-statistical design is the right tool for low-label adaptation, not a decoration.
The training objective, for flows $x$ with labels $y$ from training environments $e \in \mathcal{E}$, is
$$\min_{s, \{r_e\}, g} \sum_{e \in \mathcal{E}} \mathbb{E}_{(x,y)\sim e}\big[\ell(g([s(x), r_e(x)]), y)\big] + \lambda_1 \, \mathrm{Decor}(s, r_e) - \lambda_2 \, \ell_{\mathrm{env}}(q(s(x)), e).$$
Here $g$ is a training-time linear head, $\ell$ is cross-entropy, and $\mathrm{Decor}$ is the Pearson-correlation decorrelation loss between module outputs taken from MT-CRL. The function $q$ is an auxiliary classifier that tries to predict the environment from the shared embedding. The negative sign means $s$ is trained, through a gradient-reversal step, to make that prediction fail. After training, $g$ and all $r_e$ are discarded, and an XGBoost model $h$ is fit on $s(x)$.
Positioning
Three hybrid intrusion detectors motivate the design and show its gap. Anomaly Detection Using XGBoost Ensemble of Deep Neural Network Models stacks an MLP, a backpropagation network, and an LSTM under an XGBoost meta-learner and reaches 99.5% on UNSW-NB15. Ensemble Classification for Intrusion Detection via Feature Extraction Based on Deep Learning compresses flows with an autoencoder and fuses SVM and nearest-neighbor classifiers with Dempster-Shafer evidence theory. SIP Flooding Attack Detection Technology of Multi-agent System Covert Network Based on BiGRU Algorithm replaces a recurrent network's softmax with an SVM and is evaluated only in Mininet with Scapy-generated floods. All three train and test within a single benchmark or simulation, so none measures whether the neural features survive a change of capture environment. That question is our entire target. We reuse their decoupling of neural features from a statistical head, but we add the environment-partitioned experts and the cross-environment protocol that they lack.
The mixture-of-experts literature supplies the mechanism. MDFEND: Multi-domain Fake News Detection gates experts by known domain labels and reaches 0.9137 F1 on Weibo21. It keeps the domain-specific path at inference, whereas we deliberately discard it, because our "domains" are artifact sources rather than legitimate subpopulations. Improving Multi-Task Generalization via Regularizing Spurious Correlation uses expert decorrelation and an environment-gradient penalty to cut spurious task-module links, reporting a 5.5% average gain over vanilla multi-task learning. We borrow its decorrelation loss and its use of known environment partitions, which in our setting are free because each dataset is its own environment. We avoid its second-order penalty, which that paper itself flags as expensive. Decoding Knowledge Attribution in Mixture-of-Experts found that shared experts carry basic, general structure while routed experts do fine-grained refinement in language models. We use that finding as a hypothesis to test in traffic: does the shared expert carry protocol-level regularities while the routed experts carry testbed-specific ones?
The reality-gap literature constrains the evaluation. Towards Generating Real-life Datasets for Network Intrusion Detection argues that synthetic traffic has artificial regularity in sizes, timing, and addresses. It builds the 250-host TUIDS physical testbed and finds that false-positive rates differ substantially from synthetic benchmarks, but reports no single headline number. HADES-IoT and DeviceRadar show that evaluation on real devices and real backbone traces is feasible. DeviceRadar also shows that tree classifiers over learned packet-size embeddings compile to programmable switches at 40 Gbps, which is the deployment argument for keeping the final classifier a tree.
Research Program
The first step builds a common substrate. Benchmarks differ in their published feature sets, so where raw packet captures are available we re-extract one metadata-only flow representation: packet-length and inter-arrival sequences, flag counts, and durations, with no payload, consistent with encrypted traffic. Where captures are not available, we fall back to the intersection of published flow features. We also map attack labels onto a shared coarse taxonomy (benign, DoS/DDoS, scanning, and so on). The environments are UNSW-NB15, ToN_IoT, CICIDS2017, and the TUIDS physical testbed. Within each environment we also keep a time-ordered split, so that the cross-environment effect can be distinguished from simple temporal drift.
The second step is a diagnostic that comes before any detector work. We train a classifier to predict which environment a flow came from, separately for benign flows and for each attack class. If environment is near-perfectly predictable from benign traffic alone, the artifacts are real, large, and available for a detector to exploit. We then report which features drive this prediction, which tests the specific shortcuts Bhuyan et al. describe.
The third step trains the shared-plus-routed model under leave-one-environment-out, with a deliberately small backbone: a one-dimensional convolution over packet sequences followed by a small MLP, sized so that an integer-quantized version is plausible on a gateway. The fourth step fits XGBoost on the frozen shared embedding. We evaluate it zero-shot on the held-out environment, and then after re-fitting only the tree head on $k \in \{10, 50, 200\}$ labeled flows per class from that environment. The fifth step is attribution. We ablate the shared expert and each routed expert, then measure which environment-predictive and attack-predictive information each one carries, testing whether the basic-refinement split from the LLM study reappears here as a protocol-versus-artifact split.
Baselines
The naive reference is a single network of the same total parameter count trained on the pooled environments with a softmax head and tested on the held-out environment. It shows the size of the reality gap when nothing is done about it.
The strongest existing in-benchmark method is XGBoost directly on flow features. On a balanced UNSW-NB15 split, the EG Boosting study reports 98.81% accuracy for XGBoost and 77.46% balanced accuracy on CICIDS2017 (EG Boosting study). For hybrids, the DNN-XGBoost stack reports 99.5% accuracy on UNSW-NB15. None of these papers reports cross-environment results, and we found no reported result on Lacuna for leave-one-dataset-out transfer on these benchmarks. We will therefore rerun XGBoost-on-features and a reimplementation of the DNN-XGBoost stack under our splits, with the same features, the same training environments, and the same hyperparameter search budget per method (equal numbers of configurations, tuned on a validation environment that is distinct from the test environment, never on the test environment). The proposal beats this baseline if its held-out macro-F1 is higher and its false-positive rate at 95% attack recall is lower, averaged over held-out environments, with seed-level confidence intervals that do not overlap. A win on in-benchmark accuracy alone does not count.
The invariance baseline is one network trained with an environment-adversarial penalty and no routed experts, together with the same penalty applied in the IRM style. MT-CRL compared against both, and they test whether an invariance penalty alone does the work.
The key ablations remove one ingredient at a time. The first removes the routed experts while keeping parameters matched by enlarging the shared expert. The second keeps the routed experts at test time by routing to the nearest training environment. The third swaps the XGBoost head for a softmax head refit on the same $k$ labels. The fourth removes the decorrelation term.
Why not just…?
Why not just run XGBoost on the raw flow features? This is the most serious competitor in-benchmark, and trees are cheap and compile to switches. However, trees split greedily on whichever feature is most predictive. If testbed artifacts such as fixed addresses or tool-regular packet sizes are most predictive, the trees will split on them. The environment-prediction diagnostic and the leave-one-environment-out comparison settle this directly. If XGBoost on features matches the proposal out of environment, we will report that the neural branch is unnecessary.
Why not just delete the known shortcut features (addresses, ports, TTLs) and train a normal model? This is the alternative most likely to work as well, and we treat it as a real threat rather than a strawman. Every method, including ours, will also be run on this scrubbed feature set. The question is whether artifacts persist in timing and size distributions after scrubbing, which the environment classifier measures. If environment is no longer predictable from scrubbed benign traffic, and scrubbed XGBoost transfers as well as our model, the honest conclusion is that manual scrubbing suffices, and the expert design is only worth pursuing where artifacts are distributional rather than columnar.
Why not just train one network with an invariance or domain-adversarial penalty? Forcing a single representation to be environment-invariant can discard real attack signal whenever attack mix correlates with environment, which it does across these datasets. Routed experts give the environment-specific signal a place to go during training instead of forcing its removal from a single shared path. The parameter-matched ablation with no routed experts isolates exactly this difference.
Proposed Experiments
The decisive experiment is the leave-one-environment-out grid over UNSW-NB15, ToN_IoT, CICIDS2017, and TUIDS. It reports macro-F1, false-positive rate at 95% attack recall, and the transfer gap (in-environment minus held-out score) for every method on both raw and scrubbed features. TUIDS, being physical, is treated as the most informative held-out target.
The second experiment is the few-label recalibration curve. Refitting only the tree head on $k$ labeled flows per class should close more of the gap than refitting a softmax head on the same flows, consistent with Reinders et al.'s report of random-forest heads working at roughly 15 labels per class. If the softmax head matches it, the hybrid claim fails.
The third experiment checks whether the mechanism works as intended: environment-prediction accuracy from the shared embedding versus from the routed embeddings. The shared embedding should be near chance while the routed ones remain high. If the shared embedding still encodes environment, the quarantine is not working regardless of downstream accuracy.
As a final, smaller check, we capture traffic from a handful of physical IoT devices replaying the malware families HADES-IoT used, used purely as an unseen test environment. We also measure the size and inference latency of the quantized backbone plus tree head, to show whether gateway deployment is plausible. That deployment check is reported as a constraint and is not claimed as a contribution.
References
- Anomaly Detection Using XGBoost Ensemble of Deep Neural Network Models
- Ensemble classification for intrusion detection via feature extraction based on deep Learning
- SIP Flooding Attack Detection Technology of Multi-agent System Covert Network Based on BiGRU Algorithm
- MDFEND: Multi-domain Fake News Detection
- Improving Multi-Task Generalization via Regularizing Spurious Correlation
- Decoding Knowledge Attribution in Mixture-of-Experts: A Framework of Basic-Refinement Collaboration and Efficiency Analysis
- Towards Generating Real-life Datasets for Network Intrusion Detection
- HADES-IoT: A Practical Host-Based Anomaly Detection System for IoT Devices
- DeviceRadar: Online IoT Device Fingerprinting in ISPs using Programmable Switches
- Enhancing Network Security: A Study on Classification Models for Intrusion Detection Systems