Statistical Frameworks for Detecting Tunnelling in Cyber Defence Using Big Data

Daniel J. Lawson, Patrick Rubin-Delanchy, Nicholas A. Heard, Niall M. Adams

Introduction

Detecting "tunnelling"—where an attacker moves through a network using legitimate credentials—is a computationally expensive "needle in a haystack" problem. In modern cyber defense, while powerful statistical and machine learning models exist to detect subtle anomalies, they are often too computationally "heavy" to run across an entire enterprise network in real-time. This paper proposes a multilevel statistical framework that uses a fast "telescoping" algorithm to filter massive netflow data, allowing complex, high-fidelity statistical models to focus only on the most suspicious session chains.

The Tunnelling Problem

Tunnelling occurs when an attacker gains legitimate credentials and creates a chain of connections (e.g., SSH sessions) to move from an entry point to a target. Because the attacker uses valid logins, signature-based detection fails. Detection requires analyzing the timing and structure of connections. However, on a network with hundreds of thousands of IPs and millions of edges, comparing every possible connection pair for timing correlations is mathematically infeasible.

Methodology: A Multilevel Approach

The authors introduce a multilevel modelling approach designed to be "conservative," filtering out uninteresting data while retaining potential threats for deeper analysis.

The multilevel framework workflow (a) and the core concept of temporal telescoping (b).
  1. Sessionization: Raw netflow data is grouped into "sessions" using a 125-second time-out. If two records on the same edge occur within this window, they belong to the same session.
  2. Telescoping Discovery: The core innovation is "telescoping." If Node B receives an incoming session from Node A and initiates an outgoing session to Node C entirely contained within the first session's timeframe, these are "telescoped." A linear-time algorithm efficiently identifies these pairs.
  3. Statistical Scoring:
    • Session Level: A p-value ($p_{lm}$) is calculated to determine if telescoping happened by chance, using resampling to account for seasonal traffic.
  4. Chain Level: Fisher’s method combines p-values for longer paths (e.g., a 4-node chain).
  5. Netflow Level: Highly suspicious sessions undergo high-resolution correlation analysis of flow volumes (e.g., log-transformed flows per minute) to confirm if the data moving in matches the data moving out.

Experimental Evaluation

The study utilized real-world traffic from the Imperial College London network, containing approximately 345,098 IPs. To evaluate the framework, the authors simulated an SSH tunnel from a home PC (T1) through intermediate servers (T2, T3) to a target machine (T4), where malicious activity (software installation and video streaming) was performed to mimic data exfiltration.

Key Results and Insights

The framework achieved significant computational triage, reducing the data volume by 100x before applying expensive models.

  • SSH Analysis: In a dataset of 6,569 SSH records over 48 hours, the algorithm successfully identified the true attack path (T1-T2-T3-T4) with a highly significant p-value of 0.00011.
  • Action Identification: In an all-protocol study, the framework correctly identified a specific Google IP involved in simulated exfiltration with a correlation score of 0.96.
High-resolution netflow correlation across the identified attack chain (T1-T4), showing matching activity patterns.

Conclusion

By shifting the goal from "detect everything at once" to "filter fast, then model deep," this framework enables the use of sophisticated statistical models at scale. The temporal "telescoping" signature provides a powerful structural filter that is both physically grounded and computationally efficient, making it suitable for MapReduce environments and real-world big data volumes.

Summary updated

Abstract

How can we effectively use costly statistical models in the defence of large computer networks? Statistical modelling and machine learning are potentially powerful ways to detect threats as they do not require a human level understanding of the attack. However, they are rarely applied in practice as the computational cost of deploying all but the most simple algorithms can become implausibly large. Here we describe a multilevel approach to statistical modelling in which descriptions of the normal running of the network are built up from the lower netflow level to higher-level sessions and graph-level descriptions. Statistical models at low levels are most capable of detecting the unusual activity that might be a result of malicious software or hackers, but are too costly to run over the whole network. We develop a fast algorithm to identify tunnelling behaviour at the session level using 'telescoping' of sessions containing other sessions, and demonstrate that this allows a statistical model to be run at scale on netflow timings. The method is applied to a toy dataset using an artificial 'attack'.
BibTeX
@inproceedings{lawson2014statistical,
  author = {Daniel J. Lawson and Patrick Rubin-Delanchy and Nicholas A. Heard and Niall M. Adams},
  title = {{Statistical Frameworks for Detecting Tunnelling in Cyber Defence Using Big Data}},
  booktitle = {JISIC},
  year = {2014},
  doi = {10.1109/jisic.2014.47},
}
Figure