CoCoTen: Detecting Adversarial Inputs to Large Language Models through Latent Space Features of Contextual Co-occurrence Tensors

Sri Durga Sai Sowmya Kadali, Evangelos E. Papalexakis

Introduction

Large Language Models (LLMs) are increasingly integrated into various applications, yet they remain vulnerable to "jailbreak" attacks—carefully crafted prompts designed to bypass safety filters and elicit harmful content. Existing detection methods often rely on computationally expensive transformer models or require extensive labeled datasets, which are difficult to maintain as new attack vectors emerge.

This paper introduces CoCoTen, a novel detection framework that shifts the focus from deep semantic understanding to the structural analysis of word patterns. By utilizing multi-linear algebra and tensor decomposition, CoCoTen identifies adversarial prompts with high precision, extreme data efficiency, and significantly lower latency than state-of-the-art baselines.

Methodology: The CoCoTen Pipeline

CoCoTen treats jailbreak detection as a structural analysis problem. Instead of processing text through a deep neural network, it builds a mathematical representation of word relationships based on their proximity in the text.

The CoCoTen pipeline: from input prompts to latent space separation.

The process involves four primary stages:

  1. Windowed Co-occurrence: For each prompt, the system calculates how frequently words appear near each other within a sliding window (e.g., 5 tokens). This captures local structural context without requiring full semantic parsing.
  2. Tensor Construction: These individual matrices are stacked into a 3D Contextual Co-occurrence Tensor, where the third dimension represents different prompts in the dataset.
  3. CP Decomposition: Using CANDECOMP/PARAFAC (CP) decomposition, the high-dimensional tensor is factorized into factor matrices. One of these matrices contains latent embeddings that represent the "structural signature" of each prompt.
  4. Semi-Supervised Classification: Because the decomposition is unsupervised, it captures the underlying distribution of the entire dataset. A K-Nearest Neighbors (KNN) classifier is then used to propagate labels from a small set of known examples to the rest of the latent space.

Experimental Evaluation

The authors evaluated CoCoTen across four datasets, including standard jailbreak classification sets (DS1), repositories of known LLM jailbreaks (DS2), and in-the-wild examples from the Allen AI "WildJailbreak" dataset (DS4).

Key Results

CoCoTen demonstrates significant advantages in three areas:

1. Data Efficiency: In low-data regimes, CoCoTen significantly outperforms traditional classifiers. On DS1, it achieved an F1 score of 0.83 using only 0.5% labeled data, representing a 96.6% improvement over Support Vector Machine (SVM) baselines under similar constraints.

2. Computational Speed: CoCoTen is designed for real-time deployment. It achieved speedups ranging from 2.3x to 128.4x compared to standard detectors. Specifically, it was 22x to 72x faster than BERT-based classifiers.

3. Latent Space Separation: Visualization of the latent embeddings confirms that the tensor decomposition effectively captures the differences between benign and adversarial prompts, creating distinct clusters.

t-SNE visualization showing clear separation between benign and jailbreak prompts in the latent space.

Discussion and Limitations

The effectiveness of CoCoTen stems from its ability to capture the "structural signature" of jailbreaks—such as repetitive roleplay instructions or unusual formatting—which often differ from benign queries. However, performance varied across datasets. The authors hypothesized that lower performance on certain sets (like DS2 and DS4) might be due to dataset-specific irregularities, such as high redundancy or the artificial nature of GPT-generated prompts.

Conclusion

CoCoTen offers a robust, fast, and data-efficient alternative to heavyweight LLM safety filters. By leveraging the latent structure of word co-occurrences, it provides a scalable defense mechanism capable of identifying novel adversarial inputs with minimal supervision.

Summary updated

Abstract

The widespread use of Large Language Models (LLMs) in many applications marks a significant advance in research and practice. However, their complexity and hard-to-understand nature make them vulnerable to attacks, especially jailbreaks designed to produce harmful responses. To counter these threats, developing strong detection methods is essential for the safe and reliable use of LLMs. This paper studies this detection problem using the Contextual Co-occurrence Matrix, a structure recognized for its efficacy in data-scarce environments. We propose a novel method leveraging the latent space characteristics of Contextual Co-occurrence Matrices and Tensors for the effective identification of adversarial and jailbreak prompts. Our evaluations show that this approach achieves a notable F1 score of 0.83 using only 0.5% of labeled prompts, which is a 96.6% improvement over baselines. This result highlights the strength of our learned patterns, especially when labeled data is scarce. Our method is also significantly faster, speedup ranging from 2.3 to 128.4 times compared to the baseline models.
BibTeX
@inproceedings{kadali2025cocoten,
  author = {Sri Durga Sai Sowmya Kadali and Evangelos E. Papalexakis},
  title = {{CoCoTen: Detecting Adversarial Inputs to Large Language Models through Latent Space Features of Contextual Co-occurrence Tensors}},
  booktitle = {ACM International Conference on Information and Knowledge Management},
  year = {2025},
}
Figure