An Intelligent Framework for Issue Ticketing System Based on Machine Learning

Ruanda Qamili, Shaban Shabani, Johannes SchneiderEDOC Workshops 2018 · OpenReview

Abstract

Due to the rapid shift of companies towards superb customer experience and satisfaction, ticketing systems have come into a prominence and represent a strategic element in business competitiveness. Different software companies have developed very effective software tools for issue tracking, nevertheless, some sub-processes and tasks within the ticketing systems are still performed manually. These manually performed tasks represent bottlenecks, especially at large organizations they result in declined productivity and increased response time. Advancements in machine learning can be used in a novel way in which they are combined with the traditional issue tracking and ticketing systems on the market, to enable optimal operational efficiency in the Customer Service and Support(CSS) Department of large-scale businesses that deal with customer reports. This paper proposes an integrated approach to customer support by treating three seemingly different bottlenecks in the ticketing system: spam detection, ticket assignment and sentiment analysis. We use primary data to implement and apply the proposed machine learning approach. The evaluation shows promising results in terms of accuracy and efficiency of our approach.

Introduction

In large-scale Customer Service and Support (CSS) departments, manual processes often lead to significant operational bottlenecks, resulting in long wait times and decreased customer satisfaction. This research identifies three primary hurdles: the high volume of spam and unwanted automated notifications (often exceeding 20% of total volume), the slow and error-prone manual triage required to assign tickets to appropriate technicians, and the delayed nature of traditional feedback mechanisms. To address these, we propose an intelligent framework that embeds machine learning directly into existing ticketing infrastructure to automate spam detection, ticket assignment, and sentiment analysis.

Methodology

The proposed framework utilizes standard Natural Language Processing (NLP) techniques combined with supervised learning, prioritizing operational safety and precision.

The proposed intelligent framework for automated ticket management, illustrating the flow from initialization to ticket assignment and sentiment analysis.

Preprocessing and Classification

The system processes tickets in both English and German. Textual data from the subject and description fields are preprocessed using a Bag-of-Words approach with TF-IDF weighting. We evaluated several classification algorithms, including Support Vector Machines (SVM), Random Forest (RF), Naïve Bayes (NB), and Stochastic Gradient Descent (SGD).

The "Conservative Unanimity" Algorithm

A key contribution of this work is the "conservative unanimity" ensemble strategy for spam detection. Recognizing that a false positive (discarding a legitimate customer request) is a critical failure, the system only flags a ticket as spam if classifiers reach a high probability threshold (e.g., 0.95) or show unanimous agreement. This ensures that precision is prioritized over recall.

Sentiment Analysis

Due to a lack of domain-specific labeled sentiment data, the framework utilizes the pattern.en library. It analyzes adjectives to assign a polarity score (positive, neutral, or negative) to each ticket, serving as a real-time trigger for managerial intervention.

Data and Experimental Setup

The study utilized a primary dataset from a software company in Liechtenstein, comprising 18,917 records from 2009 to 2017. The data is linguistically diverse, with 72% German and 28% English entries. The dataset included 3,880 manually labeled spam or unwanted tickets (such as automated server logs), which were used to train and validate the models.

Results and Discussion

The experimental results demonstrate the high efficacy of machine learning in automating ticketing tasks.

Box plot comparing the accuracy of various machine learning classifiers across the tested tasks.
  • Spam Detection: Random Forest and SVM achieved nearly 99% accuracy in cross-validation. By implementing the "unanimity with confidence" threshold, the system achieved 100% precision, effectively eliminating false positives while still filtering out approximately 67% of incoming spam.
  • Ticket Assignment: SVM emerged as the top performer for automated triage, reaching an F1-score of 0.75. This performance level significantly reduces the manual workload for service desk staff.
  • Sentiment Trends: Analysis revealed distinct linguistic trends; German-speaking customers tended to submit more "positive" tickets (43.6%) than English-speaking customers (31.2%), who submitted a higher proportion of "neutral" tickets.

Conclusion and Future Work

This research demonstrates that integrating machine learning into ticketing systems can drastically improve operational efficiency without compromising service quality. The "conservative unanimity" approach provides a safe path for automating critical tasks like spam filtering.

Future work will focus on implementing unsupervised learning techniques like LDA to discover new issue categories without manual labeling, refining sentiment analysis for more granular emotional identification, and fully embedding the framework into commercial platforms like Jira or Zendesk via their APIs.

Summary updated

Abstract

Due to the rapid shift of companies towards superb customer experience and satisfaction, ticketing systems have come into a prominence and represent a strategic element in business competitiveness. Different software companies have developed very effective software tools for issue tracking, nevertheless, some sub-processes and tasks within the ticketing systems are still performed manually. These manually performed tasks represent bottlenecks, especially at large organizations they result in declined productivity and increased response time. Advancements in machine learning can be used in a novel way in which they are combined with the traditional issue tracking and ticketing systems on the market, to enable optimal operational efficiency in the Customer Service and Support(CSS) Department of large-scale businesses that deal with customer reports. This paper proposes an integrated approach to customer support by treating three seemingly different bottlenecks in the ticketing system: spam detection, ticket assignment and sentiment analysis. We use primary data to implement and apply the proposed machine learning approach. The evaluation shows promising results in terms of accuracy and efficiency of our approach.
BibTeX
@inproceedings{qamili2018intelligent,
  author = {Ruanda Qamili and Shaban Shabani and Johannes Schneider},
  title = {{An Intelligent Framework for Issue Ticketing System Based on Machine Learning}},
  booktitle = {EDOC Workshops},
  year = {2018},
  url = {https://openreview.net/forum?id=67J7D0BFs8L},
}
Figure