APPLICATION OF ARTIFICIAL INTELLIGENCE IN COMPUTER NETWORK TECHNOLOGY UNDER THE BACKGROUND OF BIG DATA

Lei Ma, V Talanin, O Seregina, M Mansourvar, M Ismail, S Kareem, R Raj, F Nasaruddin, R Mahmud, R Abdullah, N Idris, N Li, N Matsuda, W Cohen, K Koedinger, I Ahmad, J Kamruzzaman, D Habibi, Y Hu, C Guo, E Ngai, S Chen, C Ollivier-Gooch, L Diachin, M Shephard, T Tautges, J Kraftcheck, V Leung, X Luo, M Miller, C Modi, D Patel, B Borisaniya, C Panos, C Xenakis, P Kotzias, H Jo, S Kim, D Won, G Sakkis, I Androutsopoulos, G Paliouras, V Karkaletsis, C Spyropoulos, P Stamatopoulos, A Martn, C Len, F Biscarri, Y Sandamirskaya, M Burtsev, S Al-Fedaghi, A Mohamoud, T Subramaniam, H Jalab, A TaqaInternational Journal of Mechatronics and Applied Mechanics 2018 · DOI

TL;DR

This paper explores how artificial intelligence can be leveraged to secure computer networks within the "Big Data" landscape. The author develops and tests an intelligent anti-spam system that uses an improved Bayesian classification algorithm to filter malicious emails with high accuracy and increasing efficiency as data volume scales.

What Problem Does This Paper Address?

As we move deeper into the era of Big Data—characterized by the "4Vs" of Volume, Velocity, Variety, and Veracity—traditional manual network management has become obsolete. The sheer scale of data transmission makes networks vulnerable to security threats, particularly spam, which serves as a primary vector for viruses and phishing attacks.

The challenge lies in creating a system that can not only identify spam with high precision but also handle the high-speed processing requirements of modern networks without becoming a bottleneck. The author seeks to bridge the gap between theoretical AI and practical network security by designing a system capable of intercepting, reconstructing, and classifying email traffic in real-time.

Methods

The researchers designed an intelligent anti-spam system based on a Linux platform firewall. The technical approach consists of three main stages:

  1. Intercepting and Redirection: The system monitors SMTP (port 25) and POP3 (port 110) traffic. Using a "Hook" function at the kernel level, it redirects mail data packages to a user-mode program via NF-QUEUE for analysis.
  2. Restoration: Because IP data is often fragmented during transmission, the system uses the sk_buff structure to reassemble fragments into complete emails. This ensures the AI classifier analyzes the full context of the message rather than isolated packets.
  3. Improved Bayesian Classification: The core "brain" of the system is a Bayesian algorithm. Unlike standard filters, this version incorporates variables for "background information concentration" and "hypothetical probability." It calculates the probability of an email being spam by comparing word frequencies against a learned lexicon of known spam and legitimate ("ham") emails.

Datasets

The paper does not utilize a standard public benchmark (like the Enron Spam dataset). Instead, the authors created a controlled Testing Environment consisting of a mail sender, a mail receiver, and a mail gateway.

  • Evaluation Sets: The system was tested using four distinct batches of emails: 500, 1,000, 1,500, and 2,000 messages.
  • Data Types: The tests were split between sets of confirmed spam and sets of confirmed non-spam (legitimate) emails to measure identification degrees and processing times.

Key Insights and Ideas

  • The "More is Faster" Paradox: One of the most interesting findings is that the system's identification speed actually increased as the number of emails grew. This suggests that the overhead of the system is well-optimized for high-volume environments, making it particularly suitable for the Big Data era.
  • Asymmetric Accuracy: The system was consistently better at identifying legitimate emails (non-spam) than it was at identifying spam. This is a critical design choice in networking; "false positives" (blocking a real email) are generally considered more damaging than "false negatives" (letting a spam email through).
  • Kernel-to-User Efficiency: By using NF-QUEUE to move complex AI processing out of the OS kernel and into user space, the system maintains network stability while allowing for sophisticated computational analysis that would be too risky to run in the kernel.

Key Results

The experimental results validated the system's performance across several metrics:

  • Accuracy (Identification Degree):
    • Non-spam: Achieved a high success rate between 97.3% and 98.7%.
  • Spam: Remained stable between 91.7% and 93.6%.
  • Processing Speed: As the volume increased from 500 to 2,000 emails, the identification time per batch actually trended downward. For example, the time to process a non-spam email dropped from 541.1 ms (at the 500-email mark) to 520.3 ms (at the 2,000-email mark).
  • Stability: The system demonstrated "favourable filtering performance" and operated stably even under cost-sensitive conditions when integrated with the MetaCost framework.

Limitations and Future Work

While the paper demonstrates a robust proof-of-concept, there are a few areas for further exploration:

  • Scope of Filtering: The current study focuses exclusively on email. The authors mention that the system is capable of filtering websites and text messages, but these were not discussed in the experimental results.
  • Algorithm Evolution: While the improved Bayesian approach is effective, the paper does not compare its performance against more modern deep learning architectures (like Transformers or RNNs) which are currently the state-of-the-art in Natural Language Processing.
  • Real-world Noise: The testing environment was controlled. Future work would benefit from testing against "live" network traffic where the variety of protocols and encryption (like SMTPS) might present additional hurdles for packet interception.

Summary updated

Abstract

With the rapid development of society and technology, computer technology is also developing. Under the background of big data, the emergence of artificial intelligence is greatly beneficial to the life and work of people and moreover drives computer technology to be informationalized and intelligentized. The emergence and application of artificial intelligence enhances working efficiency and promotes the further development of many new technology fields in addition to enriching the life of people. Application of artificial intelligence in computer network can be classified to network security management, system evaluation and network management and artificial intelligence agent. This study firstly analyzed relevant theories of big data era and artificial intelligence, then designed the model structure of intelligent anti-spam in network security management, and tested the model. The results suggested that the intelligent anti-spam model had favourable filtering performance.
BibTeX
@article{ma2018application,
  author = {Lei Ma and V Talanin and O Seregina and M Mansourvar and M Ismail and S Kareem and R Raj and F Nasaruddin and R Mahmud and R Abdullah and N Idris and N Li and N Matsuda and W Cohen and K Koedinger and I Ahmad and J Kamruzzaman and D Habibi and Y Hu and C Guo and E Ngai and S Chen and S Chen and C Ollivier-Gooch and L Diachin and M Shephard and T Tautges and J Kraftcheck and V Leung and X Luo and M Miller and C Modi and D Patel and B Borisaniya and C Panos and C Xenakis and P Kotzias and H Jo and S Kim and D Won and G Sakkis and I Androutsopoulos and G Paliouras and V Karkaletsis and C Spyropoulos and P Stamatopoulos and A Martn and C Len and F Biscarri and Y Sandamirskaya and M Burtsev and S Al-Fedaghi and A Mohamoud and T Subramaniam and H Jalab and A Taqa},
  title = {{APPLICATION OF ARTIFICIAL INTELLIGENCE IN COMPUTER NETWORK TECHNOLOGY UNDER THE BACKGROUND OF BIG DATA}},
  journal = {International Journal of Mechatronics and Applied Mechanics},
  year = {2018},
  doi = {10.17683/ijomam/issue4.10},
}
Figure