Enhancing Cloud-Based Large Language Model Processing with Elasticsearch and Transformer Models

Chunhe Ni, Jiang Wu, Hongbo Wang, Wenran Lu, Chenwei ZhangarXiv 2024 · DOI · arXiv

Introduction

Deploying Large Language Models (LLMs) in cloud environments presents a significant challenge: balancing the immense power of these models with the computational costs of training and querying massive datasets. Traditional keyword-based search mechanisms often fail to capture the semantic intent of user queries, leading to suboptimal retrieval. This paper explores a hybrid architecture that integrates Elasticsearch as a high-performance vector database with Transformer models to enhance search relevance and resource optimization.

Proposed Hybrid Architecture

The researchers propose a system that moves beyond simple keyword matching to semantic/vector search. By utilizing Elasticsearch to perform "hybrid search" (simultaneous text and vector search), the system can store vast amounts of unstructured data while providing the contextual information LLMs need to interpret query intent accurately.

Hybrid Search Architecture using Elasticsearch and LLMs

Key technical components of this methodology include:

  • Elasticsearch as a Vector Store: Acts as the backbone for similarity searches, allowing the LLM to retrieve relevant document summaries based on semantic embeddings.
  • RMSNorm (Root Mean Square Layer Normalization): Implemented to stabilize Transformer training, preventing overfitting and improving generalization by normalizing input vectors before attention layers.
  • Model Ensembling: Techniques such as Bagging, Boosting, and Stacking are discussed to combine multiple Transformer models into a more stable, higher-performing integrated model.

Dataset and Preprocessing

The study utilizes the Yelp Dataset, focusing on the restaurant category, which includes over 4.7 million reviews. A primary challenge identified was the significant data bias; the raw data was heavily skewed toward 4 and 5-star reviews, accounting for 67.94% of the total.

Initial Distribution of Yelp Review Star Ratings showing heavy bias toward positive reviews

To prevent the model from developing a "positivity bias," the authors created a balanced sub-dataset of 1 million comments, ensuring exactly 200,000 reviews for each star rating (1 through 5). This balanced approach is critical for accurate sentiment analysis and emotion prediction.

Key Insights and Performance Analysis

Challenging Scaling Laws

A major theoretical contribution of this work is the reassessment of compute logic. While traditional literature suggests model size should grow faster than training tokens (a 5.5x vs. 1.8x split), the authors find that model size and training tokens should scale in equal proportion to optimize cloud resources effectively.

Model Comparisons

The researchers compared several Transformer architectures on the balanced Yelp test set:

  • RoBERTa: Emerged as the optimal model, outperforming multi-parameter BERT by 0.62% in accuracy with similar latency.
  • BERT (Base/Large): Found to be 1.02% more accurate than the "Cased" version.
  • DistilBERT: Identified as the best choice for production environments, offering a 2x speedup over basic BERT with only a marginal 1.36% dip in accuracy.

The study achieved a peak classification accuracy of 69.75% for sentiment prediction across the five-star scale.

Conclusion and Future Work

The integration of Elasticsearch with Transformer models provides a robust framework for semantic intent recognition and efficient data retrieval. While computational constraints limited the exploration of multimodal models, future work will focus on incorporating non-textual data (images and video) and expanding this framework to fields such as automated driving and financial forecasting. By optimizing training times and resource allocation, the authors aim to reduce the environmental and financial costs of large-scale cloud deployments.

Summary updated

BibTeX
@misc{ni2024enhancing,
  author = {Chunhe Ni and Jiang Wu and Hongbo Wang and Wenran Lu and Chenwei Zhang},
  title = {{Enhancing Cloud-Based Large Language Model Processing with Elasticsearch and Transformer Models}},
  year = {2024},
  eprint = {2403.00807},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2403.00807},
}
Figure