Efficient Large Language Model Application Development: A Case Study of Knowledge Base, API, and Deep Web Search Integration

Xiangyu Wang, Yan Tan, Tao Yang, Meng Yuan, Shaohan Wang, Min Chen, Feiyang Ren, Zijian Zhang, Yuqi ShaoJournal of Computer and Communications 2024 · DOI

Abstract

This paper presents a reference methodology for process orchestration that accelerates the development of Large Language Model (LLM) applications by integrating knowledge bases, API access, and deep web retrieval. By incorporating structured knowledge, the methodology enhances LLMs’ reasoning abilities, enabling more accurate and efficient handling of complex tasks. Integration with open APIs allows LLMs to access external services and real-time data, expanding their functionality and application range. Through real-world case studies, we demonstrate that this approach significantly improves the efficiency and adaptability of LLM-based applications, especially for time-sensitive tasks. Our methodology provides practical guidelines for developers to rapidly create robust and adaptable LLM applications capable of navigating dynamic information environments and performing effectively across diverse tasks.

Introduction

Despite their impressive reasoning capabilities, Large Language Models (LLMs) suffer from three primary "bottlenecks" when deployed in professional settings: a lack of structured knowledge, the "frozen in time" problem of static training data, and the risk of hallucinations. In specialized fields like maritime logistics, medicine, or law, these limitations prevent LLMs from providing the precision and real-time accuracy required for production-ready applications. This paper proposes a methodology to move beyond simple chatbots toward "orchestrated agents" that can navigate dynamic information environments through a triple-stream integration framework.

Methodology: The Triple-Stream Framework

The proposed framework orchestrates three critical data streams to ground LLM responses in reality:

  • Knowledge Base Integration: For stable, structured facts, such as regulatory frameworks and standard operating procedures.
  • Open API Access: For real-time data retrieval, such as current weather or live GPS coordinates.
  • Deep Web Retrieval: Utilizing search APIs to scrape, clean, and rerank web results for "breaking news" or highly specific external context.

To ensure efficiency, the system utilizes an embedding-based router. Instead of using expensive LLM calls to classify user intent, the framework uses vector similarity to map queries to specific data routes (e.g., Ship APIs vs. Port APIs) with near 100% accuracy and lower latency. Safety is maintained via a Deterministic Finite Automaton (DFA) for high-speed sensitive word filtering.

Maritime Domain Case Study

The framework was evaluated using domain-specific datasets from the shipping industry, including IMO (International Maritime Organization) reports and real-time vessel tracking data. A critical component of this implementation is the standardized API design, which allows the LLM to reliably parse and interact with external maritime services.

List of Ship APIs used in the maritime case study

As illustrated in the figure above, the system integrates various endpoints to fetch vessel details by MMSI, retrieve latest AIS information, and gather operational efficiency statistics. This integration allows the agent to answer complex queries, such as "Berthing situation at Yangshan Port," by combining static knowledge with live operational data.

Performance and Results

The framework was tested on real-world maritime queries, demonstrating qualitative success in integrating disparate data points. Key performance metrics include:

  • Latency: Total system latency ranged from 16.6 to 35 seconds, depending on the complexity of the web retrieval.
  • TTFT (Time to First Token): Averaged between 5.2s and 14.2s.
  • TPOT (Time Per Output Token): Remained stable at approximately 30–35ms.

While web search remains the most time-consuming element, the resulting context density significantly reduces hallucinations and provides comprehensive answers that a base LLM could not generate.

Conclusion and Future Work

This methodology provides a practical roadmap for building robust LLM applications. Future work will focus on conflict resolution heuristics when data sources provide contradictory information. Additionally, the authors suggest exploring GraphRAG for better relationship mapping and N2SQL (Natural Language to SQL) to allow LLMs to query structured databases directly without the need for pre-defined API endpoints.

Summary updated

Abstract

This paper presents a reference methodology for process orchestration that accelerates the development of Large Language Model (LLM) applications by integrating knowledge bases, API access, and deep web retrieval. By incorporating structured knowledge, the methodology enhances LLMs’ reasoning abilities, enabling more accurate and efficient handling of complex tasks. Integration with open APIs allows LLMs to access external services and real-time data, expanding their functionality and application range. Through real-world case studies, we demonstrate that this approach significantly improves the efficiency and adaptability of LLM-based applications, especially for time-sensitive tasks. Our methodology provides practical guidelines for developers to rapidly create robust and adaptable LLM applications capable of navigating dynamic information environments and performing effectively across diverse tasks.
BibTeX
@article{wang2024efficient,
  author = {Xiangyu Wang and Yan Tan and Tao Yang and Meng Yuan and Shaohan Wang and Min Chen and Feiyang Ren and Zijian Zhang and Yuqi Shao},
  title = {{Efficient Large Language Model Application Development: A Case Study of Knowledge Base, API, and Deep Web Search Integration}},
  journal = {Journal of Computer and Communications},
  year = {2024},
  doi = {10.4236/jcc.2024.1212011},
}
Figure