Hybrid AI for Responsive Multi-Turn Online Conversations with Novel Dynamic Routing and Feedback Adaptation

Priyaranjan Pattnayak, Amit Agarwal, Hansa Meghwani, Hitesh Laxmichand Patel, Srikant Panda

Abstract

Modern enterprise customer support requires conversational agents that are both highly accurate and extremely responsive. Traditional systems often force a trade-off between the speed of intent-based architectures and the flexibility of Retrieval-Augmented Generation (RAG). This paper presents a hybrid conversational AI framework that bridges this gap using a dynamic, confidence-based router and a continuous feedback-driven adaptation loop. Our system achieves a 95% accuracy rate with a median latency of 180ms, significantly outperforming standalone RAG or intent-based baselines in multi-turn environments.

1. Introduction

In enterprise customer support, the "latency-accuracy gap" remains a significant barrier to user satisfaction. Users typically expect sub-200ms responses for routine tasks, such as password resets or status checks, which are traditionally handled by intent-based systems. However, these systems are brittle and fail when queries deviate from predefined scripts. Conversely, RAG systems offer the reasoning capabilities required for complex troubleshooting but are often slow, expensive, and prone to hallucinations or the use of stale information.

This research introduces a modular architecture designed to provide the "best of both worlds." By dynamically routing queries based on semantic confidence and incorporating real-time user feedback to adjust system behavior, we provide a solution that is both conversationally fluid and computationally efficient.

2. System Architecture

The proposed framework is composed of four primary modules designed to handle the complexities of multi-turn dialogue:

2.1 Intent Classifier and Context Manager

The system utilizes a fine-tuned BERT model to generate semantic embeddings for user queries. To maintain coherence across multi-turn conversations, a Dialogue Context Manager employs a Bi-Encoder to track historical turns. A lightweight Cross-Encoder then augments the current query embedding with this historical context, ensuring the system understands the user's intent within the broader conversation flow.

2.2 The Dynamic Router

The core of the framework is the Dynamic Router, which assigns a confidence score ($c$) to each query. Queries are categorized into three distinct execution paths:

  1. FAQ ($c > 0.85$): The system immediately returns a high-confidence "canned" response.
  2. Out-of-Domain ($c \leq 0.5$): The query is routed exclusively to the RAG pipeline for deep retrieval.
  3. Contextual ($0.5 < c \leq 0.85$): The system triggers a Hybrid Response. In this mode, an LLM blends the canned response with RAG-retrieved context using a weighted prompt to ensure the output is both grounded and natural.

2.3 Feedback-Driven Adaptation

Unlike static systems, this framework incorporates a Negative Feedback Rate (NFR). If a specific intent receives consistent negative user feedback (e.g., "thumbs down"), the system automatically raises the confidence threshold for that intent. This forces the router to be more cautious, shifting future queries on that topic toward the RAG pipeline. Additionally, the system clusters unhandled queries to suggest new intents for human review.

3. Experimental Setup

The system was evaluated using two internal datasets from Oracle:

  • Fine-tuning Set: 35,000 human-to-customer conversations used to train the BERT-based intent classifier.
  • Evaluation Set: 10,000 queries (40% FAQ, 30% Contextual, 30% Out-of-Domain). Notably, 20% of these queries included follow-up turns to rigorously test multi-turn coherence.

4. Results and Discussion

Experimental results demonstrate that the hybrid approach significantly improves performance across all key metrics compared to pure RAG and canned-response baselines.

4.1 Accuracy and Latency

The hybrid system achieved an overall accuracy of 95%, surpassing pure RAG (91%) and the canned-response baseline (53%). Crucially, the median latency was recorded at 180ms, whereas the pure RAG approach required 380ms—more than double the response time.

4.2 Operational Efficiency

We introduced "Turn Efficiency" as a metric to measure the average number of turns required to resolve a query. The hybrid model reduced this to 1.7 turns, compared to 2.3 turns for pure RAG. Furthermore, the system achieved a cost efficiency score of 0.7 (on a scale where 1.0 is the theoretical minimum cost), more than doubling the efficiency of the RAG-only baseline (0.3).

4.3 Scalability

Stress tests with up to 50,000 concurrent queries showed that the system maintained 92% accuracy, confirming its viability for large-scale enterprise deployments.

5. Limitations and Future Work

While the system performs well, several challenges remain. We observed "long-turn drift" in conversations exceeding 10 turns, where the context manager occasionally lost the initial intent. Additionally, 25% of errors were attributed to "retrieval inaccuracy" caused by stale documents in the knowledge base. Future research will focus on distributed architectures to further reduce peak-load latency and expanding the framework to support multilingual deployments beyond the current English-centric model.

6. Conclusion

By combining the speed of intent-based systems with the generative power of RAG through dynamic routing and feedback loops, we have demonstrated a framework capable of meeting the rigorous demands of enterprise customer support. This hybrid approach ensures that responses are not only fast but also contextually accurate and self-improving over time.

Summary updated

BibTeX
@misc{pattnayak2025hybrid,
  author = {Priyaranjan Pattnayak and Amit Agarwal and Hansa Meghwani and Hitesh Laxmichand Patel and Srikant Panda},
  title = {{Hybrid AI for Responsive Multi-Turn Online Conversations with Novel Dynamic Routing and Feedback Adaptation}},
  year = {2025},
  eprint = {2506.02097},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.02097},
}
Figure