LiveVision: Real Time Object Recognition and Speech Synthesis using Machine Learning

Ms. Likita S. Shetty, Mayur Naik, Mr. Manjuram G. PrabhudessaiInternational Journal of Advanced Research in Science Communication and Technology 2025 · DOI

TL;DR

LiveVision is an Android-based assistive technology that merges real-time object detection (YOLOv4) with speech synthesis to create a more inclusive learning environment. By providing immediate auditory feedback for identified objects, the application serves a dual purpose: acting as an interactive educational tool for school-aged children and a vital navigation aid for visually impaired individuals.

What Problem Does This Paper Address?

The authors identify a "critical gap" in the current landscape of educational and accessibility software. Most existing applications are siloed—they either focus on interactive learning for children or accessibility features for the blind, but rarely both.

For visually impaired individuals, the lack of real-time environmental awareness tools creates significant barriers to independence. Meanwhile, school-aged children often lack engaging, hands-on tools that can adapt to their developmental needs. LiveVision seeks to bridge this divide by creating a "holistic, adaptive solution" that uses Machine Learning (ML) to turn visual data into immediate, actionable knowledge for diverse user groups.

Methods

The LiveVision system is built on a standard but robust machine learning pipeline optimized for mobile deployment:

  • Object Detection: The core engine utilizes YOLOv4 (You Only Look Once), a state-of-the-art real-time object detection algorithm known for its balance between speed and accuracy.
  • Framework: The model was developed using TensorFlow. To ensure the application could run smoothly on mobile hardware without requiring a constant high-speed server connection, the authors converted the models to TensorFlow Lite.
  • Speech Synthesis: The system integrates a high-quality Text-to-Speech (TTS) engine that translates the labels of detected objects into clear, articulate spoken descriptions.
  • Mobile Integration: The application is designed for the Android ecosystem (Minimum OS version 6.0), featuring camera integration for live detection and a gallery option for processing existing images.

Datasets

The paper notes that a "diverse dataset was curated," specifically encompassing images relevant to real-time object detection and age-appropriate educational content.

  • Training/Validation: The authors standardized the dataset through resizing and pixel normalization before training.
  • Specifics: While the paper references standard architectures like YOLOv4 (which are typically pre-trained on datasets like COCO or ImageNet), it does not explicitly name a specific public dataset or provide the exact size of their custom curated collection.

Key Insights and Ideas

  • Technology as an Equalizer: The central "aha moment" is the realization that the same technological stack—computer vision + speech synthesis—can solve two seemingly different problems: early childhood education and assistive navigation.
  • Mobile-First Accessibility: By optimizing for Android via TensorFlow Lite, the authors prioritize cost-effectiveness. This reduces the need for expensive, specialized hardware or human interpreters, making accessibility more attainable.
  • Universal Design: The UI is intentionally simplified. By focusing on "Simplicity" as a core feature, the app accommodates the cognitive level of a child and the tactile/auditory needs of a visually impaired user simultaneously.

Key Results

The paper focuses primarily on the functional implementation and qualitative success of the application:

  • Real-Time Performance: The system successfully performs instantaneous identification. Screenshots within the paper demonstrate the model identifying objects like laptops (80.08% confidence), people (65.63%), and cell phones (74.22%) in live environments.
  • User Interface: The authors successfully implemented a flow that includes secure OTP-based login, a documentation module explaining requirements (minimum 1GB RAM), and a seamless transition from camera capture to auditory output.
  • Functional Versatility: The app proved capable of adapting to various environments, including educational settings, public spaces, and workplaces.

Limitations and Future Work

While the paper presents a successful prototype, certain areas remain open for exploration:

  • Environmental Constraints: The paper does not explicitly discuss how the model performs in challenging conditions, such as low-light environments or highly cluttered scenes, which are common in real-world navigation.
  • Quantitative Benchmarking: While "high accuracy" is claimed, the paper lacks a detailed breakdown of Mean Average Precision (mAP) or latency metrics across different mobile chipsets.
  • Future Directions: The authors suggest that future iterations will incorporate Augmented Reality (AR), gamified learning elements, and collaborative tools to further enhance the interactive experience for children. They also aim to expand the "Customization Features," allowing users to select different languages and adjust the speed of object recognition.

Summary updated

Abstract

In the realm of education and accessibility, a significant gap exists for school-aged children as well as for visually impaired individuals, who often face challenges in accessing interactive and inclusive learning tools. Existing educational applications have yet to fully address the unique needs of these diverse user groups, particularly in the areas of real-time object recognition and speech synthesis. This paper presents LiveVision, an innovative solution designed to bridge these gaps. LiveVision integrates real-time object recognition with speech synthesis using Machine Learning to provide an engaging and accessible educational experience for children, while simultaneously offering a vital tool for enhancing the independence and learning of visually impaired individuals. By leveraging cutting-edge technologies, LiveVision empowers children with interactive learning opportunities, fosters a more inclusive environment, and promotes equal access to education for those with visual impairments. This research explores the design, development, and potential impact of LiveVision, aiming to break barriers, enhance learning outcomes, and create an inclusive educational landscape for all
BibTeX
@article{shetty2025livevision,
  author = {Ms. Likita S. Shetty and Mayur Naik and Mr. Manjuram G. Prabhudessai},
  title = {{LiveVision: Real Time Object Recognition and Speech Synthesis using Machine Learning}},
  journal = {International Journal of Advanced Research in Science Communication and Technology},
  year = {2025},
  doi = {10.48175/ijarsct-24460},
}
Figure