CloudCV: Large Scale Distributed Computer Vision as a Cloud Service

Harsh Agrawal, Clint Solomon Mathialagan, Yash Goyal, Neelima Chavali, Prakriti Banik, Akrit Mohapatra, Ahmed Osman, Dhruv BatraarXiv 2015 · arXiv

Introduction

We are currently in an era of "visual data proliferation," with billions of images uploaded to platforms like Facebook and Flickr daily. However, a significant gap exists between the availability of this data and the ability of most researchers to analyze it at scale. Scaling algorithms to millions of images requires building and maintaining clusters, designing multi-threaded primitives, and managing complex caching—tasks that are often outside a computer vision researcher's core expertise.

CloudCV is a comprehensive cloud-based platform designed to democratize access to state-of-the-art distributed computer vision algorithms. By providing a robust back-end infrastructure accessible via web interfaces and APIs, it allows researchers and developers to scale complex vision tasks to massive datasets without needing expertise in distributed computing or big data infrastructure.

High-level overview of the CloudCV ecosystem connecting back-end frameworks to front-end users.

System Architecture and Methods

CloudCV addresses the challenges of scalability and parallelization through a sophisticated, multi-layered cloud architecture hosted on Amazon Web Services (AWS). The system utilizes a "share-nothing" architecture to ensure modularity and reliability.

Back-end Infrastructure

  • Web Layer: The system employs Django for handling web requests and Node.js with Socket.IO to maintain persistent, two-way communication, enabling real-time progress updates for the user.
  • Task Distribution: Celery acts as an asynchronous task queue, distributing jobs across a heterogeneous mix of CPU and GPU worker nodes. Redis serves as a high-performance in-memory message broker to pipeline results back to the user.
  • Distributed Frameworks: For tasks with complex dependencies, CloudCV utilizes GraphLab, a high-level abstraction for "graph-parallel" computing. This is specifically used for their gigapixel image stitching pipeline.
  • Deep Learning: The system integrates Caffe at the back-end to provide high-performance image classification and feature extraction using Convolutional Neural Networks (CNNs).
Detailed CloudCV system architecture illustrating the flow from web servers through the job scheduler to distributed compute workers.

Key Insights and Innovations

The core philosophy of CloudCV is the democratization of AI. By moving the infrastructure burden from the user to the provider, the platform lowers the barrier to entry for biologists, astronomers, and hobbyists.

One of the most innovative features is Instantaneous Transfer Learning. Instead of full backpropagation, which is computationally expensive and slow, the "Train a New Category" module uses Linear Discriminant Analysis (LDA) on top of fixed CNN features (DeCAF). This allows users to add new categories to a model almost instantaneously. Additionally, by framing image stitching as a "message-passing" operation on a data graph, the platform utilizes GraphLab to handle global refinement tasks that are traditionally difficult to parallelize.

Functional Cloud Services and Results

The platform demonstrates its utility through several functional services:

  • Image Classification: Provides top-5 object categories with confidence scores using the CaffeNet architecture.
  • Feature Extraction: Allows users to extract 4096-dimensional DeCAF features, which serve as generic descriptors for various vision tasks.
  • VIP (Finding Important People): A unique service that predicts the relative importance of individuals in group photographs based on visual cues.
  • Gigapixel Stitching: Successfully creates composite panoramas from large collections of images by distributing feature extraction and matching across multiple nodes.

Conclusion and Future Work

CloudCV successfully bridges the gap between complex distributed computing and computer vision research. Future iterations aim to integrate more closely with NVIDIA DIGITS for custom CNN design, expand data source support to include Amazon S3, and implement private user workspaces for job history and custom model management. Plans are also underway to integrate with Scikit-learn, allowing users to train more diverse classifiers on top of extracted cloud features.

Summary updated

Abstract

We are witnessing a proliferation of massive visual data. Unfortunately scaling existing computer vision algorithms to large datasets leaves researchers repeatedly solving the same algorithmic, logistical, and infrastructural problems. Our goal is to democratize computer vision; one should not have to be a computer vision, big data and distributed computing expert to have access to state-of-the-art distributed computer vision algorithms. We present CloudCV, a comprehensive system to provide access to state-of-the-art distributed computer vision algorithms as a cloud service through a Web Interface and APIs.
BibTeX
@misc{agrawal2015cloudcv,
  author = {Harsh Agrawal and Clint Solomon Mathialagan and Yash Goyal and Neelima Chavali and Prakriti Banik and Akrit Mohapatra and Ahmed Osman and Dhruv Batra},
  title = {{CloudCV: Large Scale Distributed Computer Vision as a Cloud Service}},
  year = {2015},
  eprint = {1506.04130},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1506.04130},
}
Figure