An Advert Creation System for Next-Gen Publicity

Atul Nautiyal, Killian McCabe, Murhaf Hossari, Soumyabrata Dev, Matthew Nicholson, Clare Conran, Declan McKibben, Jian Tang, Wei Xu, François Pitié

Abstract

With the rapid proliferation of multimedia data on the internet, there has been a fast rise in the creation of videos for viewers. This enables viewers to skip advertisement breaks using ad blockers and 'skip ad' buttons, bringing online marketing and publicity to a stall. In this paper, we demonstrate a system that can effectively integrate a new advertisement into a video sequence. We use state-of-the-art techniques from deep learning and computational photogrammetry for the effective detection of existing adverts and the seamless integration of new adverts into video sequences. This is helpful for targeted advertisement, paving the path for next-gen publicity.

Introduction

The traditional digital advertising model is under siege. As ad-blockers and "Skip Ad" buttons become ubiquitous, viewers are increasingly bypassing commercial breaks, leading to a decline in marketing effectiveness. To address this, we propose "Next-Gen Publicity": a system that integrates advertisements directly into video content. By dynamically swapping static billboards in a scene for targeted products, we can create native ads that are both unskippable and contextually relevant.

Methodology

Our system utilizes a modular pipeline to transform raw video into a composite result with injected advertisements. The process involves three primary stages:

1. Advert Recognition

To identify frames containing candidate billboards, we employ a VGG-based Deep Neural Network (DNN). Using transfer learning, we fine-tuned a model pre-trained on ImageNet to act as a binary classifier, distinguishing between "billboard" and "no billboard" scenes.

2. Advert Localization

Once a billboard is detected, the system determines its precise boundaries using an encoder-decoder architecture. This model generates a pixel-wise probability map, or heatmap, which identifies the likely location of the advert. By thresholding this map and identifying the largest closed contour, the system extracts the four corners of the billboard.

Figure 5: A probabilistic heatmap generated by the encoder-decoder architecture to localize the billboard within a frame.

3. Advert Integration and Tracking

To ensure visual fidelity, we use Poisson image editing to blend the new advertisement into the scene. This technique matches local illumination and color tones, preventing the new ad from appearing unnaturally sharp or bright. To maintain temporal consistency as the camera moves, we utilize the Kanade-Lucas-Tomasi (KLT) tracking algorithm to keep the ad pinned to the billboard across frames.

Key Results

The primary output of this research is a functional end-to-end prototype. Our qualitative results demonstrate that the system can successfully replace existing advertisements while maintaining the perspective and ambient lighting of the original environment. The combination of deep learning for localization and classical computer vision for blending results in a seamless integration that feels like a natural part of the scene.

Figure 1: Side-by-side comparison showing the original billboard (left) and the system's seamless replacement (right).

Conclusion and Future Work

This system represents a significant step toward automated, targeted native advertising in video. While the current prototype excels in outdoor scenes with clear billboards, future work will focus on generalizing detection to diverse surfaces and handling complex occlusions where objects pass in front of the advertisement. By refining these modules, we aim to reduce the need for manual user adjustment and provide a fully autonomous solution for next-generation publicity.

Summary updated

Abstract

With the rapid proliferation of multimedia data in the internet, there has been a fast rise in the creation of videos for the viewers. This enables the viewers to skip the advertisement breaks in the videos, using ad blockers and 'skip ad' buttons -- bringing online marketing and publicity to a stall. In this paper, we demonstrate a system that can effectively integrate a new advertisement into a video sequence. We use state-of-the-art techniques from deep learning and computational photogrammetry, for effective detection of existing adverts, and seamless integration of new adverts into video sequences. This is helpful for targeted advertisement, paving the path for next-gen publicity.
BibTeX
@misc{nautiyal2018advert,
  author = {Atul Nautiyal and Killian McCabe and Murhaf Hossari and Soumyabrata Dev and Matthew Nicholson and Clare Conran and Declan McKibben and Jian Tang and Wei Xu and François Pitié},
  title = {{An Advert Creation System for Next-Gen Publicity}},
  year = {2018},
  eprint = {1808.00163},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1808.00163},
}
Figure