Abstract
With the rapid proliferation of multimedia data on the internet, there has been a fast rise in the creation of videos for viewers. This enables viewers to skip advertisement breaks using ad blockers and 'skip ad' buttons, bringing online marketing and publicity to a stall. In this paper, we demonstrate a system that can effectively integrate a new advertisement into a video sequence. We use state-of-the-art techniques from deep learning and computational photogrammetry for the effective detection of existing adverts and the seamless integration of new adverts into video sequences. This is helpful for targeted advertisement, paving the path for next-gen publicity.
Introduction
The traditional digital advertising model is under siege. As ad-blockers and "Skip Ad" buttons become ubiquitous, viewers are increasingly bypassing commercial breaks, leading to a decline in marketing effectiveness. To address this, we propose "Next-Gen Publicity": a system that integrates advertisements directly into video content. By dynamically swapping static billboards in a scene for targeted products, we can create native ads that are both unskippable and contextually relevant.
Methodology
Our system utilizes a modular pipeline to transform raw video into a composite result with injected advertisements. The process involves three primary stages:
1. Advert Recognition
To identify frames containing candidate billboards, we employ a VGG-based Deep Neural Network (DNN). Using transfer learning, we fine-tuned a model pre-trained on ImageNet to act as a binary classifier, distinguishing between "billboard" and "no billboard" scenes.
2. Advert Localization
Once a billboard is detected, the system determines its precise boundaries using an encoder-decoder architecture. This model generates a pixel-wise probability map, or heatmap, which identifies the likely location of the advert. By thresholding this map and identifying the largest closed contour, the system extracts the four corners of the billboard.
3. Advert Integration and Tracking
To ensure visual fidelity, we use Poisson image editing to blend the new advertisement into the scene. This technique matches local illumination and color tones, preventing the new ad from appearing unnaturally sharp or bright. To maintain temporal consistency as the camera moves, we utilize the Kanade-Lucas-Tomasi (KLT) tracking algorithm to keep the ad pinned to the billboard across frames.
Key Results
The primary output of this research is a functional end-to-end prototype. Our qualitative results demonstrate that the system can successfully replace existing advertisements while maintaining the perspective and ambient lighting of the original environment. The combination of deep learning for localization and classical computer vision for blending results in a seamless integration that feels like a natural part of the scene.
Conclusion and Future Work
This system represents a significant step toward automated, targeted native advertising in video. While the current prototype excels in outdoor scenes with clear billboards, future work will focus on generalizing detection to diverse surfaces and handling complex occlusions where objects pass in front of the advertisement. By refining these modules, we aim to reduce the need for manual user adjustment and provide a fully autonomous solution for next-generation publicity.
Summary updated