Abstract
Competitive analysis is a critical part of any business. Product managers, sellers, and marketers spend time and resources scouring through a huge volume of online and offline content, aiming to discover what their competitors are doing in the marketplace and to understand what type of threat they pose to their business' financial well-being. Currently, this process is slow, costly and labor-intensive. We demonstrate Clarity, a data-driven unsupervised system for assessment of products, which is currently in deployment at IBM. Clarity has been running for more than a year and is used by over 1,500 people to perform over 160 competitive analyses involving over 800 products. The system considers multiple factors from a collection of online content: numeric ratings by users, sentiment towards key product drivers, content volume, and recency of content. The results and explanations of factors leading to the results are visualized in an interactive dashboard that allows users to track the performance of their products as well as understand the main contributing factors.
Introduction
In today's fast-paced market, understanding a product's competitive standing is essential for strategic decision-making. Traditionally, this has been a manual, labor-intensive process where researchers manually track product drivers and annotate user feedback. However, manual analysis is not only expensive and slow but also prone to human error and rapid obsolescence as consumer sentiment shifts.
To address these challenges, we developed Clarity, an unsupervised, data-driven system designed to automate the evaluation of product competitiveness. By leveraging Natural Language Processing (NLP) on public user-generated content, Clarity provides a continuous, real-time view of the market landscape.
The Clarity Methodology
Clarity calculates a "Clarity Score" to represent perceived competitiveness through an end-to-end pipeline:
- Data Ingestion: The system collects unstructured text from various online forums and review sites. It utilizes a pre-trained Word2Vec model to create numerical representations of the vocabulary, alongside tracking metadata like review volume and star ratings.
- Sentiment and Driver Analysis: Using the Watson Natural Language Understanding API, Clarity extracts keywords and calculates targeted sentiment. These keywords are then mapped to specific product "drivers" (such as usability or price) using cosine distance on Word2Vec embeddings.
- Normalization and Weighting: To ensure comparability across diverse data sources, raw metrics are converted into percentile scores. Furthermore, a time-decay weighting is applied to 3-month rolling windows over an 18-month period, prioritizing recent data to reflect the current product lifecycle.
Visualization and User Interface
A core innovation of Clarity is its interactive dashboard, which translates complex data into actionable insights. The primary visualization tool is the "sorted stream graph."
As shown in Figure 1, the dashboard displays high-level competitive scores alongside a temporal view of market positioning. In the sorted stream graph, the x-axis represents time, while the y-axis represents a product's rank. The thickness of each stream corresponds to the absolute competitiveness score. This dual-layered approach allows users to distinguish between a drop in rank caused by internal product decline (thinning stream) versus a drop caused by a competitor's rapid improvement (stable stream thickness, but downward vertical shift).
Deployment and Impact
Clarity has seen significant enterprise adoption within IBM. Over its first year of deployment, it supported more than 1,500 users in conducting 160+ competitive analyses across 800 products.
Validation of the system's unsupervised rankings against expert human evaluations—such as Gartner Magic Quadrants and Net Promoter Scores (NPS)—demonstrated strong alignment. This confirms that the automated pipeline can reliably replicate high-level market insights at a fraction of the traditional cost and time. Furthermore, the system's scoring engine has been integrated via APIs into other business functions, including pricing recommendation and talent management.
Conclusion
Clarity represents a shift from manual to automated competitive intelligence. By combining NLP-driven sentiment analysis with innovative visualizations like the sorted stream graph, the system provides a scalable and objective framework for product assessment. Future work will focus on enhancing the system's robustness against noise in public data, such as fake reviews or sarcasm, to further refine its predictive accuracy.
Summary updated