TL;DR
This paper investigates what makes sponsored search ad landing pages "mobile friendly" and "aesthetically appealing," two crucial aspects of user experience that impact engagement and conversions. By analyzing over 120 features and collecting extensive human annotations, they found that while mobile friendliness is largely driven by technical and structural page elements, aesthetics are primarily influenced by visual characteristics, and these two dimensions are distinct yet related.
What Problem Does This Paper Address?
When users click on a sponsored search ad, they land on a dedicated page. The experience on this "ad landing page" is critical for user satisfaction and ultimately for the advertiser's success. However, many of these pages are not optimized for mobile devices, leading to a poor user experience. Furthermore, the visual appeal of these pages, or their "aesthetic appeal," is also a significant factor in user engagement. While existing tools can check for general mobile friendliness, they often lack transparency and aren't tailored to the specific context of advertising. This paper aims to bridge this gap by systematically characterizing what makes ad landing pages both mobile friendly and aesthetically pleasing, and how these two aspects relate to each other and to user behavior like conversions and dwell time.
Methods
The researchers employed a multi-faceted approach:
- Feature Engineering: They extracted a comprehensive set of over 120 features from ad landing pages, categorized into:
- HTML-based features: These capture structural elements, text content, and mobile optimization indicators (e.g., page size, readability scores, number of input fields, link structure).
- Image-based features: These leverage computational aesthetics techniques to analyze visual aspects like color palettes, textures, contrast, image quality, and layout.
- Machine Learning Models: They trained and evaluated four different classification models (Logistic Regression, Gradient Boosted Machines, Random Forest, and Gradient Boosted Regression Trees) to predict mobile friendliness and aesthetic appeal. Random Forest emerged as the best performing model.
- Annotation Collection: They designed specific guidelines for human annotators to rate ad landing pages on a 4-point scale for mobile friendliness and a binary scale for aesthetic appeal. This involved a rigorous annotation process with multiple judges and arbitration to ensure consistency.
Datasets
- Newly Introduced Benchmark: The paper introduces a new dataset of approximately 4,000 ad landing pages with human annotations for mobile friendliness and aesthetic appeal. This dataset is specifically tailored to the context of sponsored search advertising, differentiating it from datasets focused on general web pages.
- Evaluation Data: The collected annotations were used for both training and evaluating the machine learning models.
Key Insights and Ideas
- Distinct Dimensions: Mobile friendliness and aesthetic appeal are not the same thing. While they are correlated, they capture different aspects of the user experience. A page can be mobile friendly without being aesthetically pleasing, and vice versa.
- Visuals for Aesthetics, Structure for Friendliness: Aesthetic appeal is primarily driven by visual features (color, brightness, contrast), whereas mobile friendliness is more strongly influenced by HTML structure, page size, and technical optimization.
- The "Perfect" Paradox: While "perfect" mobile-friendly pages (those that are highly usable and facilitate conversion) are desirable, the data suggests they don't necessarily lead to more conversions. This might be due to the complexity of conversion funnels or the specific definition of "conversion" used.
- W3C Limitations: The W3C mobileOK checker, while useful, has limitations in predicting mobile friendliness for ad landing pages, especially for specific categories like "perfect" pages. This highlights the need for ad-specific evaluation.
- Aesthetics Drive Conversions: Aesthetically pleasing pages, on the other hand, are positively associated with conversions, suggesting that visual appeal plays a significant role in persuading users.
Key Results
- Mobile Friendliness Distribution: Only a small percentage (less than 10%) of ad landing pages were rated as "perfect" in terms of mobile friendliness, with most falling into "fair" or "good" categories. A significant portion (16%) provided a "bad" mobile experience.
- Feature Importance:
- For mobile friendliness, key features included image width, original HTML size, CSS count, and W3C error classes related to page size and stylesheets. JPEG blockiness (a measure of image compression noise) was also important, with cleaner images being more mobile-friendly.
- For aesthetic appeal, visual features like Pleasure, Dominance, Mean Brightness, and Brightness Contrast were most discriminative.
- Correlation with User Behavior:
- Mobile-friendly pages showed a positive correlation with long clicks (indicating higher dwell time), suggesting better user engagement.
- Aesthetically appealing pages were positively associated with conversions.
- Model Performance: The Random Forest model, using a combination of HTML and image-based features, achieved the best performance in predicting both mobile friendliness and aesthetic appeal.
Limitations and Future Work
- Conversion Data Limitations: The paper notes that conversion data was too sparse for robust statistical analysis in some cases.
- Subjectivity of Aesthetics: While efforts were made to standardize aesthetic judgments, it remains a somewhat subjective dimension.
- Future Directions: The authors suggest that their models can be used to filter out bad ad landing pages, potentially leading to improved click-through rates and reduced short clicks (indicating negative post-click experiences). They also emphasize the growing importance of visual features and aesthetic appeal in tools for assessing ad quality and user experience.
Summary updated