Introduction
This article provides a formal correction to the research paper "Data-driven prototyping via natural-language-based GUI retrieval," originally published in the journal Automated Software Engineering. This update is critical for researchers and engineers benchmarking their own models, as it rectifies a formatting error in the original publication that misaligned numerical performance data with their corresponding metrics.
The Problem: Natural Language GUI Retrieval
In modern software development, "prototyping by example" is a common practice. Designers often look for existing examples to adapt rather than building from scratch. However, searching through massive repositories of UI designs using only keywords is often ineffective due to the "vocabulary mismatch" problem—where a user describes a "profile page" but the metadata labels it an "account settings screen."
The original research addressed this by enabling natural language queries (e.g., "a login screen with a social media toggle") to retrieve relevant GUI designs. This correction ensures the quantitative evaluation of the models used for this task is accurately presented.
Methodology
The study benchmarked several Information Retrieval (IR) and Machine Learning (ML) techniques to determine which best handles the nuances of GUI descriptions:
- Traditional Baselines: Classic text-matching algorithms like TF-IDF, BM25, and nBOW (neural Bag-of-Words).
- Query Expansion: Variants of Pseudo-Relevance Feedback (+PRF), which refine searches by assuming top results are relevant.
- Transformer-based Models: Modern NLP approaches including Sentence-BERT (for semantic similarity) and BERT-LTR (Learning-to-Rank), which treats retrieval as a ranking problem.
Corrected Results and Analysis
The corrected data confirms that BERT-based Learning-to-Rank models significantly outperform traditional IR baselines. By explicitly training the model to order results by relevance, the LTR approach captures the semantic nuances of GUI components more effectively than simple similarity measures.
As shown in the table above, BERT-LTR (2) emerged as the strongest model across most metrics, achieving a Mean Reciprocal Rank (MRR) of 0.631 and an Average Precision (AveP) of 0.501. Notably, all BERT-LTR variants achieved a perfect HITS@15 score of 1.00, meaning the target GUI was successfully found within the top 15 results in every test case.
Precision at the top of the results list is critical for a useful developer tool. The corrected P@k and NDCG@k metrics (Table 2) demonstrate that BERT-LTR (2) excels at Precision@3 (0.400). This indicates that the model is highly effective at surfacing the correct UI design within the first three suggestions, providing a significant improvement in user experience over traditional BM25 baselines (MRR 0.520).
Conclusion
This correction restores the scientific accuracy of the performance benchmarks for natural-language-based GUI retrieval. The results highlight the clear superiority of Learning-to-Rank approaches over traditional keyword matching and standard semantic similarity, providing a reliable foundation for future research in automated prototyping tools.
Summary updated