<i>CLCMiner</i>: Detecting Cross-Language Clones without Intermediates

Xiao Cheng, Zhiming Peng, Lingxiao Jiang, Hao Zhong, Haibo Yu, Jianjun Zhao

Abstract

The proliferation of diverse programming languages and platforms makes it a common need to have the same functionality implemented in different languages for different platforms, such as Java for Android applications and C# for Windows phone applications. Although versions of code written in different languages appear syntactically quite different from each other, they are intended to implement the same software and typically contain many code snippets that implement similar functionalities, which we call cross-language clones. When the version of code in one language evolves according to changing functionality requirements and/or bug fixes, its cross-language clones may also need be changed to maintain consistent implementations for the same functionality. Thus, it is needed to have automated ways to locate and track cross-language clones within the evolving software. In the literature, approaches for detecting cross-language clones are only for languages that share a common intermediate language (such as the .NET language family) because they are built on techniques for detecting single-language clones. To extend the capability of cross-language clone detection to more diverse kinds of languages, we propose a novel automated approach, CLCMiner, without the need of an intermediate language. It mines such clones from revision histories, based on our assumption that revisions to different versions of code implemented in different languages may naturally reflect how programmers change cross-language clones in practice, and that similarities among the revisions (referred to as clones in diffs or diff clones) may indicate actual similar code. We have implemented a prototype and applied it to ten open source projects implementations in both Java and C#. The reported clones that occur in revision histories are of high precisions (89% on average) and recalls (95% on average). Compared with token-based code clone detection tools that can treat code as plain texts, our tool can detect significantly more cross-language clones. All the evaluation results demonstrate the feasibility of revision-history based techniques for detecting cross-language clones without intermediates and point to promising future work.

Introduction

In modern software development, it is common to implement the same functionality across different platforms (e.g., an Android app in Java and an iOS app in Swift). These are "cross-language clones." When a bug is fixed or a feature is updated in one language, developers must manually find and update the corresponding code in other languages to maintain consistency. Traditional clone detection tools usually rely on syntactic analysis or Intermediate Languages (IL), which limits their use to specific language families (like the .NET family). CLCMiner addresses the gap of detecting clones across languages that do not share an intermediate representation.

Methodology

The core innovation of CLCMiner is that it analyzes revision histories (git logs) rather than static source code. The underlying assumption is that when a developer implements a change in a multi-language project, they likely update the same logic in both versions using similar identifier names and logic flows.

The CLCMiner pipeline: from log parsing to clone reporting.

As illustrated in the pipeline above, the process involves four main steps:

  1. Log Parsing: Extracting "diffs" (the specific lines changed in a commit) and their metadata.
  2. Normalizing: Cleaning the code by removing comments and splitting CamelCase or snake_case identifiers into individual tokens to normalize naming conventions across languages.
  3. Diff Matching: Using a "Bag of Words" (BOW) approach to calculate the distance between token streams. A sliding window approach is used to compare shorter diffs against subsequences of longer diffs to find the best match.
  4. Ranking & Reporting: Pairs of diffs are ranked by similarity and reported as clones if they fall below a specific distance threshold.

Key Insights: Lexical Resilience

Full source files are often noisy, but diffs provide a "distilled" version of a developer's intent. By focusing on how code changes, CLCMiner bypasses much of the syntactic clutter that makes cross-language comparison difficult. Even if syntax changes significantly—for example, Java using a method call while C# uses a direct property assignment—the core identifiers (the "nouns" of the code) usually remain the same.

Comparison of code diffs in Java and C# showing similar logic and identifier names despite syntactic differences.

Evaluation and Results

The researchers evaluated CLCMiner on 10 open-source projects implemented in both Java and C#, including Antlr3, Lucene, and Spring. The dataset included over 3.6 million lines of code and thousands of commits.

  • High Accuracy: CLCMiner achieved an average precision of 89.1% and a recall of 95.0%.
  • Sliding Window Success: The sliding window approach significantly improved performance by allowing the tool to find small logic snippets embedded within larger changes.
  • Superiority over Token-Based Tools: Compared to standard tools like CCFinder, CLCMiner detected significantly more clones because its normalization and history-based matching are specifically tuned for cross-language evolution. For instance, in the Lucene project, CLCMiner found over 1.1 million clone pairs, while CCFinder found only about 6,800.

Limitations and Future Work

Because CLCMiner relies on revision history, it can only detect clones that have been modified at least once in the git log. Code that was written once and never touched remains invisible to this method. Future directions include integrating deep learning for better vector representations of code and exploring "clone transitivity" to detect larger clone groups across even more diverse languages like Swift or Go.

Summary updated

Abstract

The proliferation of diverse kinds of programming languages and platforms makes it a common need to have the same functionality implemented in different languages for different platforms, such as Java for Android applications and C# for Windows phone applications. Although versions of code written in different languages appear syntactically quite different from each other, they are intended to implement the same software and typically contain many code snippets that implement similar functionalities, which we call cross-language clones. When the version of code in one language evolves according to changing functionality requirements and/or bug fixes, its cross-language clones may also need be changed to maintain consistent implementations for the same functionality. Thus, it is needed to have automated ways to locate and track cross-language clones within the evolving software. In the literature, approaches for detecting cross-language clones are only for languages that share a common intermediate language (such as the .NET language family) because they are built on techniques for detecting single-language clones. To extend the capability of cross-language clone detection to more diverse kinds of languages, we propose a novel automated approach, CLCMiner, without the need of an intermediate language. It mines such clones from revision histories, based on our assumption that revisions to different versions of code implemented in different languages may naturally reflect how programmers change cross-language clones in practice, and that similarities among the revisions (referred to as clones in diffs or diff clones) may indicate actual similar code. We have implemented a prototype and applied it to ten open source projects implementations in both Java and C#. The reported clones that occur in revision histories are of high precisions (89% on average) and recalls (95% on average). Compared with token-based code clone detection tools that can treat code as plain texts, our tool can detect significantly more cross-language clones. All the evaluation results demonstrate the feasibility of revision-history based techniques for detecting cross-language clones without intermediates and point to promising future work.
BibTeX
@inproceedings{cheng2017lt,
  author = {Xiao Cheng and Zhiming Peng and Lingxiao Jiang and Hao Zhong and Haibo Yu and Jianjun Zhao},
  title = {{\<i\>CLCMiner\</i\>: Detecting Cross-Language Clones without Intermediates}},
  booktitle = {IEICE},
  year = {2017},
  doi = {10.1587/transinf.2016edp7334},
}
Figure