On mining sensor network software repositories

Andreas Loukas, Matthias Woehrle, Koen LangendoenSESENA@ICSE 2011 · DOI

TL;DR

This paper dives into the software repositories of TinyOS and Contiki, the two dominant operating systems for Wireless Sensor Networks (WSNs). By analyzing commit histories, they uncover insights into development practices, community structure, and importantly, identify "cross-cutting concerns" – pieces of code that are hard to isolate and often related to resource constraints in WSNs.

What Problem Does This Paper Address?

Developing software for Wireless Sensor Networks (WSNs) presents unique challenges. These tiny, resource-constrained devices (limited power, memory, and processing) require specialized operating systems. TinyOS and Contiki are the leading open-source platforms for WSN development, offering frameworks for building applications. However, understanding how these complex software systems evolve, who contributes to them, and how their inherent constraints manifest in the code is crucial for future development. This paper aims to shed light on these aspects by mining the software repositories of TinyOS and Contiki, and comparing them to a more general embedded system, Ethernut. A key focus is on identifying "cross-cutting concerns" (CCCs) – code that spans multiple modules and is difficult to separate, often arising from the strict resource limitations of WSNs.

Methods

The researchers employed a multi-pronged approach:

  • Repository Mining: They extracted and analyzed commit histories from the TinyOS and Contiki repositories. This involved tracking file changes, authors, and commit messages. They also used a heuristic to detect file renames, which are often obscured in version control systems like CVS.
  • Comparative Analysis: They compared the development characteristics (number of files, lines of code, developers, transactions) of TinyOS and Contiki with Ethernut, a more general embedded operating system, to understand WSN-specific trends.
  • Cross-Cutting Concern (CCC) Identification: For TinyOS, they specifically focused on identifying CCCs. Their method involved mining for "frequent filesets" (sets of files frequently committed together) and then filtering these to identify potential CCCs. This process included:
  • Building a transaction database.
  • Computing "closed frequent filesets" (CFFs) using an algorithm.
  • Filtering out irrelevant commits and obvious dependencies (e.g., files within the same directory).
  • Using mutual information to identify probable filesets.
  • Grouping filesets into "grouped frequent filesets" (GFFs) to identify related concerns.

Datasets

The primary datasets used were the software repositories of:

  • TinyOS 2: The main focus for cross-cutting concern analysis.
  • Contiki: Used for comparative analysis of WSN operating systems.
  • Ethernut: Used as a baseline for comparison with a general embedded operating system.

The paper also references bug tracking data from OpenEmbedded as a point of comparison for bug tracking activity.

Key Insights and Ideas

  • Power Law Distribution in Commits: Like many open-source projects, the commit activity in TinyOS and Contiki follows a power-law distribution. This means a few files are heavily modified, while many others see infrequent changes.
  • Core Developer Groups: The study confirms the hypothesis that open-source projects have a core group of developers responsible for the majority of code changes. TinyOS had the largest core group among the WSN projects studied.
  • WSN-Specific Hotspots: In TinyOS, the most frequently committed files are related to data collection stacks (using CC2420 radios) and support for various radio hardware. Contiki shows a similar focus on data collection but also a strong emphasis on IP/6lowpan and MAC protocols.
  • Radio Drivers are Tricky: The high commit activity in radio device drivers for both TinyOS and Contiki suggests that implementing reliable wireless communication is a significant challenge.
  • Resource Constraints Drive CCCs: The paper strongly supports the idea that resource constraints in WSNs are a primary driver of cross-cutting concerns. These concerns often manifest as dependencies related to memory management, arbitration, and power management.
  • NesC and TinyOS Conventions: Many identified CCCs in TinyOS are not necessarily due to poor design but are artifacts of the nesC language and TinyOS's programming conventions, which prioritize optimization over strict modularity.

Key Results

  • Repository Characteristics: Table 1 provides a quantitative overview of the repositories, showing that TinyOS and Contiki have histories of at least 5 years, with over 1000 files and similar commit characteristics.
  • Developer Contribution: For TinyOS, 14 developers were responsible for 80% of code changes, compared to 6 for Contiki and 3 for Ethernut, confirming the core developer group hypothesis.
  • CCCs in TinyOS: After filtering for a specific platform (TelosB), the analysis identified 25 concerns, with a significant portion falling into the "resource" class (memory, arbitration, power management).
  • Example CCC: A notable example of a resource CCC is message handling, where dependencies exist between interfaces, radio/serial stacks, and network protocols to optimize buffer usage. Resource arbitration also showed widespread propagation of changes across chip-specific components.

Limitations and Future Work

  • Obvious Dependencies: The authors acknowledge that their mining process can identify many "obvious" dependencies (e.g., files within the same directory or tightly coupled module/configuration pairs). Future work aims to refine the filtering mechanisms to better distinguish these from true CCCs.
  • Semantic vs. Syntactic Relations: Distinguishing between purely syntactic relations (like file naming conventions) and semantic relations (actual code dependencies) remains a challenge.
  • Manual Filtering: Some manual filtering and interpretation were necessary, particularly for grouping filesets into concerns. Automating this process further is a direction for future research.
  • Refining CCC Mining: The authors plan to refine their method to more easily filter out obvious relations and improve the fidelity of CCC identification in WSN software. They also suggest exploring other mining approaches beyond file-level analysis.

Summary updated

BibTeX
@inproceedings{loukas2011mining,
  author = {Andreas Loukas and Matthias Woehrle and Koen Langendoen},
  title = {{On mining sensor network software repositories}},
  booktitle = {SESENA@ICSE},
  year = {2011},
  doi = {10.1145/1988051.1988057},
}
Figure