TL;DR
This paper explores the burgeoning field of software development leveraging "collective intelligence" amplified by the internet. It argues that the internet's ability to connect vast numbers of people and facilitate rapid information exchange offers a novel solution to the inherent complexities and challenges of software engineering, moving beyond traditional approaches.
What Problem Does This Paper Address?
The authors tackle the enduring challenge of software development's inherent difficulties, famously articulated by Frederick Brooks' "no silver bullet" thesis. These difficulties include complexity, conformity, changeability, and invisibility. The paper posits that while traditional methods struggle with the ever-increasing scale and dynamism of software, particularly in the internet era, a new paradigm – collective intelligence – can offer a path forward. The internet, by enabling "internet-of-intelligences," provides the infrastructure for this collective intelligence to emerge and address these fundamental problems.
Methods
The paper doesn't present a novel algorithm or a specific technical method in the traditional sense. Instead, it offers a conceptual framework and analysis. The authors review existing practices like open-source software development, software crowdsourcing, and application stores as "primitive forms" of internet-based collective intelligence in software engineering. They analyze the feasibility, current state, and challenges of this approach by examining the evolution of human organizational structures and the inherent properties of software as a model-driven artifact. They also propose an "ideal form" of internet collective intelligence for software development, characterized by a continuous, incremental, and iterative process involving free exploration, automatic fusion, and active feedback.
Datasets
Not discussed in the paper.
Key Insights and Ideas
- The Internet as an "Internet of Intelligences": Beyond connecting computers and things, the internet is evolving to connect human intelligence, creating a powerful new resource for problem-solving.
- Revisiting "No Silver Bullet": While Brooks' assertion about inherent software development difficulties remains valid, the context has changed dramatically with the internet, potentially offering a "silver bullet" in the form of collective intelligence.
- Organizational Evolution: The paper draws parallels between the evolution of human societies and businesses (from centralized to decentralized structures) and the shift in software development organization, particularly towards more decentralized models facilitated by the internet.
- Software's Malleability: Software's abstract, model-driven nature makes it uniquely suited for collective intelligence approaches, allowing for decomposition and recomposition of tasks.
- Primitive Forms of Collective Intelligence: Open-source, crowdsourcing, and app stores are not just development models but early manifestations of how collective intelligence can be harnessed for software.
- The Ideal Form: The authors envision a future where automated fusion and active feedback mechanisms, powered by algorithms, augment human creativity in a continuous, iterative development loop.
Key Results
The paper doesn't present empirical results from experiments. Instead, its "results" are analytical and conceptual:
- Feasibility: The authors argue that internet-based collective intelligence is theoretically feasible due to the nature of software and the evolving organizational structures enabled by the internet.
- State-of-the-Practice: They identify open-source, crowdsourcing, and app stores as existing, albeit primitive, examples of this paradigm, demonstrating its practical application.
- Challenges: The paper highlights significant challenges, including the lack of systematic platforms, the difficulty in understanding the underlying mechanisms of collective intelligence, and the need for better metrics and control.
- Ideal Form: A conceptual framework for an ideal internet collective intelligence system for software development is proposed, emphasizing human-machine collaboration.
Limitations and Future Work
The authors acknowledge that internet-based collective intelligence for software development is still in its "inevitable kingdom" phase, far from a mature, systematic methodology. Key limitations and areas for future work include:
- Understanding Drivers: Further research is needed to understand the psychological and sociological drivers of human collective intelligence formation in the internet age.
- Formation Mechanisms: A unified framework for understanding how collective intelligence emerges is lacking.
- Decomposition Mechanisms: Developing effective methods for decomposing complex software problems for large, distributed, and uncertain developer groups is crucial.
- Measurement: Establishing metrics for measuring the efficiency, quality, and individual contributions within collective intelligence systems is essential.
- Automation: The ideal form requires significant advancements in automated information fusion and active feedback mechanisms, moving beyond current manual or semi-automated approaches.
- Scalability: Adapting current models to handle large-scale, complex software systems remains a significant challenge.
Summary updated