Introduction
Citizen science represents a transformative approach to large-scale data collection, leveraging public participation to advance scientific research. However, the effectiveness of these initiatives is often hampered by two primary challenges: data quality and technical barriers. Non-expert participants frequently struggle to accurately identify complex subjects in the field, and the cost of developing custom mobile applications with integrated machine learning (ML) guidance is often prohibitive, requiring specialized programming expertise and significant financial investment.
SmartCS addresses these hurdles by providing an open-source, no-code platform that empowers researchers and students to build mobile citizen science apps with integrated, offline-capable ML. By focusing on client-side inference, SmartCS ensures that high-quality computer vision tools remain functional in remote areas where internet connectivity is unavailable.
The SmartCS Workflow
The platform streamlines the development process through a three-step automated workflow designed for users with no prior coding experience.
1. Dataset Creation
The workflow begins with data acquisition and preparation. SmartCS provides integrated tools for extracting individual frames from video data and facilitates integration with open-source labeling software such as AnyLabeling or LabelImg. The platform automatically handles the formatting and splitting of data into training, testing, and evaluation sets, ensuring a robust foundation for model development.
2. ML Model Training
To optimize performance on mobile hardware, SmartCS utilizes transfer learning on lightweight, mobile-optimized models from the TensorFlow Lite library. The platform supports "one-stage" detectors like SSD and YOLO, which prioritize inference speed—a critical factor for real-time mobile applications. Users have the flexibility to train models locally or utilize cloud-based resources like Google Colab to access free GPU acceleration.
3. Mobile App Building
The final stage involves the assembly of the mobile application. SmartCS offers GUI-based templates for both iOS and Android, allowing users to customize visual elements such as colors and logos. The trained ML model is integrated into the app without any manual coding. Furthermore, the system incorporates Fastlane to automate the deployment process to digital distribution platforms.
Applications and Case Studies
The versatility of SmartCS has been demonstrated through several specialized applications:
- RipSnap: Designed to identify rip currents using a dataset of 3,360 labeled images sourced from drone footage and beach webcams.
- Recycle This: An app for identifying household recyclables, trained on 2,500 images from various public datasets.
- TidalNow: Focused on marine biology, this app identifies five different tide pool species using a dataset of approximately 600 images per class.
Key Insights: Democratizing Development
A significant finding of this research is the platform's ability to democratize app development. In pilot studies, high school students with no prior programming experience were able to design and build functional, research-grade ML applications within just one to two weeks.
Beyond simple data collection, the platform utilizes ML as an educational tool. Real-time bounding boxes provide immediate feedback to users, helping them learn to recognize specific subjects (e.g., a particular shell or a rip current) independently over time. For projects starting without existing data, the "Expert Mode" allows for initial raw data collection by volunteers, which is then curated by experts to train subsequent iterations of the ML model.
Results and Performance
Benchmarking of various model architectures revealed that YOLOv8m provided the highest precision (50.2 mAP) with an inference speed of 32ms. However, EfficientDet D2 was noted for offering the most stable performance across a wide range of modern smartphone hardware.
User satisfaction surveys indicated high levels of approval. App creators (high school students) specifically praised the platform for being free and requiring no programming skills, both scoring approximately 4.5/5 on a Likert scale. Beta testers of the resulting apps also reported high levels of satisfaction with the ML capabilities and overall utility of the tools.
Limitations and Future Directions
While successful, the current iteration of SmartCS has areas for improvement. User feedback suggested that the UI templates are currently too simplistic; future updates will include more diverse and aesthetically refined designs. Additionally, the authors plan to implement a "human-in-the-loop" feedback system, allowing users to correct ML errors in real-time. These corrections will then be used to automatically retrain and improve the models, creating a seamless cycle of refinement. As mobile hardware continues to evolve, the platform will also incorporate more advanced ML architectures into its no-code pipeline.
Summary updated