CreativeBioMan: A Brain- and Body-Wearable, Computing-Based, Creative Gaming System

Min Chen, Yingying Jiang, Yong Cao, Albert Y. Zomaya

Introduction

CreativeBioMan represents a paradigm shift in digital art creation, moving from manual tools to a "human-machine coalition." This novel wearable computing system translates a user's neurophysiological signals and emotional states into personalized digital artwork. By integrating electroencephalogram (EEG) data with emotion-aware neural style transfer, the system allows AI to handle the technical labor of rendering while the human user provides the creative spark through biological signals.

The Challenge of Aesthetic Creativity

While artificial intelligence has achieved mastery in logical domains such as data analysis and computation, "aesthetic creativity" remains a complex frontier. Traditional AI art generation often relies on explicit text prompts or manual style selection, which can feel disconnected from the artist's internal state. CreativeBioMan addresses this by positing that true artistic creation is a synthesis of an artist’s history, current emotions, and subconscious intent. The system aims to capture these intangible elements using wearable sensors, effectively "reading" the artist's mind to automate repetitive tasks while preserving their unique creative identity.

System Architecture and Methodology

The CreativeBioMan architecture is a multi-layered pipeline involving specialized hardware, cloud computing, and a suite of neural networks.

Data Collection and Style Classification

Users are equipped with an EEG headset (utilizing the ADS1299-8 chip) to monitor brain oscillations and wearable clothing integrated with an AIWAC BOX emotional robot. This robot collects multimodal data, including voice and imagery, to recognize up to 21 distinct human emotions.

To process EEG data, the system employs Common Spatial Pattern (CSP) algorithms to identify patterns related to "motor imagery"—the mental rehearsal of the creative process. These patterns are then classified into four primary artistic styles (oil painting, traditional Chinese painting, sketch, or cartoon) using a Long Short-Term Memory (LSTM) network.

Technical Pipeline

The technical pipeline of CreativeBioMan, illustrating the flow from input data through the VGG-19 and RNN modules to the final artwork. CreativeBioMan: Brain and Body Wearable Computing based Creative Gaming System, page 4

The technical pipeline of CreativeBioMan, illustrating the flow from input data through the VGG-19 and RNN modules to the final artwork.

As shown in the technical pipeline, the system utilizes a VGG-19 network as its core "painter." This network performs neural style transfer by merging a content image (such as a real-time scenery photo or a tactile sketch) with the style identified from the user's EEG and historical artwork database. Simultaneously, an attention-based Recurrent Neural Network (RNN) processes multimodal emotional data. The attention mechanism allows the model to prioritize signals with the highest emotional intensity, which are then used to "rectify" the final output—for instance, by shifting the color palette to warmer hues to reflect a state of happiness.

Artistic Synthesis and Results

The researchers validated the system using a combination of historical artwork datasets and real-time biological signals from test subjects. The primary goal was to ensure that the generated art felt authentic to the user's intent.

Visual examples of the system's output, showing the transformation of content images into various styles based on user data.

Key Performance Metrics

  • Creative Fidelity: In a blind "verisimilitude" test, judges compared AI-generated works to human-created art. The system achieved a life-like score higher than 50% for the majority of subjects, with top scores reaching 85%, indicating a high degree of success in mimicking personal artistic styles.
  • System Latency: The system is optimized for real-time interaction. Data transmission to the cloud occurs within 0.6–0.8 seconds, and model inference takes less than 50ms, providing a responsive and immersive creative experience.

Conclusion and Future Directions

CreativeBioMan introduces the "BioMan" concept: a humanoid biological robot that acts as a partner rather than a tool, mimicking human cognitive processes to emancipate artists from repetitive labor. While currently limited to four broad styles, future work will focus on "brain awareness," aiming for more granular readings of a user's visual imagination and a more seamless human-cloud integration. This research paves the way for a future where biological signals and AI work in tandem to expand the boundaries of human expression.

Summary updated

Abstract

Current artificial intelligence (AI) technology is used mainly in rational work, i.e., computational and logical analysis. How to best make the machine as aesthetic and creative as humans has gradually gained attention. This article presents a unique, creative game system called CreativeBioMan. This system combines brain wave and multimodal emotion data and uses an AI algorithm for intelligent decision fusion, which can then be used in artistic creation with the goal being to separate the artist from repeated labor creation.
BibTeX
@article{chen2020creativebioman,
  author = {Min Chen and Yingying Jiang and Yong Cao and Albert Y. Zomaya},
  title = {{CreativeBioMan: A Brain- and Body-Wearable, Computing-Based, Creative Gaming System}},
  journal = {IEEE Systems Man and Cybernetics Magazine},
  year = {2020},
  doi = {10.1109/msmc.2019.2929312},
}
Figure