The Design and Development of Instrumented Toys for the Assessment of Infant Cognitive Flexibility

Vishal Ramanathan, Mohammad Zaidi Ariffin, Guo Dong Goh, Mohammad Adhimas Rikat, Xing Xi Tan, Wai Yee Yeong, Juan‐Pablo Ortega, Victoria Leong, Domenico CampoloSensors 2023 · DOI

TL;DR

This paper introduces instrumented toys designed to objectively assess infant cognitive flexibility, a crucial aspect of early executive function development. By embedding sensors (IMU and barometer) into 3D-printed toys, the system automates the detection of object interaction and squeezing, overcoming the limitations of manual video coding in existing infant cognitive assessments.

What Problem Does This Paper Address?

Assessing cognitive flexibility in infants, a key component of executive functions (EFs) vital for complex thinking and socioemotional development, is challenging. Current methods rely on manual coding of infant behavior during object interaction tasks, which is time-consuming, subjective, and prone to rater bias. This manual process hinders scalability and the ability to gather rich, objective data on how infants adapt their behavior. The paper aims to bridge this gap by developing a more objective, reliable, and scalable tool for measuring infant cognitive flexibility.

Methods

The core innovation lies in the design and implementation of instrumented toys. Each toy incorporates a Physilog 6® (P6) sensor, a commercial off-the-shelf unit containing a 9-axis Inertial Measurement Unit (IMU) and a barometer.

  • Toy Construction: The toys are constructed using a modular approach. A 3D-printed octahedral lattice structure made from thermoplastic polyurethane (TPU) filament forms the toy's shell, allowing for variations in rigidity (soft vs. hard). This structure encloses the P6 sensor, which is further sealed with Dragon Skin™ 20 silicone to act as a tactile sensor for detecting squeezing.
  • Sensing Capabilities:
    • IMU: Detects motion and orientation, enabling the tracking of touching sequences and interaction patterns.
  • Barometer: Measures changes in internal air pressure caused by squeezing the toy, quantifying grip force.
  • Data Processing:
    • Squeezing Detection: Raw barometer data is filtered (5 Hz low-pass, then 0.5 Hz high-pass) to remove noise and drift, allowing for the detection of squeezing events.
  • Touching Sequence Detection: IMU accelerometer and gyroscope data are processed through a series of filters (band-pass, full-wave rectification, low-pass) to extract linear envelopes. Thresholds are applied to identify touch events, and the sequence of touches is reconstructed.

Datasets

The paper does not explicitly mention training datasets. Instead, it focuses on the validation of the instrumented toys using experimental data.

  • Squeezing Detection Validation: A robotic arm was used to repeatedly squeeze the instrumented toys (cubes and balls) with a controlled force. The sensor readings from the instrumented toys were compared against a calibrated force sensor.
  • Touching Sequence Detection Validation: A preliminary experiment involved a human participant interacting with four instrumented toys. The data from the instrumented toys was compared against manually coded ground truth data derived from video recordings.

Key Insights and Ideas

  • Barometric Sensing for Tactile Input: The use of a barometer within a sealed silicone enclosure to detect squeezing forces is an innovative and low-cost approach to tactile sensing in toys. This leverages existing sensor technology in a novel application.
  • IMU for Motion and Interaction Tracking: IMUs offer a robust and non-intrusive way to capture infant-object interactions, overcoming line-of-sight issues inherent in camera-based tracking.
  • 3D Printing for Customizable Rigidity: The ability to 3D print toys with varying infill densities and materials (TPU hardness) allows for the creation of toys with different physical properties (soft vs. hard), which is crucial for assessing cognitive flexibility based on material properties.
  • Bridging Lab and Ecological Settings: The design aims to be suitable for both controlled lab environments and more naturalistic home settings, increasing the ecological validity of cognitive assessments.
  • Automating Subjective Tasks: The core idea is to automate the objective measurement of behaviors that were previously subjectively coded, leading to more reliable and scalable data collection.

Key Results

  • Squeezing Detection: The barometer successfully detected squeezing forces applied by a robot, with the instrumented toys registering between 0.58% and 11.46% of the applied pressure. The system was able to differentiate between soft and hard toys, with soft toys absorbing more force. The detected pressure ranges were well within the barometer's dynamic range.
  • Touching Sequence Detection: The IMU-based touch detection achieved an average RMSE of 0.19 seconds when combining accelerometer and gyroscope data, significantly improving upon individual sensor RMSEs (0.32s for accelerometer, 0.23s for gyroscope). This demonstrates the system's ability to accurately identify the timing and sequence of infant touches.

Limitations and Future Work

  • Limitations:
    • The squeezing detection validation was performed with a robot applying a force significantly lower than typical infant grip strength. The paper acknowledges that more representative conditions are needed to fully validate the system's sensitivity.
  • A drift in baseline pressure readings was observed in some sensors, though it was deemed not to be a critical issue for detecting dynamic changes.
  • The current study focused on initial design and validation; further studies are needed to assess robustness, reliability, and practicality in real-world infant interactions.
  • Future Work:
    • Evaluate the instrumented toy's performance in studies with mothers and infants in both lab-based and home-based environments.
  • Further investigate factors such as robustness, reliability, and ease of use.
  • Expand the application to assess other aspects of EF and neurodevelopmental disorders.
  • The modular design allows for integration of other sensors and physical structures for a wider range of applications.

Summary updated

Abstract

The first years of an infant's life represent a sensitive period for neurodevelopment where one can see the emergence of nascent forms of executive function (EF), which are required to support complex cognition. Few tests exist for measuring EF during infancy, and the available tests require painstaking manual coding of infant behaviour. In modern clinical and research practice, human coders collect data on EF performance by manually labelling video recordings of infant behaviour during toy or social interaction. Besides being extremely time-consuming, video annotation is known to be rater-dependent and subjective. To address these issues, starting from existing cognitive flexibility research protocols, we developed a set of instrumented toys to serve as a new type of task instrumentation and data collection tool suitable for infant use. A commercially available device comprising a barometer and an inertial measurement unit (IMU) embedded in a 3D-printed lattice structure was used to detect when and how the infant interacts with the toy. The data collected using the instrumented toys provided a rich dataset that described the sequence of toy interaction and individual toy interaction patterns, from which EF-relevant aspects of infant cognition can be inferred. Such a tool could provide an objective, reliable, and scalable method of collecting early developmental data in socially interactive contexts.
BibTeX
@inproceedings{ramanathan2023design,
  author = {Vishal Ramanathan and Mohammad Zaidi Ariffin and Guo Dong Goh and Guo Liang Goh and Mohammad Adhimas Rikat and Xing Xi Tan and Wai Yee Yeong and Juan‐Pablo Ortega and Victoria Leong and Domenico Campolo},
  title = {{The Design and Development of Instrumented Toys for the Assessment of Infant Cognitive Flexibility}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23052709},
}
Figure