DynTex: A real-time generative model of dynamic naturalistic luminance textures

Andrew Isaac Meso, Jonathan Vacher, Nikos Gekas, Pascal Mamassian, Laurent Perrinet, Guillaume S. MassonJournal of Vision 2025 · DOI

TL;DR

This paper introduces DynTex, an open-source toolbox for generating dynamic, naturalistic visual textures called Motion Clouds (MCs). DynTex allows for real-time, GPU-accelerated control over stimulus parameters, bridging the gap between highly artificial synthetic stimuli and complex natural videos for vision research.

What Problem Does This Paper Address?

Understanding how the visual system processes information in a dynamic environment is crucial for fields like psychophysics and computational neuroscience. However, existing synthetic stimuli (like gratings or random dot kinematograms) are often too simplistic, while natural videos lack the precise experimental control needed for rigorous scientific study. This creates a need for stimuli that are both naturalistic in their statistical properties and fully controllable, allowing researchers to probe specific aspects of visual processing. Previous attempts with "Motion Clouds" (MCs) were computationally intensive, limiting their real-time application.

Methods

The core of DynTex is its implementation of Motion Clouds (MCs) using an autoregressive (AR2) algorithm. This approach allows for efficient, frame-by-frame generation of dynamic textures. Key aspects include:

  • Autoregressive Generation: Instead of pre-computing entire stimulus sequences, the AR2 method generates frames iteratively, enabling real-time parameter adjustments.
  • GPU Acceleration: The computationally intensive calculations are offloaded to the GPU, significantly speeding up generation and allowing for real-time manipulation.
  • Object-Oriented Design: The toolbox is built with two classes, DynTex (parent) and MotionCloud (child), providing a modular and extensible structure.
  • Parameter Control: Users can precisely control various spatiotemporal parameters, including orientation, spatial frequency, and speed, along with their distributions (mean and bandwidth).
  • GUI Visualization: A graphical user interface (GUI) provides real-time visualization of stimulus changes and their spectral properties, offering intuitive feedback to researchers.
  • Integration: The toolbox is designed for easy integration with popular psychophysics environments like Psychtoolbox and PsychoPy.

Datasets

Not discussed in the paper. The paper focuses on the generation of synthetic stimuli rather than using pre-existing datasets for training or evaluation.

Key Insights and Ideas

  • Bridging the Gap: MCs, as implemented in DynTex, offer a powerful way to bridge the gap between highly controlled but artificial stimuli (gratings, dots) and complex, less controllable natural videos.
  • Real-time Control is Key: The ability to adjust stimulus parameters in real-time, enabled by the AR2 algorithm and GPU acceleration, is a significant advancement for psychophysical experiments. This allows for more dynamic and adaptive experimental designs.
  • Intuitive Parameterization: The GUI provides an intuitive way for researchers to understand how changes in parameters affect the visual texture, facilitating experimental design and hypothesis generation.
  • Naturalistic Statistics: MCs are designed to mimic the statistical properties of natural images and movies, making them more ecologically relevant for studying visual processing.
  • Versatility for Diverse Research: The toolbox's flexibility makes it applicable to a wide range of research areas, from low-level motion perception to perceptual organization and even cross-modal studies.

Key Results

  • Speed: GPU-accelerated MC generation can achieve real-time performance (under 10 ms per frame), enabling stimulus synthesis at 100 Hz for typical stimulus sizes. CPU computations are significantly slower (13-19x).
  • Efficiency: The AR2 method is orders of magnitude faster than previous FFT-based methods for MC generation, especially for larger image sizes.
  • Parameter Control: The toolbox allows for fine-grained control over stimulus properties like spatial frequency, orientation, and speed bandwidths, which have been shown to influence perceptual judgments and neural responses.
  • Experimental Examples: The paper demonstrates the utility of DynTex with examples of low-level motion tasks (speed discrimination) and perceptual organization tasks (distinguishing one vs. two motion directions).

Limitations and Future Work

  • Computational Requirements: While GPU acceleration is a major improvement, high-end GPUs are still recommended for optimal performance, especially for very large image sizes or complex stimuli.
  • Future Development: The authors express their intention to continue developing DynTex and hope to foster a wider community of users and contributors. They suggest that the toolbox is well-suited for continuous psychophysics paradigms, which require rapid stimulus adjustment.
  • Further Exploration: The paper highlights the potential for using MCs to investigate a broader range of perceptual phenomena, including cross-modal integration, developmental studies, and aging research.

Summary updated

Abstract

The visual systems of animals work in diverse and constantly changing environments where organism survival requires effective senses. To study the hierarchical brain networks that perform visual information processing, vision scientists require suitable tools, and Motion Clouds (MCs)-a dense mixture of drifting Gabor textons-serve as a versatile solution. Here, we present an open toolbox intended for the bespoke use of MC functions and objects within modeling or experimental psychophysics contexts, including easy integration within Psychtoolbox or PsychoPy environments. The toolbox includes output visualization via a Graphic User Interface. Visualizations of parameter changes in real time give users an intuitive feel for adjustments to texture features like orientation, spatiotemporal frequencies, bandwidth, and speed. Vector calculus tools serve the frame-by-frame autoregressive generation of fully controlled stimuli, and use of the GPU allows this to be done in real time for typical stimulus array sizes. We give illustrative examples of experimental use to highlight the potential with both simple and composite stimuli. The toolbox is developed for, and by, researchers interested in psychophysics, visual neurophysiology, and mathematical and computational models. We argue the case that in all these fields, MCs can bridge the gap between well- parameterized synthetic stimuli like dots or gratings and more complex and less controlled natural videos.
BibTeX
@article{meso2025dyntex,
  author = {Andrew Isaac Meso and Jonathan Vacher and Nikos Gekas and Pascal Mamassian and Laurent Perrinet and Guillaume S. Masson},
  title = {{DynTex: A real-time generative model of dynamic naturalistic luminance textures}},
  journal = {Journal of Vision},
  year = {2025},
  doi = {10.1167/jov.25.11.2},
}
Figure