When humans look at a sequence of numbers—such as a stock ticker or a series of sensor readings—we struggle to find immediate meaning. To understand the data, we plot it. By converting one-dimensional numbers into two-dimensional lines, shapes, and textures, we unlock our brain's powerful visual cortex to instantly spot trends, cycles, and anomalies.
A rapidly growing body of research applies this exact intuition to machine learning. Instead of forcing neural networks to process raw, one-dimensional numerical arrays, researchers are transforming time series data into two-dimensional images. This shift allows us to bypass the limitations of traditional sequence models and tap into the massive, highly optimized ecosystem of pre-trained computer vision models, large vision models (LVMs), and multimodal large language models (MLLMs).
The Bottleneck of One-Dimensional Sequences
Traditional time series analysis relies on recurrent neural networks (RNNs), LSTMs, or one-dimensional transformers. While effective for short sequences, these architectures face significant bottlenecks when scaling:
- The Tokenization Problem: As noted by Haoxin Liu et al., standard language model tokenizers are fundamentally designed for discrete text, making them notoriously inefficient and inaccurate when handling long sequences of floating-point numbers.
- Long-Range Dependencies: Capturing periodic behaviors that span thousands of time steps requires massive context windows and struggles with phase shifts or variable sampling rates.
- Data Scarcity: Training a high-capacity 1D model from scratch requires vast amounts of domain-specific data, which is rarely available in specialized fields like medicine or industrial manufacturing.
By encoding temporal data as images, we can treat time series analysis as a visual pattern recognition task. This allows models to leverage the rich spatial reasoning and hierarchical feature extraction capabilities already learned by vision models pre-trained on millions of natural images.
How to Map Time into Space
Transforming a 1D signal into a 2D image requires more than just plotting a line on a graph. To maximize the effectiveness of convolutional layers or vision transformers, the transformation must preserve the underlying temporal correlations, amplitudes, and phase relationships in a spatially coherent format. Researchers utilize several core mathematical frameworks to achieve this.
Gramian Angular Fields (GAF)
Gramian Angular Fields represent one of the most widely adopted encoding techniques. The process begins by scaling a 1D time series so that all values fall within the interval $[-1, 1]$. These scaled values are then mapped onto a polar coordinate system, where the amplitude determines the angular cosine ($\phi$) and the time step determines the radius ($r$).
Once in polar coordinates, the authors calculate the trigonometric correlation between every pair of time steps $i$ and $j$. If we compute the cosine of the sum of angles ($\cos(\phi_i + \phi_j)$), we generate a Gramian Angular Summation Field (GASF). If we compute the sine of the difference ($\sin(\phi_i - \phi_j)$), we get a Gramian Angular Difference Field (GADF).
1D Time Series ──> Scale to [-1, 1] ──> Map to Polar (Angle φ, Radius r) ──> Compute Matrix [i, j] = cos(φ_i + φ_j) ──> 2D Image
Why does this work so well? As explained by H. Long et al. and Jianyuan Ni et al., this polar mapping preserves absolute temporal positions while explicitly representing the relationship between any two points in time as a geometric intersection. In the resulting image, temporal continuity is preserved along the main diagonal, while long-range periodic patterns manifest as distinct, symmetric visual textures that convolutional kernels can easily detect.
Markov Transition Fields (MTF)
While GAF focuses on angular correlations, Markov Transition Fields focus on dynamic probabilities. To construct an MTF, the continuous values of a time series are discretized into $Q$ quantile bins. The algorithm then calculates a $Q \times Q$ Markov transition matrix, which measures the probability that a point in bin $i$ is followed by a point in bin $j$.
To preserve the temporal structure, this transition matrix is mapped back to the time domain, creating an $N \times N$ matrix (where $N$ is the sequence length) where the pixel at coordinate $(i, j)$ represents the transition probability between the bins containing data points $X_i$ and $X_j$. As demonstrated by Zhen Wang et al., this transformation reveals distinct topological patterns—such as chaotic loops or sudden jumps—making it highly effective for identifying structural anomalies in noisy biological signals like ECGs.
Recurrence Plots and Delay Embeddings
Recurrence plots visualize the times at which a dynamic system visits roughly the same state. By calculating the distance between all pairs of time-series vectors in a higher-dimensional phase space (often reconstructed via delay embedding), a binary or grayscale matrix is formed.
Ahmet Alkan demonstrates that this approach allows computer vision models to extract complex, non-linear dependencies and textural features from temporal data that traditional statistical methods overlook. Similarly, Tal Gonen leverages delay embeddings to make 1D sensor data compatible with high-performance generative architectures like diffusion-based UNets.
Sequence Folding and Multi-Scale Matrices
The simplest yet highly intuitive method involves "folding" a 1D sequence into a 2D grid based on known periodicities. For example, Luyao Ma reshapes smart meter data into a date-time matrix where the x-axis represents the time of day and the y-axis represents the calendar date.
This simple reshaping changes the nature of the problem: a long-running appliance like a pool pump, which appears as disconnected intervals in a 1D vector, transforms into a solid, predictable rectangular shape in a 2D image. This allows standard image segmentation models (like U-Net) to isolate the signal from background noise with high precision.
Why Vision Models Excel at Temporal Tasks
Reframing time series as a visual task works because of the inherent architectural strengths of modern vision models.
Texture vs. Sequence
Convolutional neural networks (CNNs) and Vision Transformers (ViTs) are exceptionally good at identifying spatial hierarchies. When a time series is converted via GAF or folding, macro-level trends (like gradual seasonal shifts) become large-scale gradients or color transitions, while micro-level behaviors (like high-frequency noise or sudden faults) become fine-grained visual textures. Gaoliang Peng et al. show that folding techniques place data points from consecutive rotation cycles in close physical proximity, allowing standard 2D convolutional kernels to capture long-range periodic correlations that would span too many steps in a linear sequence.
The "Free Lunch" of Pre-trained Weights
Training a deep network on raw time series data often leads to overfitting due to limited sample sizes. However, models like ResNet, ViT, or Swin Transformer have already learned robust feature extractors from millions of natural images. Mouxiang Chen et al. demonstrate this "free lunch" effect with VisionTS, showing that visual masked autoencoders pre-trained purely on ImageNet can act as zero-shot time series forecasters. By treating forecasting as an image inpainting task—where the future is a masked region of the visual matrix—the model outperforms purpose-built time series foundation models without any temporal training.
Expanding into Multimodal and Generative AI
The success of visual signal encoding has triggered an expansion into multimodal reasoning and generative modeling.
Multimodal LLMs as Analysts
Instead of feeding raw numbers into language models, frameworks like TimeMaster (Junru Zhang et al.) and Time-VLM (Siru Zhong et al.) convert temporal data into visual plots or multi-channel frequency images. This allows multimodal models to utilize their pre-trained spatial reasoning to identify qualitative features like spikes, trends, or complex seasonal patterns. It drastically reduces token consumption while enabling the model to provide structured, interpretable clinical or industrial advice aligned with human visual intuition.
Generative Diffusion and Simulation
The transformation works symmetrically for generation. By converting historical financial data into 2D OHLC (Open-High-Low-Close) images, Jinseong Park et al. trained generative diffusion models to simulate highly realistic future price paths. Because the diffusion model operates on images, it naturally captures visual signatures of human decision-making—such as support levels, momentum, and reversals—that are notoriously difficult to mathematically formalize in 1D equations.
Conclusion
Visual signal encoding bridges the gap between two traditionally distinct domains. By mapping the abstract, non-linear relationships of time series data into geometric and textural patterns, researchers have unlocked a way to process temporal data using the world's most advanced vision architectures. Whether diagnosing a heart arrhythmia, predicting a cyber attack, or forecasting stock prices, treating time as a spatial dimension allows neural networks to "see" the answers hidden within the numbers.