What Image Metrics See When They Look at a Time Series

A growing family of methods turns a time series into a picture so that image models can do the work. The best-known encoding is the Gramian Angular Field (GAF). It rescales the series, turns each value into an angle, and fills an $N \times N$ image whose pixel $(i,j)$ is the cosine of the sum of the angles at times $i$ and $j$. When these pipelines generate or reconstruct data, it is tempting to judge the output with standard image metrics such as Fréchet Inception Distance (FID, a distance between feature statistics from an ImageNet-trained network), learned perceptual distances, or structural similarity. Image researchers already know these metrics drift away from what humans judge as good quality. This note argues that on GAF images the drift is worse, and that it can be predicted in advance. The GAF is a fixed, known function of the series, so any image metric applied to it is secretly a time-series metric with specific blind spots. We can derive those blind spots on paper.

The proposal is to derive the invariances that GAF-space metrics inherit, test the predictions against controlled temporal distortions and real generators, and then build a small GAF evaluator whose invariances are chosen on purpose rather than inherited from ImageNet. In two sentences: off-the-shelf image metrics on GAF images should be close to blind to amplitude damping and time shifts, which are exactly the failures that mean-squared-error-trained time-series models commit. We show this analytically and empirically, and we test whether a corrected GAF evaluator adds anything beyond the best one-dimensional evaluators.

Core Claim

Three properties of the GAF map determine what an image metric can see.

The first is amplitude. With per-window min–max scaling, any rescaling $a x + b$ with $a>0$ leaves the GAF image unchanged. A forecaster that predicts a damped, flattened version of the truth therefore produces an image identical to the truth's whenever the prediction is normalized on its own. Damped, flattened predictions are the "structural blindness" that Patch-wise Structural Loss for Time Series Forecasting documents for MSE-trained models.

The second is timing. Shifting a series by $k$ steps translates its image along the diagonal:

$$\mathrm{GASF}^{(x_{\cdot+k})}_{i,j} = \mathrm{GASF}^{(x)}_{i+k,\,j+k},$$

where $\mathrm{GASF}^{(x)}$ is the summation-field image of series $x$. Away from the window edges, a time shift is just a spatial translation of the image. Convolutional features are built to be roughly insensitive to translation, so deep-feature metrics should barely register phase errors. Pixel-wise distances do the opposite and over-penalize them, the "geometry of blur" effect described for images in Perceptual Losses for Real-Time Style Transfer and Super-Resolution. Image metrics on GAFs therefore get timing wrong in both directions.

The third is noise. Each pixel is a smooth product of two bounded values, so the image spreads high-frequency noise across many low-contrast pixels. Transfer Learning in Financial Time Series with Gramian Angular Field found this smoothing useful: choosing source domains by similarity on GAF images lowered an LSTM's error on HSBC from MAPE 1.4436 to 1.1058, which the authors credit to noise suppression. In evaluation, the same property becomes a risk, because a generator that adds or removes small-scale noise may look unchanged.

The new contribution is not "apply FID to GAFs." It is three things together: predicting from the algebra of the map which temporal errors each class of image metric will miss, confirming those predictions quantitatively, and building an evaluator whose sensitivities are set deliberately. The evaluator uses normalization shared with the reference, separate scores for the near-diagonal band (local dynamics) and the far off-diagonal region (long-range structure), and a feature extractor trained on GAF images rather than ImageNet.

Relation to Prior Work

Three generative papers motivate the question but do not ask it. Utilizing Image Transforms and Diffusion Models for Generative Modeling of Short and Long Time Series (ImagenTime) generates series by converting them to images with delay embeddings or short-time Fourier transforms, running an image diffusion model, and inverting the result. Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach extends this to few-shot settings. VisionTS folds series into periodic grids and has an ImageNet-trained masked autoencoder fill in the future. All three judge results with one-dimensional scores, never use GAF, and never ask whether image-space quality tracks temporal quality. We ask that question, and we include their transforms as comparison encodings.

Two image-metric papers supply methods we reuse and critique. Using Skew to Assess the Quality of GAN-generated Image Features shows that Inception features are strongly skewed and adds a third-moment term to FID, called SID. We reuse SID as one audited metric and as a statistic in our evaluator, and we keep in mind that it is a pseudometric: it returns zero for distinct distributions whose first three moments match. BiRQA: Bidirectional Robust Quality Assessment for Images builds a full-reference quality model from handcrafted features: structural-similarity maps, local information content, color differences, and local binary patterns. Its color and texture features assume natural RGB scenes, so it is a good test of whether low-level hand-designed features transfer to single-channel trigonometric images better than deep ones.

Two papers serve as the one-dimensional evaluators our proposal must beat or complement. A Geometry-Aware Metric for Mode Collapse in Time Series Generative Models (DMD-GEN) decomposes series into linear dynamic modes and compares generators with optimal transport between the subspaces those modes span. It needs no training, but it only sees linear dynamics. Information Geometry Loss for Time Series Forecasting and the patch-wise structural loss above show that one-dimensional fixes for MSE's blind spots work well in forecasting. That evidence pushes us to test the GAF evaluator against these baselines rather than assume a two-dimensional view helps.

Cross-modal Knowledge Distillation for Vision-to-Sensor Action Recognition uses GAF images only as classifier input. It shows that image networks extract useful signal from GAFs, but it says nothing about evaluation.

Research Program

The first stage is analysis. For each metric class, we derive its exact invariances under the GAF map: pixel distance, structural similarity, deep-feature distances (FID, SID, learned perceptual distance), and BiRQA-style handcrafted features. We do this for both normalization choices, per-window and shared with the reference, and for both summation and difference fields. The difference field $\sin(\phi_i-\phi_j)$ is antisymmetric and keeps sign information that the summation field's diagonal $2\tilde{x}_i^2-1$ discards. The analysis yields a table of predictions, kept in our notes, stating which distortions each metric should be blind to, over-penalize, or track faithfully.

The second stage is a controlled distortion suite on real windows from ETTh1, ETTm1, Stocks, Energy, and Sines. We apply graded distortions with known severity: amplitude damping, level offset, circular and non-circular time shifts, spike deletion, added white and colored noise, low-pass smoothing, frequency drift, and, at the set level, mode collapse created by sampling from a shrinking subset of clusters. The gold standard is the known severity, together with the drop in train-on-synthetic, test-on-real forecasting accuracy.

The third stage uses real generators: TimeGAN, TimeVAE, and DiffusionTS (the generators DMD-GEN audits), ImagenTime, and a GAF variant of the ImagenTime pipeline. For the GAF variant, inversion recovers $\tilde{x}_i$ from the diagonal of the difference field and the first row, then symmetrizes the image. We record how often a generated GAF cannot be inverted cleanly, because that failure is itself a quality signal.

The fourth stage is the corrected evaluator. It normalizes each generated image using the reference's minimum and maximum. It computes separate distribution distances on the near-diagonal band and the far off-diagonal region. For features, it uses an encoder pretrained self-supervised on GAF images from held-out datasets instead of Inception, and it applies the third-moment statistic of SID to those features. It also reports, as a separate score, how far each generated image is from a valid Gramian matrix, measured by asymmetry and by the residual after projecting onto the nearest valid GAF.

Baselines

The naive reference point is image FID with Inception features and per-window normalization on GAF images, along with pixel MSE and structural similarity. This is what a practitioner would plug in. Its expected failure is the claim under test, and showing that the failure matches the algebraic prediction is the first result.

The strongest image-side competitors are SID Using Skew to Assess the Quality of GAN-generated Image Features and BiRQA BiRQA. BiRQA reports Spearman rank correlations of 0.988 on LIVE and 0.979 on CSIQ with human scores, but those are natural-image results. No reported result was found on Lacuna for BiRQA or SID on GAF or any other time-series image. We run SID with the published 256-dimensional reduction. For BiRQA, we fine-tune on our distortion suite under the same budget and data as our evaluator; the color features receive the GAF replicated across three channels.

The strongest overall competitors are one-dimensional: contextFID and the discriminative score used by ImagenTime and the data-scarcity paper, and DMD-GEN. For context, ImagenTime reports a 58.17% improvement in discriminative score over earlier diffusion baselines, and the data-scarcity paper reports 54.25% better contextFID in few-shot settings. These are improvements in generator quality, not measures of how well the evaluators agree with ground truth. No reported rank-agreement number for these evaluators was found on Lacuna, and DMD-GEN's summary gives rankings without raw values. All evaluators see the same windows, the same reference sets, and the same number of samples. Any learned component gets the same pretraining data, drawn only from datasets excluded from evaluation.

Beating the baselines means two things. Rank correlation with known distortion severity must be higher on the distortions the analysis flags. Agreement with forecasting utility under train-on-synthetic, test-on-real must match or exceed contextFID and DMD-GEN across generator rankings. A further acceptable outcome is that the evaluator adds significant explained variance when combined with them.

Three ablations isolate the main ingredients. The first swaps shared normalization back to per-window normalization, which should restore amplitude blindness. The second swaps the GAF-trained encoder for Inception, which should restore near-invariance to time shifts. The third collapses the band-versus-far split into one global distance.

Why not just…?

Why not just evaluate in one dimension with contextFID, discriminative score, or DMD-GEN and ignore image metrics? This is the real threat, and the program is built to give it a fair hearing. If the corrected GAF evaluator neither beats nor adds to these scores, the conclusion is that GAF-space evaluation should not be used. The audit then still has value, because it shows why image metrics should not be reported for transformed pipelines. Our reason to expect complementarity is that DMD-GEN only sees linear modes and the discriminative score depends on how well its classifier is trained, whereas the far off-diagonal region of a GAF encodes pairwise long-range structure directly. The combined-variance test settles the question.

Why not just use delay embeddings or STFT, which invert cleanly? ImagenTime chose these transforms for exactly that reason. The audit applies to them as well: delay-embedding images also turn time shifts into translations, so deep features should be blind to phase errors there too. We therefore run the full distortion suite on all three encodings. If delay embeddings give better-aligned image metrics with no inversion failures, we will report that GAF is the worse choice for evaluation.

Why not just apply FID, SID, or BiRQA as they are? They are the baselines. The analysis predicts specific blind spots, with amplitude invisible under per-window normalization and timing poorly resolved by deep features, and these predictions can be checked directly. If those metrics turn out to track the distortion severities, the proposal's central claim is wrong. That result would itself be clear and publishable.

Decisive Checks

The first check tests the predictions. Under per-window normalization, FID and SID on GAFs should show near-zero rank correlation with amplitude-damping severity, and deep-feature metrics should rise much more slowly than pixel distance as the time shift grows. If either prediction fails, the analytic account is wrong.

The second check is a hidden-collapse test. We build generator pairs whose GAF-image FID is equal but whose forecasting utility under train-on-synthetic, test-on-real differs sharply, through damping or phase drift. We then ask which evaluator ranks the pair correctly.

The third check uses the real generators. Across the five generators and five datasets, we compute Spearman correlation between each evaluator's ranking and the forecasting-utility ranking, and we use bootstrap intervals over random seeds. The corrected evaluator must be significantly better than image FID on GAFs and at least tied with the best one-dimensional evaluator, or it must add significant explained variance alongside it. If it fails both tests, the note's recommendation becomes simply to evaluate time-series image pipelines in one dimension.