CoolerSpace: A Language for Physically Correct and Computationally Efficient Color Programming

Ethan Chen, Jiwon Chang, Yuhao ZhuProc. ACM Program. Lang 2024 · arXiv

TL;DR

This paper introduces CoolerSpace, a domain-specific language (DSL) for color programming that enforces physical correctness and optimizes performance. By integrating a type system that understands color physics, CoolerSpace prevents common bugs and automatically generates efficient code, outperforming existing libraries.

What Problem Does This Paper Address?

Color programming, the manipulation of lights, materials, and their resulting colors, is fundamental across many fields, from computer graphics and imaging to art and conservation. However, existing programming tools, like Python libraries (NumPy, OpenCV), offer only a thin abstraction over raw tensor operations. This means programmers are responsible for manually tracking the physical meaning of data (e.g., spectral power distributions, material reflectance) and ensuring physically valid computations. This often leads to subtle "silent data corruption" bugs where mathematically permissible operations are physically meaningless or incorrect. Furthermore, implementing physically accurate color science algorithms and optimizing them for performance is a significant challenge, especially for domain experts who may not be proficient in advanced programming techniques.

Methods

CoolerSpace addresses these issues through a multi-pronged approach:

  • Domain-Specific Language (DSL): CoolerSpace is implemented as a Python library, making it accessible to a wide range of users.
  • Physics-Aware Type System: The core of CoolerSpace is its type system. It assigns physical meaning and dimensions to data (e.g., LightSpectrum, sRGB, Pigment). Typing rules are designed to enforce the principles of color physics and perception, preventing operations that are physically meaningless or mathematically incorrect.
  • Compilation to ONNX: CoolerSpace programs are compiled into ONNX (Open Neural Network Exchange), a standard intermediate representation for tensor computations. This allows leveraging existing ONNX runtimes and tools.
  • Equality Saturation-Based Optimizer: After translation to ONNX, an optimizer uses equality saturation to find semantically equivalent but more performant ONNX graphs. This technique enumerates equivalent programs using rewrite rules and selects the one with the lowest estimated cost.
  • Formal Translational Semantics: The paper defines formal semantics to prove that the translation from CoolerSpace to ONNX preserves the program's meaning and type safety.

Datasets

Not discussed in the paper. The paper focuses on the language and its compilation/optimization pipeline, using example programs for evaluation rather than specific datasets.

Key Insights and Ideas

  • The "Silent Data Corruption" Problem: A significant insight is the prevalence of subtle bugs in color programming that don't cause crashes but lead to physically incorrect results. These arise from the disconnect between mathematical operations and physical meaning.
  • Type System as a Physics Enforcer: CoolerSpace demonstrates how a well-designed type system can act as a powerful tool to enforce physical laws and perceptual principles, rather than just mathematical constraints.
  • Abstraction for Domain Experts: The system is designed to empower domain experts (color scientists, vision scientists) by abstracting away complex implementation details and optimization concerns, allowing them to focus on the physics of their problems.
  • Performance Without Runtime Overhead: By performing compilation and optimization offline, CoolerSpace aims to provide correctness guarantees without introducing runtime overhead, and in fact, to improve performance.

Key Results

  • Bug Prevention: CoolerSpace's type system prevents common errors like incorrect color space conversions and physically meaningless operations (e.g., adding non-linear sRGB values directly).
  • Performance Improvement: Unoptimized CoolerSpace programs outperform existing Python libraries (Colour-Science, Numba) by up to 5.7x.
  • Optimization Gains: The equality saturation optimizer provides an additional speed-up of up to 1.4x.
  • Compilation/Optimization Time: Compilation and optimization times are minimal, typically under 3 seconds, making the offline phase cost-effective.

Limitations and Future Work

  • Limited Color Space Support: CoolerSpace currently supports a small number of color spaces, though the authors note this is not an inherent limitation and can be extended.
  • Bidirectional Type Checking Feasibility: Implementing bidirectional type checking for tristimulus space addition was deemed infeasible in a Python library context due to Python's dynamic typing.
  • Future Optimizations: The authors suggest that further optimizations could be achieved by exploring more advanced cost functions (e.g., hardware-aware models) and potentially compiling to more efficient targets than ONNX.
  • Extending Type System: The type system could be extended to support more complex physical phenomena like non-diffuse surfaces (BRDF/BSSDF).

Summary updated

BibTeX
@inproceedings{chen2024coolerspace,
  author = {Ethan Chen and Jiwon Chang and Yuhao Zhu},
  title = {{CoolerSpace: A Language for Physically Correct and Computationally Efficient Color Programming}},
  booktitle = {Proc. ACM Program. Lang},
  year = {2024},
}
Figure