90 Metrics, Complete
Transparency
Every claim on this site is backed by reproducible benchmarks. We show our wins, our losses, and even our overfitting analysis.
ColorBench: deterministic, float64, 13 categories. All data, scripts, and checkpoints are open source. No cherry-picking. Full train/test split analysis included.
83 internal + 7 independent validation = 90 total metrics
Color Difference Accuracy
MetricSpace predicts human color perception more accurately than any existing standard, including the industry-standard CIEDE2000 formula.
STRESS (CIE 217:2016) evaluated on COMBVD (3,813 pairs from 6 sub-datasets) and the held-out MacAdam 1974 set (128 pairs). Lower is better. Our own human-feedback set is reported separately below — rank-order only.
Our Own Human-Feedback Set rank-order only
3,477 judgements by 71 observers over 47 color pairs, collected on helmlab.space. We audited this set and do NOT publish STRESS on it: ratings are 5-level categorical, which breaks interval-scale metrics — any STRESS number from it (ours included) is a scale artifact. Rank order is the honest statistic. Two more caveats: pairs were sampled from helmlab's own hue zones (not independent), and with 47 pairs the MetricSpace–CIEDE2000 gap is only marginally significant (Δρ 95% CI [+0.002, +0.117], paired bootstrap).
More Held-Out Datasets
STRESS on independent psychophysical sets MetricSpace was never trained on (same protocol: Bradford CAT to D65, CAM16-UCS via colour-science with a gray-ramp sanity check). One of these we lose — published anyway.
| Dataset | MetricSpace | CIEDE2000 | CAM16-UCS | CIE Lab | OKLab |
|---|---|---|---|---|---|
| Munsell renotation (neighbor pairs) n=3590 constant perceptual spacing between adjacent chips; illuminant C, Bradford-adapted | 30.34 | 42.94 | 43.70 | 43.99 | 51.95 |
| He 2022 (wide-gamut display pairs) n=82 10° observer data, DV published in CIELAB-scaled units — we lose this one | 35.89 | 32.58 | 34.42 | 30.77 | 49.04 |
Generation Benchmark: 62-9
GenSpace is purpose-built for creating colors: gradients, palettes, gamut mapping. Head-to-head against OKLab (the current CSS standard), it wins 62 out of 90 metrics.
83 internal metrics (deterministic, float64) + 7 independent validation metrics. Opponent: OKLab with standard Euclidean deltaE. Same test harness, same precision.
Category Breakdown
Click any card to expand and see every individual metric in that category.
Performance breakdown across 13 test categories. Click any card to see individual metrics.
83 internal metrics + 7 independent validation metrics in 13 categories. All deterministic, float64.
Gamut
How well the space maps to real device screens
Cusp validity, boundary smoothness, clipping across sRGB/P3/Rec2020
Application
Real-world tasks like palettes, tints, and accessibility
Palette generation, gamut mapping, WCAG contrast, animation
Independent
Tests using data we never trained on
Hung-Berns, Ebner-Fairchild, Pointer's Gamut
Gradient
How smooth colors blend between two endpoints
CV of perceptual step size, hue drift, banding metrics
Perceptual
Agreement with how humans actually see color
Munsell, MacAdam, Hung-Berns hue linearity validation
Structural
Mathematical properties that affect reliability
Hue reversals, OOG excursion, chroma amplification, LMS
Achromatic
Perfect grays without color contamination
Gray ramp chroma residual under sRGB and D65
Advanced
Edge cases and stress tests
1000-trip roundtrip, Jacobian condition, 8-bit precision
Hue
Whether hue labels match human expectation
Hue RMS vs Munsell, primary lightness range
Special
Problem areas where OKLab is known to struggle
Yellow chroma, blue-to-white midpoint, red-to-white shift
Accessibility
Usability for colorblind viewers
CVD simulation minimum step deltaE (protan/deutan)
Banding
Visible stepping artifacts in gradients
Invisible step ratio, duplicate 8-bit bucket count
Numerical
Mathematical precision of conversions
Round-trip error across sRGB, P3, Rec2020 (float64)
Metric Explorer
Search and filter all 90 benchmark metrics. Every number is reproducible.
Sortable, filterable table of all metrics. Values are from the canonical ColorBench run: GenSpace v0.11.1 vs OKLab, both at float64 precision.
| Metric | Category | OKLab | GenSpace | Winner |
|---|---|---|---|---|
| CVD deutan min step dEΔE | Accessibility | 0.16 | 0.11 | OKLab |
| CVD protan min step dEΔE | Accessibility | 0.13 | 0.13 | GenSpace |
| Gray ramp pure C*C* | Achromatic | 7.61e-7 | 1.88e-15 | GenSpace |
| Gray ramp sRGB C*C* | Achromatic | 6.04e-7 | 1.76e-15 | GenSpace |
| 1000-trip RTmax ΔE | Advanced | 6.17e-13 | 2.53e-14 | GenSpace |
| 8-bit exact/10Kcount | Advanced | 10,000 | 10,000 | Tie |
| Animation frame-to-frame CV% | Advanced | 55.3 | 53.4 | GenSpace |
| Channel mono violationscount | Advanced | 0 | 0 | Tie |
| Cross-gamut amplification× | Advanced | 1× | 1× | Tie |
| Jacobian condition | Advanced | 6.49 | 6.47 | Tie |
| Chroma preservation (no mud) | Application | 0.414 | 0.41 | Tie |
| Data viz min pairwise dEΔE | Application | 14.08 | 13.27 | OKLab |
| Eased animation CV% | Application | 55.1 | 55.3 | Tie |
| Muddy gradients (C drop >50%)count | Application | 12 | 12 | Tie |
| Multi-stop gradient CV% | Application | 37.4 | 36.9 | GenSpace |
| Palette harmony accuracy° | Application | 11.7 | 9.1 | GenSpace |
| Palette L* spacing% | Application | 78.9 | 76.5 | GenSpace |
| Photo gamut map fidelity° | Application | 0.98 | 0.96 | GenSpace |
| Shade palette hue drift° | Application | 8.6 | 6 | GenSpace |
| Shade palette worst hue drift° | Application | 20.9 | 20.4 | GenSpace |
| Tint/shade hue preservation° | Application | 8.8 | 7.9 | GenSpace |
| WCAG midpoint contrast:1 | Application | 2.73 | 2.88 | GenSpace |
| Duplicate 8-bit steps% | Banding | 16.1 | 13.8 | GenSpace |
| Invisible gradient steps% | Banding | 96.7 | 97 | Tie |
| Cusp smoothness (max jump) | Gamut | 0.805 | 0.075 | GenSpace |
| Gamut volume fill% | Gamut | 1 | 1 | Tie |
| P3 boundary bad huescount | Gamut | 121 | 4 | GenSpace |
| P3 boundary continuity | Gamut | 0.444 | 0.079 | GenSpace |
| P3 boundary mean jump | Gamut | 0.02 | 0.003 | GenSpace |
| P3 cliff max% | Gamut | 0.2 | 0.2 | Tie |
| P3 cusp mean smoothness | Gamut | 0.008 | 0.005 | GenSpace |
| P3 cusp smoothness | Gamut | 0.778 | 0.043 | GenSpace |
| P3 invalid cuspscount | Gamut | 52 | 0 | GenSpace |
| P3 mono violationscount | Gamut | 71 | 0 | GenSpace |
| P3 valid cuspscusps | Gamut | 308/360 | 360/360 | GenSpace |
| Rec2020 boundary bad huescount | Gamut | 130 | 20 | GenSpace |
| Rec2020 boundary continuity | Gamut | 0.562 | 0.248 | GenSpace |
| Rec2020 boundary mean jump | Gamut | 0.025 | 0.006 | GenSpace |
| Rec2020 cliff max% | Gamut | 0.7 | 0.2 | GenSpace |
| Rec2020 cusp mean smoothness | Gamut | 0.007 | 0.006 | GenSpace |
| Rec2020 cusp smoothness | Gamut | 0.756 | 0.157 | GenSpace |
| Rec2020 mono violationscount | Gamut | 60 | 1 | GenSpace |
| Rec2020 valid cuspscusps | Gamut | 360/360 | 360/360 | Tie |
| sRGB boundary bad huescount | Gamut | 123 | 15 | GenSpace |
| sRGB boundary continuity | Gamut | 0.545 | 0.301 | GenSpace |
| sRGB boundary mean jump | Gamut | 0.02 | 0.005 | GenSpace |
| sRGB cliff max% | Gamut | 0.7 | 0.2 | GenSpace |
| sRGB cusp mean smoothness | Gamut | 0.009 | 0.005 | GenSpace |
| sRGB invalid cuspscount | Gamut | 61 | 0 | GenSpace |
| sRGB mono violationscount | Gamut | 88 | 0 | GenSpace |
| sRGB valid cuspscusps | Gamut | 299/360 | 360/360 | GenSpace |
| 3-color gradient CV% | Gradient | 26.59 | 23.42 | GenSpace |
| Banding mean | Gradient | 1.8 | 1.8 | Tie |
| Bright gradient CV (L>0.6)% | Gradient | 23.52 | 25.43 | OKLab |
| Cross-lightness gradient CV% | Gradient | 15.51 | 11.21 | GenSpace |
| Dark gradient CV (L<0.4)% | Gradient | 46.53 | 33.68 | GenSpace |
| Gradient CV (mean)% | Gradient | 30.27 | 30.12 | Tie |
| Gradient CV (p95)% | Gradient | 138.2 | 138.73 | Tie |
| High-chroma gradient CV% | Gradient | 21.76 | 19.27 | GenSpace |
| Max hue drift (non-crossing)° | Gradient | 112.7 | 77.5 | GenSpace |
| Near-achromatic gradient CV% | Gradient | 79.26 | 102.1 | OKLab |
| Worst-case gradient CV% | Gradient | 362.2 | 500 | OKLab |
| Hue RMS° | Hue | 30.1 | 27.5 | GenSpace |
| Primary L range | Hue | 0.516 | 0.6 | GenSpace |
| Ebner-Fairchild hue surfaces (max)° | Independent | 8.1 | 8.6 | OKLab |
| Ebner-Fairchild hue surfaces (mean)° | Independent | 2.23 | 2.11 | GenSpace |
| Hung-Berns hue linearity (max)° | Independent | 25.5 | 25.2 | GenSpace |
| Hung-Berns hue linearity (mean)° | Independent | 4.96 | 4.72 | GenSpace |
| Pointer gamut boundary smoothness | Independent | 0.132 | 0.125 | GenSpace |
| Pointer gamut chroma isotropy | Independent | 0.413 | 0.404 | GenSpace |
| Pointer gamut hue uniformity | Independent | 0.37 | 0.261 | GenSpace |
| Round-trip P3 16.7Mmax ΔE | Numerical | 1.67e-15 | 1.89e-15 | Tie |
| Round-trip Rec2020 2.1Mmax ΔE | Numerical | 1.55e-15 | 1.67e-15 | Tie |
| Round-trip sRGB 16.7Mmax ΔE | Numerical | 1.78e-15 | 1.78e-15 | Tie |
| Hue agreement with CIE Lab° | Perceptual | 8.5 | 8.3 | GenSpace |
| Hue leaf constancy° | Perceptual | 73.3 | 59.8 | GenSpace |
| MacAdam isotropyratio | Perceptual | 1.99 | 1.78 | GenSpace |
| Munsell Hue spacing% | Perceptual | 18.5 | 11.4 | GenSpace |
| Munsell Value uniformity% | Perceptual | 2.8 | 0.16 | GenSpace |
| Blue-White midpoint G/R | Special | 1.408 | 1.514 | GenSpace |
| Red-White midpoint G-B | Special | 0.062 | 0.063 | OKLab |
| Yellow chroma | Special | 0.211 | 0.333 | GenSpace |
| Extreme chroma amplification× | Structural | 5.79× | 3.79× | GenSpace |
| Hue reversal max angle° | Structural | 3 | 0.6 | GenSpace |
| Hue reversals (count)count | Structural | 80 | 67 | GenSpace |
| Negative LMS colors% | Structural | 0 | 0 | Tie |
| OOG excursion pairs% | Structural | 9.8 | 9.8 | Tie |
| OOG max distance | Structural | 0.11 | 0.103 | GenSpace |
| Primary hue disc (P3)° | Structural | 1.08 | 1.37 | OKLab |
| Primary hue disc (sRGB)° | Structural | 1.31 | 1.65 | OKLab |
Tested on Data We Never Trained On
Three independent datasets from published color science research (1980-1998). GenSpace wins 6-1 against OKLab on data it never saw.
Hung & Berns 1995 (hue linearity, 168 samples), Ebner & Fairchild 1998 (constant-hue surfaces, 321 samples), Pointer 1980 (real surface color gamut, 576 boundary points). None used in optimization.
Hung & Berns 1995
168 samplesDo straight lines in the color space match straight lines in human hue perception?
Hue linearity: angular deviation from constant-hue lines. 12 hues, 13 targets each, 9 observers.
Ebner & Fairchild 1998
321 samplesWhen you change lightness and chroma but keep the hue name the same, does the color space agree?
Constant perceived-hue surface deviation. 15 hues. Mean and max angular deviation from ideal.
| Space | Mean | Max |
|---|---|---|
| CIE Lab | 2.95 | 16.0 |
| OKLab | 2.23 | 8.1 |
| GenSpace | 2.10 | 8.6 |
Pointer's Gamut 1980
576 ptsHow uniformly does each space represent real-world surface colors?
Real surface color boundary (16 L levels, 36 hue angles). Chroma CV, boundary smoothness, hue uniformity.
| Space | C* CV | Smooth | Hue CV |
|---|---|---|---|
| CIE Lab | 0.479 | 0.144 | 0.034 |
| OKLab | 0.413 | 0.132 | 0.370 |
| GenSpace | 0.404 | 0.125 | 0.262 |
Independent Validation Total
Across 3 published datasets (1980-1998), none used in training
Overfitting Analysis
We optimized MetricSpace on color difference data. Could it have just memorized the answers? We tested this honestly and show you the results.
80/20 stratified split (seed=42), multiple DOF configurations. Train-test gap exists (+1.8) but held-out test still beats all competitors.
Does the model genuinely predict color perception, or did it just memorize the training data? We tested this rigorously with held-out data the model never saw during training.
80/20 train-test split (seed=42, 3050/763 pairs). Multiple DOF configurations tested. Cross-validated estimate: STRESS 24.3.
| Model | ParamsDOF | Train | Test | Gap |
|---|---|---|---|---|
| v20b baseline | 0 | 27.72 | 27.57 | -0.15 |
| v21 (full-data) | 72 | 22.14 | 23.91 | +1.77 |
| Phase 1 train-only | 6 | 25.35 | 25.65 | +0.30 |
| Phase 1+2 train-only | 48 | 22.78 | 24.59 | +1.82 |
Key Findings
Published STRESS: 22.48 (full-data optimized) | Cross-validated: ~24.3 | Both are still #1 among all tested competitors.
When to Use MetricSpace
MetricSpace is purpose-built for color difference prediction — not generation. Use it when you need to measure, not create.
Quality Control
Print, display, textile color matching. 23% lower STRESS than CIEDE2000 on COMBVD.
Color Matching Tolerance
Pair-dependent SL/SC weighting adapts to the specific lightness and chroma of each color pair.
A/B Testing
Ranks real user judgments better than CIEDE2000 (Spearman 0.954 vs 0.907) on our own 47-pair human-feedback set — rank-order only, see note below.
Accessibility Checking
Euclidean deltaE that's actually perceptually calibrated. OKLab STRESS = 47.35 — not designed for distance prediction.
Research
Transparent pipeline, fully invertible, open source. All parameters, datasets, and optimization scripts published.
Are We the Best?
Color difference measurement: Yes.
MetricSpace v21 achieves the lowest published STRESS on COMBVD, is within 0.8 of the best (CAM16-UCS) on the held-out 128-pair MacAdam set, and ranks our own human-feedback set better than CIEDE2000 (Spearman 0.954 vs 0.907). No other metric matches human perception this accurately across this range of datasets. Caveat: cross-validated estimate is ~24.3 (not the published 22.48), and CIEDE2000 wins on 3 of 6 COMBVD sub-datasets.
Generation tasks: Best-rounded, not best at everything.
GenSpace wins 62-9 vs OKLab across 90 metrics, including 6-1 on independent 3rd-party datasets OKLab was optimized on. However: OKLab is better for near-achromatic gradients (22%), CVD deutan palettes (43%), and native CSS oklch(). CIE Lab's hue angles remain the established industry reference for hue naming.
Overall: First to do both.
Helmlab is the first color space library to achieve state-of-the-art in both perceptual color difference measurement and visual generation quality simultaneously. No other space does both.
How We Test
Deterministic
Every metric is computed at float64 precision with fixed seeds. Run the same code, get the same numbers. No stochastic variation.
Head-to-head
Same test harness, same input colors, same precision for both spaces. Winner is determined by the metric's natural direction (lower or higher is better).
No Cherry-picking
All metrics are reported, including our 9 losses. We do not add or remove metrics based on whether we win them.
Open Source
ColorBench source code, all data files, and checkpoint parameters are publicly available on GitHub for independent verification.
What We Did NOT Test
- HDR color differences — no HDR psychophysical dataset available
- Cross-surround conditions — all data is standard viewing conditions
- Display-specific gamuts — only standard sRGB / Display P3 / Rec.2020 primaries
- Computational performance — not formally benchmarked (JS microbenchmark: GenSpace ≈2.5–2.7× OKLab per color)
- Perceptual ranking with human observers — GenSpace metrics test geometric/mathematical properties, not direct human preference