Skip to content

Consider blazediff for faster pixel-by-pixel and SSIM comparisons #375

@teimurjan

Description

@teimurjan

Hey team! I've been working on performance optimization for visual regression testing and wanted to share some benchmarking results that might be of interest to jest-image-snapshot.

Background

I maintain blazediff, a performance-focused image comparison library that started as a pixelmatch optimization. After seeing some performance concerns in the community (like #28), I thought it might be worth proposing as an alternative backend for jest-image-snapshot.

Performance Optimization

I've benchmarked blazediff against the current backends (pixelmatch and ssim.js) on the HuggingFace Img-Diff dataset (50 real-world image pairs):

Pixel-by-pixel comparison:

  • blazediff/core: ~1.5x faster than pixelmatch
  • Same YIQ perceptual color space and anti-aliasing detection
  • Zero memory allocations during comparison
  • Block-based optimization with 32-bit integer comparison for quick filtering of unchanged regions

SSIM comparison:

  • blazediff/ssim (Gaussian): 3.2x faster than ssim.js Original, 0.9999 correlation with MATLAB reference
  • blazediff/ssim (Hitchhiker's): 8.8x faster than ssim.js Original, 0.987 correlation with MATLAB reference
  • Uses integral images for O(1) window computatio

Why this might matter

From what I've seen in the issues, folks running large test suites (especially in CI/CD) are hitting performance bottlenecks. The speedup compounds quickly:

  • 1000 snapshots taking 30 seconds → 20 seconds (pixel-by-pixel)
    • 1000 identical snapshots taking 30 seconds → 4 seconds (pixel-by-pixel)
  • 1000 snapshots taking 30 seconds → 9-3 seconds (SSIM, depending on variant)

Validation

The implementations have been thoroughly validated using the MATLAB implementations from the original papers:

  • Pixel-by-pixel: Same algorithm as pixelmatch (YIQ + anti-aliasing), just optimized
  • SSIM (Gaussian): 0.9999 correlation with MATLAB reference implementation
  • Hitchhiker's SSIM: 0.987 correlation with MATLAB, validated on 50 real-world image pairs

Full benchmarks and validation methodology: IMGDIFF_ANALYSIS.md

Proposal

Would the maintainers be open to:

  1. Adding blazediff as an optional comparison backend (similar to how pixelmatch and SSIM are currently options)?
  2. Or perhaps just documenting it as a drop-in alternative for users who need the extra performance?

I'm happy to contribute a PR if there's interest. No pressure though—just thought it might help folks dealing with slow test suites!

Repo: https://github.com/teimurjan/blazediff
Docs: https://blazediff.dev/

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions