Additional CanvasPixelFormats and getImageData/putImageData/createImageData

[zbjornson]

I stumbled across this proposal while working on adding support for different pixel formats to node-canvas. I didn't realize there was an effort to add similar functionality to native HTML Canvas. Currently I'm working on supporting all of Cairo's formats (listed here). Some of my comments are influenced by working with Cairo's API; I don't know much about Skia.

---

It would be great if this specification was forward-compatible with other pixel formats. For example, one that is especially important to me is A8 (or R8), which can be used to make indexed PNGs. Things get trickier with the APIs that deal with pixel data (getImageData/putImageData/createImageData), especially with formats that have widths that don't map nicely to an integral type.

• RGB10_A2 (10-10-10-2) would presumably map directly to a Uint32Array, but that's not one of the currently listed ImageDataStorageTypes. Unpacking it to four uint16s would be expensive, and then you have ill-defined behavior when you store a number with more bits than the backing format supports. Likewise:

  • R11F_G11F_B10F -> Uint32
  • RGB16_565 -> Uint16 (This type is of special interest for node.js because it can be used to display canvases on some hardware devices like some TFT LCDs.)

• What's the use-case for "12-12-12-12"? #11 points out that 12-12-12-12 doesn't map to a standard bit width. Likewise: RGB30 and A1; Cairo packs pixels together in these cases, but rows are padded.

• It would be good to address byte ordering (and in the case of A1, bit ordering). The HTML Canvas spec essentially makes ImageData big-endian by specifying the values as bytes in RGBA order. Some of the more esoteric types that pack channels together (e.g. 10-10-10/RGB30) become tedious and slow to flip from native to big-endian. I'm not positive, but if byte ordering isn't normalized, then I think this would be the first web API that gains a platform-specific behavior, however. Since AFAIK Skia doesn't support any of these other types yet, perhaps big-endian ordering would be built-in from the beginning.

[zbjornson]

FYI, experimental support for getContext("2d", {pixelFormat: string}) just landed in node-canvas 2. Might be a useful place to try out implementation details.

https://github.com/Automattic/node-canvas#image-pixel-formats-experimental