Comparison of three ETC1 and ETC2 block encoders

Update 9/19: John Brooks at Blue Shift (BSI) and I investigated this on a single image (kodim24.png), and after a bunch of back and forth email debugging I noticed the "effort" parameter the benchmark code passed into etc2comp wasn't in the same range as what BSI's tool passes in. It's not [0,1], it's [0,100]. Looks like that explains its weak performance here. I'm re-running the benchmark tonight at various effort levels, and given what I've seen on kodim24 etc2comp should perform *much* better now. It would be awesome if the etc2comp's compression function parameters were documented here, and I recommend that "effort" be changed to an enum with 10-20 values vs. a float with an odd range:

https://github.com/google/etc2comp/blob/master/EtcLib/Etc/Etc.h

This post shows that compressed texture quality ultimately depends on the library you use to encode your textures. It's possible for a ETC1-only compressor to output textures that actually look better than an ETC2-capable compressor, if that ETC2 compressor's handling of the ETC1 block types is lacking, or if the ETC2 encoder doesn't support all the ETC2 block types, like etcpak. Good software matters here.

This first graph compares all three encoders in ETC1 mode, using RGB (average) PSNR. (See my previous post for Luma PSNR.) Note that these RGB PSNR's are computed the same way as ImageMagick's "compare" tool. etc2comp and etcpak's encoders just aren't very strong in ETC1 mode:


Now the next two graphs are really interesting: I enabled ETC2 support in etc2comp and etcpak (basislib doesn't support ETC2 block encoding yet, just decoding). Notice that, even with ETC2 support enabled, basislib in ETC1 mode is still doing pretty well much of the time. It's kinda cool to watch an ETC1 encoder beat an ETC2 encoder a lot of the time. (I can't wait to implement ETC2 support on top of my ETC1 encoder!)

Notice, at the lower PSNR's, etc2comp shines vs. basislib ETC1. I'm guessing this is where the new ETC2 block types (T, H, and planar) really help. As you go up to higher PSNR's etc2comp gets weaker and weaker compared to basislib's ETC1 encoder, and my guess is it's being held back by its relatively weak support for ETC1. (These are just guesses, I don't have the time to dive in and verify these assertions at the moment.)

I tend to relentlessly optimize my DXT/ETC/etc. block encoders for perceptual error, not RGB error, and maybe this is reflected in this graph.
Anyhow, I think etc2comp is relying too much on the ETC2 block types. So basically, etc2comp's ETC1 encoder (which is usually used by the majority of blocks) isn't taking full advantage of what's possible in ETC1 yet.