Texture Compression Tools - integrate in CGE

Creating data (models, textures...)
1

Let’s talk about integrating more 3rd-party texture compression tools in CGE.

The goal is to

  • make some necessary tools for Auto-generated compressed and scaled textures | Creating Game Data | Castle Game Engine be available out-of-the-box in CGE, and in turn allow to use texture compression more easily in CGE.

  • This can also provoke progress in how you use these textures in CGE – right now you have to manually craft XML file material_properties.xml and then run castle-engine auto-generate-textures. If we will use this more, we can find better ways to configure it, maybe just visually design in CGE editor information about which textures are compressed and how.

@eugeneloza already did the important first step in Third party tools by eugeneloza · Pull Request #351 · castle-engine/castle-engine · GitHub (integrating ASTCENC), here I want to talk about integrating more tools.

NVIDIA:

Looking at it, it seems NVidia doesn’t really want version 3 to be open-source.

We don’t want to go into the legal maze of integrating with NVTT 3, esp. as we only need it to DXT*, and we can get it through other software too, including Compressonator.

Compressonator:

This tool seems “worth fighting for”, as it is properly open-source ( compressonator/license at master · GPUOpen-Tools/compressonator · GitHub , with core being just trivial license on compressonator/frameworklicense.txt at master · GPUOpen-Tools/compressonator · GitHub ). And it is even recommended by the nvtt2 README on GitHub. And it supports a lot of output encodings, including ASTC, ATC, DXT*, ETC, ETC2.

Sidenote: AMD seems to be doing a lot of cool open-source things these days, I have a number of tools from https://gpuopen.com/ already on TODO list to play with :slight_smile:

As for size (as Eugene points in Third party tools by eugeneloza · Pull Request #351 · castle-engine/castle-engine · GitHub , is is 150 MB – maybe a bit too large for an optional tool in CGE):

The compressed release is 68 MB ( Releases · GPUOpen-Tools/compressonator · GitHub , Linux version) although it seems that there are parts/core that are smaller.

Looking into the unpacked fles,

compressonatorcli_linux_x86_64_4.2.5150$ du -hs *
4,0K	compressonatorcli
8,5M	compressonatorcli-bin
9,5M	documents-unused # ok, this was obvious to remove
12M	images-unused # ok, this was obvious to remove
8,0K	license
33M	pkglibs
104M	qt

So most of the size is in the qt subdirectory. Can we maybe avoid packaging some of these libs? I tried to just remove qt but I see that binary then fail.

My suggested way forward (to @eugeneloza or anyone else that wants to help, speak up here):

  1. Look into Compressonator more. Can we somehow take only a subset of it, reducing the size?

  2. Look into other software mentioned on GitHub - castano/nvidia-texture-tools: Texture processing tools with support for Direct3D 10 and 11 formats. as “If you are looking for alternative texture compression tools and processing algorithms I recommend you check these out:” . As a basic requirement, we need a good-quality open-source compressor for DXT* (as the most popular and established GPU compression methods on desktops now).

And we can talk about it in this thread.

I do not reject the possibility of “just including Compressonator in CGE distribution”. We can do it, and live with extra +68 MB zip size, for a good open-source tool.

1 Like
2

Just adding a bit here - I recall some long time ago (like 8 years…?) I was looking at different tools to compress into DDS format and all the reviews I read noted that NVidia Texture Tool was superior to alternatives. Noting that ASTCENC is far superior for compression into ASTC format than both NVidia Texture Tool 3 and PVToolsCLI (and Compressonator? - if it can at all) + one of the latest release notes in ASTCENC was “increase compression quality” - I strongly believe that there are different algorithms for DXT* compression too and thus looking up a new quality comparison research may be a good idea.

Even if the third-party tool may be not FOSS-compatible we still may recommend to use it if it provides a superior quality, just not package it with CGE. However there is an issue here - that we first of all look at tools packaged with current CGE release. Maybe we should prioritize the “current installed version” in the system or better have a parameter to (as user’s installation may be broken).

3

Ideally, we could use something that has good quality, is open-source, and we bundle it in CGE. Then we can search for it first – this way we use the tool that we have tested with CGE.

But as an alternative (if no open-source tool will exist with good quality output, but some proprietary tool will exist) we can indeed do as you write – first search for that “other” tool, and only fallback on a tool “bundled with CGE”.

4

What I’m afraid of here is that the locally installed version is bad in some way (e.g. the installation is broken or the version is not compatible with CGE). So for a better user experience I’d say we should by default prioritize the version bundled with CGE. → Otherwise we may get overrun with bugreports like “texture compression is broken” which would take a long time to figure out that the user has had an absurdly obsolete Compressonator installation and has completely forgotten about it.

But optionally, the user may say “But I want to use Nvidia Texture Tool version 3, not 2” - and this should be possible. “Should” doesn’t mean “have to” - e.g. Unity doesn’t seem to allow that in any simple way.