Float-based images (e.g. for precise terrains) full support: 4 float-based image classes (passed to GPU and shaders), loading 16-bit PNG, loading float-based KTX, loading 16/32-bit TIFFs, Castle Image Viewer improvements

[michalis]

We added to Castle Game Engine capability to read 16-bit and higher precision image formats. This is useful for precise terrains (where you want to store heightmaps with more precision than 8-bit) or any other data that you may want to process (in shaders or using Pascal code) where 0..255 range (8-bit integer, i.e. byte) is not enough.

Overall, this is a collection of inter-connected features and developments. We hope you enjoy our long news announcements :), here goes:

1. First of all, grab the demo test-high-precision-terrains: Test loading terrains from 16-bit and higher precision image formats. It’s in a separate repository (not in CGE main repo) because it necessarily has quite large data files.

   Just explore it, in editor and also run it, to see what this all means.

2. We have implemented a full set of 4 float-based image formats:

   1. TGrayscaleFloatImage,
   2. TGrayscaleAlphaFloatImage,
   3. TRGBFloatImage,
   4. TRGBAlphaFloatImage.

   All these float-based image formats are now uploaded to GPU as float textures, thus can be used to process this data on GPU in shaders. That is, we don’t “lose” the extra precision anywhere on the way from disk to the GPU.

   For example, new Castle Image Viewer loads and displays the 16-bit PNGs, or float-based KTX, or 16/32-bit TIFFs, as float images. You can confirm it by loading such image and observing that Castle Image Viewer status bar shows it has a TXxxFloatImage class.

3. 16-bit PNG images can be loaded to 4 float-based image formats mentioned above. (Previously, they have been loaded to 8-bit image formats, losing precision.)

   There’s nothing you need to do to enable it. The 16-bit PNGs are just loaded to new image formats now, and everything should just work better.

   This happens both when PNG images are read using LibPng (our default PNG handling, giving us great PNG loading performance) and Vampyre Imaging Library (as a fallback).

   In both cases, we now read all images from PNG test suite perfectly, without any precision lost. We have included some images from that test suite into CGE auto-tests to ensure it stays this way.

4. Float-based KTX images can be loaded to 4 float-based image formats mentioned above. This is great if you really need max precision. KTX contains float values, we read float values, there’s no conversion at all — everything is simple, fast, and using floats.

   Note: throughout this announcement, “float” means the 32-bit floating point type called float in C and Single in Pascal. All image libraries, including Vampyre and CGE, call it “float”. Do not confuse this with Pascal Float type defined in the Math unit, which is the most precise floating-point format and not that useful for cross-platform applications since it means something different on different platforms.

   I found test KTX float images in KTX software, more precisely inside the astc-encoder tests (look in HDR-RGB and HDR-RGBA subdirs for suffix 32.ktx).

   These test images are also included now in CGE auto-tests (tests/data/ktx/floats) to ensure they are loaded correctly forever.

5. Likewise, 16/32-bit TIFF images can be loaded to float-based image formats mentioned above.

   Note that TIFF reading capability is not enabled by default in CGE. You have to enable TIFF support in your application and distribute the TIFF library (libtiff) with your application. We have documented it better now:

   • Read details about TIFF and JPEG 2000 support – why it is disabled by default, and how to enable it. This document also describes how to build and use libtiff.so.5 necessary to make it work on Linux, and what happens on other platforms.

   • Castle Image Viewer branch extra-image-formats is a working example doing it. You can compile this to get castle-image-viewer with TIFF and JPEG 2000 support. It includes the libtiff.so.5 precompiled, ready to use.

   Test TIFFs are in the test-high-precision-terrains demo mentioned above, see data/test_tiff.

6. TCastleTerrainImage uses TGrayscaleFloatImage to read terrain data from images. This means that it always operates on float precision, and you will enjoy extra precision if you image contains it — that is, if you use 16-bit PNG, or float-based KTX, or 16/32-bit TIFF.

7. Our Castle Image Viewer (test the snapshot version) also got some improvements along the way:

   • Refactored old long code into a few nice units.

   • Previous “Image information” modal dialog is removed…

   • … Because we have a new status text. Toggable with F1. Looking similar to Castle Model Viewer status text — deliberately. Displays lots of useful image information.

   • The “Images” menu is properly updated when opening the image with directory using “o”.

   • The files loaded from the directory are sorted.

   Overall, you can now browse a directory with images and quickly see their format, sizes etc. This is available on both master and extra-image-formats branches of castle-image-viewer.

   For quickly checking a lot of file formats/sizes, remember you can also use CGE existing utility image_identify. Great to check from command-line a number of images. Combine with Unix find to scan a whole directory, recursively, and output image information. Inspired by ImageMagick’s identify.

8. Note: saving float-based images is a TODO. It is documented in the roadmap. As always, ping me if you need it.

Credits

Big thanks go to Erik Johnson for supporting Castle Game Engine development, both on Patreon and by actually using the engine to develop new projects, and providing a ton of useful feedback on our forum! See the forum thread that initiated this.

The sample data has been provided by:

• Landscape Height Maps – great detailed terrains, including 16-bit PNG images.

• Topographical Earth Tutorials for the big 16-bit TIFF image.

Engine features described in this post done by Michalis — thanks to your support.