Request for TIF support

[edj]

It would be very handy if CGE could read TIF files, specifically 16bit greyscale height maps, like geotiff. I am trying to figure out how to convert to png and not have it drop to 8bit. But not having to have that be a step in the process would be very nice.

[edj]

In the meantime ImageMagick can convert 16bit tif to 16bit png from the commandline… and is open and in pascal! So that will work.

[michalis]

Indeed, ImageMagick can do a lot of cool stuff with images, and it’s capabilities to convert images from/to everything are unparalled in my experience I mostly use it as command-line tool, but they also feature a library that can be used from Pascal (and many other languages).

Note:

1. Note that ImageMagick itself is not written in Pascal. It’s mostly in C, from what I know, GitHub - ImageMagick/ImageMagick: 🧙‍♂️ ImageMagick 7 confirms it. Debian’s build dependencies of ImageMagick also don’t include FPC, so it would be strange if ImageMagick was written in Pascal.

   The FPC includes Pascal units ImageMagick and Magick_Wand, documented on PascalMagick - Lazarus wiki . But these units are just “wrappers” – the functions they expose just call the underlying ImageMagick C library.

   There’s nothing wrong with that of course, ImageMagick is great. I’m using lots of software not written in Pascal, and CGE also doesn’t limit itself to only using libraries written in Pascal (as seen in Compiling from source | Manual | Castle Game Engine ). I absolutely recommend you to use ImageMagick, both as command-line tool, and as library, from any language including Pascal. It’s great open-source code. I just mention it to clarify – saying “ImageMagick is in Pascal” is not true :), only wrapper units are in Pascal.

2. Note that CGE doesn’t preserve accuracy in 16-bit PNG now. Reading such PNG images goes under the hood through our regular 8-bit image classes (like TRGBAlphaImages, 32-bit per pixel, 8-bits per channel).

   We don’t even have a class internally in CGE to hold exactly 16-bit channel information. Though we have TRGBFloatImage, where every pixel is 3x Single (3x 32-bit floats) values. This is suitable for cases when you need higher precision than 8-bit per channel (and/or also higher range than 0…1, which makes sense for ray-tracers where output brightness is basically unbounded since in reality there’s no upper bound on how bright things can be). Of course it’s floating-point precision, not integer, but it’s still higher precision than 8-bit per channel in practical usage.

   Image formats like RGBE are read to TRGBFloatImage (right now this actually happens in Vampyre Imaging Library that we use in CGE; in the past we did RGBE read/write in CGE natively). KTX also can be read as TRGBFloatImage when the image format indicates floats. We also have OpenGL(ES) code to actually load such float-based data into OpenGL(ES), so it stays floats on GPU.

   It’s great to read / write float image data in our simple ray-tracer CastleRayTracer. It also makes sense to use in some rendering tricks that really require more precision / higher range (we use it in our “Variance Shadow Maps” implementation).

   However we don’t read PNG to TRGBFloatImage right now internally. (Forcing it, by MyImage := LoadImage('my16bit.png', [TRGBFloatImage]) will work but just mean we read it to 8-bit precision and then convert each Byte to Single. So it works, but you don’t preserve 16-bit precision.)

   If you have a need for 16-bit precision image format in CGE, let me know, and I can then advise how to proceed

[michalis]

Oh, and to answer the thread title:

Nowadays we delegate most of our image reading/writing code to Vampyre Imaging Library, see

• More integration with Vampyre Imaging Library, used by default with both FPC and Delphi – Castle Game Engine
• Integration with Vampyre Imaging Library, new example image_display to test image loading speed and format support – Castle Game Engine

I’m very happy about it, it means we “delegated” maintaining of a lot of source code (that we previously had in CGE) to another library. We gained some features, and we gained “easier maintenance” in CGE due to smaller code in CGE → super win

The only image formats we really read/write natively in CGE are now KTX and DDS ( KTX (Khronos Texture Format) | Castle Game Engine , DDS (DirectDraw Surface) Image Format | Castle Game Engine ), these have quite unique gamedev features so we’ll likely keep them implemented in CGE, not delegating to another library. We also read PNG though libpng, for maximum performance of this format used for most images in CGE in practice. For most of other image formats, we use Vampyre Imaging Library.

…And it actually already features TIFF support

It is however not enabled by default in CGE, because it is not cross-platform. See castle-engine/src/vampyre_imaginglib/src/Extensions/ImagingExtFileFormats.pas at master · castle-engine/castle-engine · GitHub for details. Basically, we want to avoid the situation when someone develops CGE game on Linux/Windows, using TIFF, and then tries to recompile to e.g. Android/iOS… only to find that TIFF is not supported on these platforms, and the data has to be converted to e.g. PNG. That’s why we prefer to not enable by default image formats that are not available on all CGE supported platforms.

Just compile your project with CASTLE_ENABLE_PLATFORM_SPECIFIC_IMAGE_FORMATS symbol defined (specify it in CastleEngineManifest.xml compilation options, CastleEngineManifest.xml - Syntax and Examples | Manual | Castle Game Engine , if you build using CGE editor or build tool) to have TIFF support.

[edj]

Wow, that is great news! Thanks.

[edj]

Ok, I can get it to read a tif, but now I am unsure if tif or png is maintaining the data at 16bit? The elevations look really ‘stepped’ instead of showing more smoothness. But it is possible that is the nature of the data (srtm 1-arc second… 3601x3601 16 bit height map… here on castleterrain with 360 subdivisions). Does the TCastleTerrainImage retain the 16bitness of the data? (this is data for rockymountains around me… should be 100s levels of elevations instead of what looks like around 10 levels).

[michalis]

@edj

Does the TCastleTerrainImage retain the 16bitness of the data?

The image reading in CGE does not retain the 16bitness of the data. That was my point in Request for TIF support - #3 by michalis (though I see I expressed it in somewhat lengthy and convoluted way yesterday, sorry ). Whether you use PNG or TIFF doesn’t matter. 16-bit data will get loaded to 8-bit-per-channel format.

The TCastleTerrainImage by itself is happily using floats, but it doesn’t matter at that point, the image data already lost precision beyond 8 bits.

It’s a very valid feature request to add this support.

• For TIFF, we would rely on underlying Vampyre Imaging Library supports reading 16-bit OK. It may, I see ImagingTiffLib defines

  const
    TiffSupportedFormats: TImageFormats = [ifIndex8, ifGray8, ifA8Gray8,
      ifGray16, ifA16Gray16, ifGray32, ifR8G8B8, ifA8R8G8B8, ifR16G16B16,
      ifA16R16G16B16, ifR32F, ifA32R32G32B32F, ifR16F, ifA16R16G16B16F, ifBinary];
  

  … so it includes 16-bit formats.

• For PNG, we would rely on LibPNG support, which for sure also supports it (as the most standard PNG-reading library in the world, it definitely supports 100% of PNG features).

I would most likely introduce a global flag like var Load16BitImagesToFloats: Boolean = false and you would just have to set it to true to load them to TRGBFloatImage by default. To clarify (to limit my initial implementation to your exact needs):

• Are you fine with only RGB support? For heights, actually TGrayscaleFloatImage may be even more optimal. So, do you plan to use it only for heights, and TGrayscaleFloatImage or TRGBFloatImage would be good enough?

  Note that it means we cannot express alpha channel as floats, for now. Images with alpha would have to go for TRGBAlphaImage, until I gather strength to introduce also TRGBAlphaFloatImage

• Do you have any reasons for prefer TIFF over PNG, in light of the above? It seems like I could implement it for both TIFF and PNG, but still you can influence my priorities My personal preference would be to handle 16-bit PNG over 16-bit TIFF, due to fact that this will be cross-platform, i.e. mobile games will then also benefit from 16-bit PNG → can be read as floats.

I will take a look at it soon (~next week), if it takes me too long please ping me and/or create a GitHub request

[edj]

These would only for heightmaps, so RGB support and transparency are not needed. TIF is handy because it is a native format you get from EarthExplorer, but png is ok since I can convert (though the conversion seems to convert it from signed to unsigned data or vice versa). I am not concerned about cross platform. But I leave it to you to prioritize, I can proceed either way. I believe the data is either a signed or unsigned 16bit word that provides the elevation as a whole meter. This will really open the doors for gis use and real world data. Thanks.

[edj]

I now definitely prefer TIF priority. The BlenderGIS plugin for Blender uses tif files for terrain and texture map. I should be able to take its tif files directly from its temp folder, so able to build textured terrain in cge from anywhere you are viewing with BlenderGIS. It looks like the terrain tif created by BlenderGIS can be 32bit grayscale if that matters for your efforts.

[edj]

Since someone is paying me for this, I offer a $100 bounty for the 16bit and 32bit tif support for terrain height if ready some time in July.

[michalis]

Sorry for delay, I wanted to do it last week but failed. I have on TODO for next week.

That said, if anyone else wants to grab the bounty, speak up

The Vampyre Imaging Library, that we use, seem to support both 16-bit and 32-bit versions, so it should be possible to implement it easily:

const
  TiffSupportedFormats: TImageFormats = [ifIndex8, ifGray8, ifA8Gray8,
    ifGray16, ifA16Gray16, ifGray32, ifR8G8B8, ifA8R8G8B8, ifR16G16B16,
    ifA16R16G16B16, ifR32F, ifA32R32G32B32F, ifR16F, ifA16R16G16B16F, ifBinary];

Note that in the end, the heights for TCastleTerrain are floats (that is, Single in Pascal, 32-bit float) anyway. We pass them as such to GPU (which then does most calculations using half-floats, 16-bit floats, actually). So they are not exactly 16-bit integer or 32-bit integer.

For this reason, I also don’t plan to introduce new TCastleImage descendants that hold 16-bit or 32-bit data (like TCastleGrayscale16Image or TCastleGrayscale32Image). Instead, I want to introduce TCastleGrayscaleFloat and convert both 16-bit and 32-bit TIFF to it. As the TCastleGrayscaleFloat seems useful for other future general purposes (e.g. heights are not limited to 0…1 then, and e.g. OpenEXR could utilize it – OpenEXR can hold 16-bit floats).

Do let me know if you already know the above approach will be not enough for some purposes. We could brainstorm other solutions then, though things get harden then (since our processing, and passing data to GPU, is generally using Single, not Double, not 32-bit integers). So by default, my course of actions is to do TCastleGrayscaleFloat and keep things simple

[edj]

Converting them to Single or any type that maintains the height detail without losing levels like it does now is fine for terrain detail.

[edj]

Is 16bit float something that can be used in pascal directly somehow? It would be a good type for my water depth.

[michalis]

@edj

Is 16bit float something that can be used in pascal directly somehow?

Not “natively” (by having a type like HalfFloat with regular Pascal operations performed efficiently on CPU also using the 16-bit float type).

See Half-precision floating-point format - Wikipedia .

While modern CPUs support this type since ~2012 (looking at the above Wikipedia page, " Support for half precision in the x86 … fairly broadly adopted by AMD and Intel CPUs by 2012")…

… But almost no general programming language supports them, with the exception of latest Zig and Swift (again, reading that Wikipedia page). According to this stackoverflow post you can also find them in C / C++ with modern GCC . All in all, Pascal is not alone in lacking support for half-floats

I’m guessing the use-case to maintain this type in a compiler is too small. This type really mostly matters when you need to transfer/process a gigantic amounts of floats, making half-floats transfer 2x smaller. So it is mostly a big deal for GPUs, not so much for regular languages that compile to code on normal CPUs. Even if Pascal supported it, I’m not sure would we use it widely in CGE, though I would definitely make a test (/s/Single/HalfFloat/ in CGE code) to check it

However! You can find libraries that offer half-float support. Either “in software” (by decoding + processing + encoding the half-float for all operations) or by implementing necessary operations in assembly (and thus relying only on assembler support for half-floats). Note that the “in software” version will be necessarily slow, though useful if you need to store half-floats (e.g. to pass to GPU). Quick search found this:

• half: Half-precision floating-point library

• GitHub - ROCm/half

• oh, and the same stackoverflow post points to more: c++ - Why is there no 2-byte float and does an implementation already exist? - Stack Overflow

You may likely be able to convert some of these libraries (either by converting code to Pascal, or by linking to these libraries and using a Pascal header to expose their functionality).

All in all. in short: if you need to process floats with reasonable speed with Pascal, stay with Single, that’s probably a short version of the above post

[eugeneloza]

I’ve seen recently a discussion at Lazarus forum about that Half precision floating point - wish list ---- Unfortunately I’m not smart enough to tell if they’ve made it work on a bigger scale, but at least on some systems it seems like it was tested and worked.

[michalis]

Sorry again for delay, but I want to finally do this tomorrow (Friday). I got test data from Landscape Height Maps - Motion Forge Pictures , nice big PNG and OpenEXR images with 16-bit data. Using ImageMagick I also converted them to 16-bit TIFF, so I have all test data I need.

As expected, when I load them today, with heights precision beyond 8-bit lost, I can clearly see the layers – because I sample lots of points in XZ, and they all result is the same Y around. This is what will be solved by reading 16-bit PNG/TIFF → TGrayscaleFloatImage.

[edj]

Yes, that stair stepping is what we want to avoid. Thanks!

[michalis]

The most important piece of work done, in commit Support high-precision terrain image data · castle-engine/castle-engine@2063281 · GitHub !

• We have TGrayscaleFloatImage
• TCastleTerrainImage uses TGrayscaleFloatImage
• We read various more formats from Vampyre Imaging to TGrayscaleFloatImage or TRGBFloatImage, such that float/16/32 data gets converted to Pascal Single not Byte, so not losing precision

Here are 2 new images from the same example application I shown in Request for TIF support - #17 by michalis . In that post, it showed the problem (“stairs”). Now, it is nicely smooth Below you can see one image with just heights, one image with additional diffuse texture applied ( Landscape Height Maps - Motion Forge Pictures has such nice textures ).

This example uses PNG loading through Vampyre (to make it happen, compile CASTLE_DISABLE_LIBPNG, e.g. by placing it at top of castleconf.inc).

Other than that, the example just uses TCastleTerrain + TCastleTerrainImage in the most trivial way, no special tricks

In the end, I did not make any boolean like Load16BitImagesToFloats: Boolean to activate this behavior. It is just default. Let’s see if this causes any issues, hopefully not – and people who have float/16/32 images will enjoy them being loaded to float-based formats by default.

Still TODOs:

• I need to find / make TIFF images to test it. My 16-bit TIFF I made yesterday – it seems Vampyre cannot read it. @edj Do you have test TIFF images you can share publicly? Or you can just test yourself The new code should support them. You just need to enable TIFF support as before, by compiling everything with CASTLE_ENABLE_PLATFORM_SPECIFIC_IMAGE_FORMATS.

• Make PNG loaded through LibPNG (which is our default preferred PNG-reading method) also result in float-based formats. This will eliminate the need to use CASTLE_DISABLE_LIBPNG for this.

[eugeneloza]

I guess this might be a good test image Gebco Heightmap 21600x10800 - Topographical Earth Tutorials

[edj]

Beautiful. Here is a 16bit geotif. testgeotif.zip (13.3 MB)
If you install blendergis it can make geotifs of your view and seems to use both 16 and 32 bit. I will post some of those too. After breakfast.