Request for TIF support

22

Here are the BlenderGIS temp files for an area of terrain. the srtm tif files are the elevations. I am not sure why it provides two of them. One is 16 bit and one is 32 bit. And then the GOOGLE_SAT_WM is the texture. Files are not necessarily square… whatever your view area shape is when you capture the area from the basemap. don’t worry about wld files those are projection info for the corresponding image.
blendergisfiles.zip (7.5 MB)

1 Like
23

Also, how do you make the castleterrain just display the one texture layer instead of the default 4? When I put the texture in Layer1 I just see it in the low flat areas. Layer count seems hardcoded. … after looking at your screen shot your are setting the texture into all the layers. That seems a little clunky. In the future is there a plan to allow fewer (or more) layers?

24

I can load 16 bit png and it looks good now. But the tifs are still either stairstepping, or displaying corrupt looking results or nothing at all.

25

@edj @eugeneloza Thank you both for the test TIFFs!

Hm, for today (just a quick test before I fall asleep) – I noticed I cannot load TIFF files at all. That is,

… I get 0 size and “Unknown and unsupported format” for all TIFFs. Even simple ones, made from 8-bit RGBA PNGs. Even test TIFF in src/vampyre_imaginglib/src/Demos/Data/Tigers.tif that I know has to be work, because it did in the past!

Scratching my head, looking at Vampyre sources, I just realized that I miss dynamic libraries necessary for TIFF support with FPC. It needs LibTiff 4, see castle-engine/src/vampyre_imaginglib/src/Extensions/LibTiff/LibTiffDynLib.pas at master · castle-engine/castle-engine · GitHub .

I could not find today a matching libtiff on Linux, using latest buntu 24.04 LTS now. It has only libtiff6 package, which seems too new, it is not “picked up” by LibTiffDynLib.pas.

I will get back to this on ~Monday, but naturally if you want to help I’ll appreciate :slight_smile: That is, we want a LibTiff dynamic library that works, on at least

  • Linux (on normal CPU like x86_64) - libtiff.so.5 (the comment in Vampyre sources says this is actually LibTiff 4, despite suffix 5),

  • Windows (on normal CPU like x86_64) - libtiff.dll.

One could also test Delphi – it seems on Delphi, the TIFF is compiled-in statically, so it should work out-of-the-box ( castle-engine/src/vampyre_imaginglib/src/Extensions/LibTiff/ImagingTiffLib.pas at master · castle-engine/castle-engine · GitHub ).

Official site is http://www.libtiff.org/ , LibTIFF - TIFF Library and Utilities — LibTIFF 4.6.0 documentation , hopefully it contains at least Windows DLL that “just works” with Vampyre code. There’s also trusted Tiff for Windows , though it may be too old (version 3.8.2) but worth testing.

As said, I will also get back to this. But just posting now, if you want to experiment in the meantime, all the help is appreciated!

Right now, if you want to see everywhere the same texture, you should just indeed set the same URL 4 times, for all 4 layers.

Yeah, this is indeed clunky, and not how it should be in the long-run :slight_smile:

The terrain future:

  • It is in general outlined in Roadmap | Manual | Castle Game Engine . Also at TODOs Castle Game Engine: CastleTerrain .

  • Tools to modify terrain heights have been started in Base terrain editing tools for Castle Engine Editor by and3md · Pull Request #612 · castle-engine/castle-engine · GitHub . Tools to paint terrain textures (which are equal to “layers”) are not yet started, but they are planned.

  • For layers specifically, the current layers idea from Castle Game Engine: CastleTerrain: Class TCastleTerrain

    Layer 1 is used for flat terrain on lower heights.
    Layer 2 is used for steep terrain on lower heights.
    Layer 3 is used for flat terrain on higher heights.
    Layer 4 is used for steep terrain on higher heights.

    … this will be replaced by just allowing you to draw textures as you like.

    To address this case, it will mean you will be able to just assign whole terrain “layer 1” and thus use a single texture from that layer. You will actually need to “paint” other layers to see different textures in different places, so “everything is layer 1” will actually be the default state, likely. Maybe we will also add a button to “automatically paint 4 layers”, following current algorithm, but then it will be just an option.

    So, the option to “use fewer layers” will naturally emerge, by us not forcing you to really use 4 layers, and allowing you to paint explicitly where is eash layer.

    The option to “use more layers” is another feature, also planned, but technically separate. For some time we may keep the limit to 4 (one splatmap). Maybe bump to 8 (implemented by two splatmaps). Allowing for arbitrary number of layers would be harder (because each new 4 require new splatmap), so I’m unsure about it now, maybe “max 8” will be good. Unity terrains also had it like this (at least last time I used them, ~10 years ago), i.e. you were limited to 8 layers.

Feedback of course welcome :slight_smile:

1 Like
26

Progress: LibTiff on Linux tested and rocks, TGrayscaleFloatImage also on GPU.

Zrzut ekranu z 2024-07-07 22-19-06
Zrzut ekranu z 2024-07-07 22-19-061854×1040 76.4 KB

Details:

  • I changed loading from Vampyre to fail with proper exception in case some format cannot be loaded, e.g. because libtiff is not available. This is a better behavior, consistent also with rest of CGE, better than previous accidental behavior of silently loading an image with zero size.

    We want to avoid experience like @edj mentioned a few posts up – “But the tifs are still either stairstepping, or displaying corrupt looking results or nothing at all.”. If we failed to read the image, there shall be a clear exception.

  • Tested and documented how to get libtiff.so.5 on Linux . Works great on Linux, and yes, I get float-based images from samples in this thread! Please see castle-engine/doc/miscellaneous_notes/additional_image_formats.md at master · castle-engine/castle-engine · GitHub .

  • Caveat: If image contains alpha channel, it is still converted to 8-bit, for today. The reason for this is trivial, we just don’t have TRGBAlphaFloatImage (we only have TRGBFloatImage and TGrayscaleFloatImage). I should add TRGBAlphaFloatImage soon.

    For today, just make sure your image doesn’t have alpha, if you want it to see read to high precision (floats) instead of 8-bit Bytes.

  • I added branch extra-image-formats to GitHub - castle-engine/castle-image-viewer: Image viewer and converter using Castle Game Engine, supporting many common image formats (PNG, JPG...) and some exotic ones (DDS, KTX, RGBE) to demonstrate this, i.e. precompiled libtiff.

  • Note that I thought about making TIFF more out-of-the-box supported in CGE, but resigned for now – this extra need to get libtiff is another hurdle, so reason to not recommend people to use TIFF and instead recommend PNG for now (where we have LibPNG distributed everywhere already). But, if you need TIFF, the ability is there and at least better documented now.

  • We support now loading TGrayscaleFloatImage to GPU (OpenGL), so such texture formats are even more useful – e.g. we can pass float-based heightmaps to shaders.

    (This is not something necessary for usage in TCastleTerrainImage, but it’s nice anyway.)

    You can see proof of it above. Castle Image Viewer displays it as float image, so it’s really not converted to 8-bit at any point in CGE. We can also resize such images (in case they don’t fit in OpenGL max texture size).

1 Like
28

Update:

1 Like
29

I normally use FPC, but for this other project for traffic sim I am using Delphi 11 community. win64 is unimportant for now. What is your paypal address?

30

Update:

  • I have added TRGBAlphaFloatImage.

    This is now the largest “size per pixel” image format in CGE, and able to carry most information: 4x Single per pixel.

    We can read it (if Vampyre will internally read such format), we can also pass it to GPU (so we can display it without any processing on CGE, we can just give such image to OpenGL).

    TODO if you want to help: Turns out, I don’t have test images for this. I thought some of the test images you provided me in this thread (I committed all to test-high-precision-terrains/data/test_tiff at master · castle-engine/test-high-precision-terrains · GitHub ) are RGBA TIFFs with >= 16-bit data, but according to Vampyre – they are not. If you have any RGBA TIFFs with >= 16-bit, I’m interested in them!

    I recommend testing now with:

    • latest CGE,

    • latest castle-image-viewer from extra-image-formats branch. There’s a helpful status text showing what format did we load image into. We want a TIFF that loads to TRGBAlphaFloatImage :slight_smile:

    • and you can uncomment this log to see what format was reported by Vampyre under the hood. We want a TIFF that reports

      • ifA32R32G32B32F,
      • ifA32B32G32R32F,
      • ifA16R16G16B16
      • or ifA16B16G16R16. All these 4 should result in CGE TRGBAlphaFloatImage.

    screenshot-image-viewer-tiff
    screenshot-image-viewer-tiff1564×963 210 KB

  • By the way, because I’m restless :), I also did refactor of castle-image-viewer and reworked the previous “image information” modal dialog into nice persistent status text (toggable with F1, looking similar to Castle Model Viewer status text – deliberately).

    This makes castle-image-viewer nicer :slight_smile: You can browse a directory with images and quickly see their format, sizes etc.

    This is available on both master and extra-image-formats branches of castle-image-viewer.

  • By the way, TGrayscaleFloatImage and TRGBAlphaFloatImage memory formats should also be “saveable” now. That is, we pass full float data to Vampyre, and so Vampyre can save it as e.g. 16-bit PNG or 16/32-bit TIFF. This is untested, testing is welcome, I don’t know whether actually Vampyre can save 16-bit PNG or TIFF.

My remaining TODO in this:

  • Make PNG loaded through LibPNG (which is our default preferred PNG-reading method) also result in float-based formats.

Let me know if you see anything else missing :slight_smile: This is shaping to be a nice new solid feature of CGE!

Thank you!

See Donate to Castle Game Engine | Manual | Castle Game Engine for various donation options (aside from Patreon). In particular if you’d like to donate using PayPal click on this link: PayPal.Me . Or just send to PayPal address [email protected] .

31

The images I provided for elevations would have been 16 bit or 32 bit but just one channel. The image for texture used by blendergis is RGB 8bit. I don’t currently have need for multi byte RGB formats… but many people do so will help you move from just being a ‘game’ engine:)

1 Like
32

Thank you for the donation to CGE!

For now, I consider the feature (well, I turned it into a few inter-connected features :slight_smile: ) finished. I’ll make a news post about it ~tomorrow. Let me know if you miss anything for your actual use-case!

Done today:

  • Reading 16-bit PNG images, when we use LibPng (which is the default situation) also results in float-based formats (TGrayscaleFloatImage, TRGBFloatImage, TRGBAlphaFloatImage – depending on the PNG image).

    This also means that demo in GitHub - castle-engine/test-high-precision-terrains: Test loading terrains from 16-bit and higher precision image formats just works “out of the box”, terrain is precise and smooth both in CGE editor and in running application. No need to use CASTLE_DISABLE_LIBPNG anymore.

    Note that there’s no TGrayscaleAlphaFloatImage, so when input PNG has exactly 2 channels (grayscale and alpha) we expand it to 4 channels (TRGBAlphaFloatImage). I did it to avoid maintaining yet another image class, which I felt would be very seldom used… But maybe I will change my mind eventually. TGrayscaleAlphaFloatImage would be consistent with 8-bit TGrayscaleAlphaImage. For now, we have 3 float-based classes (G, RGB, RGBA).

    Note that saving PNGs using LibPng doesn’t handle float-based images to 16-bit. This is a TODO for the future.

  • I did some fixes to castle-view-image. Both the master, and extra-image-formats branches there:

    • The status text now wraps in case the URL is very long (looks better).

    • The “Images” menu is properly updated when opening the image with directory using “o”.

    • The files loaded from the directory are sorted.

    Overall, this allowed me to quickly check correctness on PNG test suite – PngSuite - the official set of PNG test images . They contain a few 16-bit PNG images. We handle everything in that testsuite perfectly now! :slight_smile:

    png_test_suite
    png_test_suite1760×934 86.7 KB

  • Added one more pretty design “Great Lakes” to demo app, using again textures from Landscape Height Maps - Motion Forge Pictures . GIMP can nicely convert EXR → 16-bit PNG giving me full control over the final PNG format.

    screenshot_great_lakes
    screenshot_great_lakes1854×1040 163 KB

1 Like
33

OK, one last batch of related improvements (I need to make a news about it finally, it’s getting long, maybe too long for one news post):

  • I added TGrayscaleAlphaFloatImage, foreshadowed above. This gives us 4 float-based image formats, consistent with 4 Byte-based image formats.

    We can read such files from 16-bit PNGs, and from KTX.

  • I extended KTX to read all 4 float-based image formats as well. The KTX actually offers the fastest and lossless way to read float-based image formats, because the data is really 32-bit floats inside KTX, there’s no conversion at all (unlike PNG or TIFF where we convert 16-bit ints ↔ floats).

1 Like