How to stitch multiple images together

[Noscow]

Good day!

I want to create a texture from few different images. And I need to create this from code. Can I do this on the CGE level? Or I need to look to the OpenGL functions?
So in steps:

1. Create array with few images
2. Stich this images together
3. Use result as texture on the some primitive

(My goal is to make a modular horizon for a skybox. Different game scenes will use same images with different order and filters to feel more differently and recognizable)

Thanks!

[michalis]

We have functions in CGE to draw “image on image”, so you can create images completely in CGE

1. Advised solution, using GPU acceleration: Use TDrawableImage, with TDrawableImage.DrawFrom method that allows to draw one TDrawableImage into another TDrawableImage. See example examples/images_videos/draw_images_on_gpu/draw_images_on_gpu.dpr .

   This is advised, because:

   • it is fast, as it is GPU-accelerated, it is using underneath OpenGL(ES) things for drawing,
   • it is easily maintainable in CGE, because we can render anything into an image using this approach (even 3D stuff), so we advise this way of image editing.
   • I have a TODO to add more examples to CGE showing usage of this.

   Underneath, the initial image contents are loaded from classes in CastleImages unit, like TRGBAlphaImage. But then, the image content “lives” and is modified only on GPU.

2. There is another approach (not advised): do not use TDrawableImage, instead use CPU-only code for image manipulation in CastleImages unit. Create 2 instances of TRGBAlphaImage and draw one on another using TCastleImage.DrawFrom method.

   • The main advantage of this may be that right now, it may be simpler. It doesn’t need OpenGL context. Maybe there are more examples about using it.

   • But we do want to eventually deprecate TCastleImage.DrawFrom method, and then (but this will not happen anytime soon, so no worries) remove them. Reason: TDrawableImage.DrawFrom method is better in long-run (this way of drawing images is more flexible, faster, easier to maintainer). And there are other libraries that offer image-on-image drawing (FpImage, Vampyre, BGRABitmap) so for cases when TDrawableImage.DrawFrom method is not enough – we can recommend users to use other libraries.

3. I should mention another solution: do not do this

   That is, do you really need to combine images? It can be sometimes avoided, with some benefits (flexibility to do things at run-time) but also drawbacks (preprocessing stuff is, in the end, often a simple optimization – if you can preprocess something, then you don’t do it at runtime). This is just a suggestion, all I’m saying is that I always consider this approach (“just modify my rendering”) before introducing a step to build the textures (using AD 1 or AD 2 above).

   • Maybe instead you can render a few primitives?
   • Maybe instead you can combine the few images in a shader? See e.g. castle-engine/examples/viewport_and_scenes/shader_effects_on_texture at master · castle-engine/castle-engine · GitHub

[Noscow]

Thanks, will experiment.

@michalis

I should mention another solution: do not do this

Maybe I really make this overly difficult.
More detailed, game locations have different types. And, when location is drawing, various sprites of the neighbor locations adds into the 3D Viewport, from all 8 sides (North-NorthEeast-East e.t.c).
This sprites are wide and sometimes they ugly crossing. And I thought make this sprites as a part of the skybox. Cylinder without top and bot is a good surface for this. Just need texture it correctly.

[michalis]

@Noscow

More detailed, game locations have different types. And, when location is drawing, various sprites of the neighbor locations adds into the 3D Viewport, from all 8 sides (North-NorthEeast-East e.t.c).
This sprites are wide and sometimes they ugly crossing. And I thought make this sprites as a part of the skybox. Cylinder without top and bot is a good surface for this. Just need texture it correctly.

Why not just add 8 quads? Like TCastleImageTransform. And see how they look in CGE editor. But I understand that the solution to “ugly crossing” depends on the exact images, so I cannot advise best here. But maybe there’s a solution without complex image composition? Using alpha at the image edges?

Note that placing things on cylinder will “bend” them, so your sprites must either be already bend, or you will need to remap texture coordinates to account for it.

[Noscow]

@michalis

Why not just add 8 quads? Like TCastleImageTransform. And see how they look in CGE editor. But I understand that the solution to “ugly crossing” depends on the exact images, so I cannot advise best here. But maybe there’s a solution without complex image composition? Using alpha at the image edges?

I made few variants with square sprites, but wasn’t satisfied enough yet
But alpha is a good idea… I can make sprite with gradient alpha and blue-grey background. Maybe this will create a fog or mist effect and hide sprites overlaps. And will save much time

[Noscow]

@michalis

Advised solution, using GPU acceleration: Use TDrawableImage, with TDrawableImage.DrawFrom method that allows to draw one TDrawableImage into another TDrawableImage.

I tryed this method, but now I need add TDrawableImage at TCastleImageTransform. Or, maybe, draw it on Plane as texture?

As result I want 2D sprite with options for comfortable control from the code. But I can’t find TCastleImageTransform method for adding TDrawableImage. What can you recommend?

[kagamma]

I don’t think TCastleImageTransform exposes any property that helps you achieve that (should be easy to add, maybe ping michalis for it ) In the mean time you need to create a custom TCastleScene yourself:

• Check the example castle-engine\examples\viewport_and_scenes\cars_demo\code\gameviewmain.pas at line 86 (function CreateAdditionalMesh), which create a new TCastleScene with a texture. You can play with it and replace the BaseTexture.SetUrl(['castle-data:/textures/tunnel_road.jpg']); with BaseTexture.LoadFromImage, which take TDrawableImage.Image as one of it’s parameters.

References:

• Castle Game Engine: X3DNodes: Class TImageTextureNode
• Castle Game Engine: CastleGLImages: Class TDrawableImage

[Noscow]

Pipeline is a little difficult, but thanks

For now I trying with 2D Viewport.
I make big images from tiles with overlapping. And, if i use many TCastleImageTransforms with same Translation.Z, I receive artifacts.

For now I have simple way to fix it. I add variable that grows at 0.001 every time when new TCastleImageTransform is added. And use this variable as NewImage.Translation.Z.

It isn’t seems elegant, but work. And, If one-two thousands of TCastleImageTransform make same PC load as ten big pictures, I can continue use that method, I think… Maybe It also gift some benefits in future, who knows (definitely not me)…

[michalis]

@kagamma

I don’t think TCastleImageTransform exposes any property that helps you achieve that (should be easy to add, maybe ping michalis for it )

I hear you both, done

That is:

• I added TCastleImageTransform.LoadFromImage to load image contents from TEncodedImage (like TCastleImage).

  Docs will soon be in API docs, for now take a look at comments.

• I also added some helpers planned for a long time, as we want to encourage generating images using TDrawableImage.

  • overloaded TDrawableImage constructor that takes only sizes, see comment with some example usage

  • trivial helper TDrawableImage.FloatRect

• And finally I added an example showing everything: examples/images_videos/image_generate_and_use.

  It’s a bit silly (doesn’t do anything useful), but it shows the point We draw on TDrawableImage (other images, primitives, text) using our advised API (GPU accelerated). Then we use the result on TCastleImageTransform. Of course all TCastleImageTransform parameters are honored, e.g. it does “repeat” in an optimal fashion (but just drawing one big quad with proper texture coordinates).

@kagamma

which take TDrawableImage.Image as one of it’s parameters.

This bit is not true, I’m afraid: Note that if you have modified contents by drawing on GPU (like using TDrawableImage.DrawFrom), then the TDrawableImage.Image will not contain the up-to-date image anymore (it will only contain the last “known” image contents in regular memory, RAM). You need to get the image contents from GPU to RAM using TDrawableImage.GetContents .

My example referenced above shows this.

Future plans to further improve this: In the future,

• It would be nice to allow loading TDrawableImage (without the need for dealing with TEncodedImage / TCastleImage at all) straight into TCastleImageTransform (and the underlying TImageTextureNode). This would make things super-fast, as the contents would not leave the GPU at all.

  That’s already an existing TODO, I need to see how to do this without causing too much chaos :), and with some callbacks to “recover” when GL context is lost (which can happen at any time on mobile or web).

• We need more uniform way of providing textures, so that this mechanism could also be used to provide image as texture for TCastlePlane, TCastleBox etc. New material components ( Roadmap | Manual | Castle Game Engine ) should enable this.

[Noscow]

Great, thanks! I start testing, works nice
Basic TDrawableImage creates without transparency? In you example:
Result := TDrawableImage.Create(1024, 1024, TRGBImage, true);

I want to make first TDrawableImage, what I will use as canvas for others, fully transparent. But in result that filled by color.

Alpha don’t adds when I create canvas TDrawableImage, or that looses later, when I load it at Transform ?

Canv := TDrawableImage.Create(1024, 1024, TRGBImage, true);
// ...
// ADD IMAGES
// ...
FinalImage := Canv.GetContents(TRGBImage);
LoadFromImage(FinalImage, True);

[michalis]

@Noscow

Basic TDrawableImage creates without transparency? In you example:
Result := TDrawableImage.Create(1024, 1024, TRGBImage, true);

See the TDrawableImage.Create docs – the given class like TRGBImage determines the storage on GPU. Simply pass TRGBAlphaImage (to TDrawableImage.Create) to have alpha during drawing.

Later use GetContents(TRGBAlphaImage) to get the final alpha values, if they matter for you (for rendering the image through TCastleImageTransform).

(Sorry for brevity, answering from phone:)

[Noscow]

@michalis

(Sorry for brevity, answering from phone:)

Thanks, the answer was exhaustive, all works perfect