About current renderer

(wall of text from someone who self-learning about computer graphics and programming in general. So please correct me if I made any mistakes in the post below :slight_smile: )

One topic I often found on free pascal / lazarus forum was: “Can CGE capable of rendering realistic scene like (insert famous game engine names)?”

First let me congratulation on the new PBR implementation in the engine. This is probably a major milestone for CGE and is a huge step towards realistic rendering!

But PBR alone is not enough. In order to achieve photorealistic, it needs to combine with various effects, in the form of post-processing.

Let’s take a look at what Unity offers in their standard post-processing library:

Some effects on the list, like FXAA or Depth of Field, are easy to implement, without the need to make changes to engine core.

Some other effects, like Bloom, need separate buffer to act as filter, so some changes to ScreenEffects is needed.

But for effects like Screen Space Reflection (which usually found in any photorealistic renderer), it is literally impossible to make it look nice in the current state of CGE renderer. Why? Because CGE renderer does not provide necessary information to make effects like this work perfectly.

These information, which are normal, roughness, and metallic (or shininess, in case of Phong material), are the result of Deferred Rendering technique, which unfortunately not the way CGE renderer works.

CGE renderer adopted Forward Rendering technique, which is the go-to way to render 3D scene since the beginning of computer graphics era, is easy to implement, and work well even on low-end devices like mobile phones.

While Forward Rendering has many advantages, it also has a lot of disadvantages. One big disadvantage was the inability to render a lot of lights in complex scene.

While it technically can, the trade-off in performance was so great made it practically useless for real-time rendering. So another technique was invented to address this issue, namely Deferred Rendering technique.

Not only this new technique was able to solve this particular issue, it also opened the door to many other complex effects that were normally too expensive to implement in a traditional Forward renderer.

Back to CGE, I think we have 2 options:

  • Work on a new renderer alongside with current renderer. This new renderer is either a Deferred renderer, or the more recent, “state-of-the-art” Cluster-Forward renderer.
  • Improve current renderer, borrow some characteristics from Deferred Rendering technique, particularly the g-buffers.

Personality I would go for 2nd option, because:

  • A new renderer is a huge task, we won’t see any results anytime soon.
  • If we can, I would also prefer the new renderer to make use of Vulkan instead of OpenGL, which add more work to the (already huge) task.
  • The current renderer is stable, all it need is to provide more information for post-processing.
  • The changes we need to make to the current renderer is small, basically we create some new buffers, modify the shader pipeline to output additional information to said buffers, and then pass these buffers to ScreenEffects.

Some stuff need to be aware if we follow the 2nd option:

  • Can we have lots of light? Of course we can’t. To keep things simple, lightning calculation process still need to be be done in Forward Rendering way, so we cannot have this feature. In other word, this change only great for post-processing effects.
  • Transparency objects: This is a Deferred rendering issue, I think we can just ignore, not to output information for these objects.
  • Bandwidth optimization: We can try to reduce the size of buffer, for example we can pack normal components to a 24-bit buffer, sacrifice precision for bandwidth and memory.
  • Mobile devices: Since GLES 2.0 doesn’t support render to multiple targets (not count those nVidia extensions), these improvements will not bring any benefits to mobile devices. This does not apply for Nintendo Switch since it has full support for OpenGL.

Thanks for the good words about the engine :slight_smile: From other important features for realistic rendering, image-based lighting will also come hopefully soon.

I agree that our current post-processing features are not really emphasized enough. I would like to have a rich library of standard screen effects. Lots of stuff is possible even within current implementation, as you say. Practically speaking, I think that having a nice library of screen effects, that can be added/tweaked in CGE editor, has even more priority than G-buffer. I’ll try to do it soon, and unblock your https://github.com/castle-engine/castle-engine/pull/181 .

As for deferred rendering:

I agree with your points. Basically, I would like to have “Deferred Rendering” as an alternative to “Forward Rendering”. But not as an “entirely new renderer”, just as “an option of the current renderer”.

  • “Deferred Rendering” should be a variant (sharing as much code as possible) of the current renderer. It should not be a completely new renderer, and it should continue to use OpenGL(ES).

  • I would not mix Vulkan and other APIs into this discussion. While I would very like to have a Vulkan option in the future (and we will almost certainly have this one day), but the choice OpenGL/Vulkan/Metal/etc. should be independent from the choice of forward/deferred rendering.

I agree with your reasoning and approach here. It is better to start with a “minimal modifications to get G-buffer for screen effects” than to try to attack multiple problems at once. Implementing a new renderer is a huge task :), so it’s better to modify existing renderer to offer this option.

P.S. As for the lights limit in forward rendering: Remember that this is only about limit of lights on a single shape. The scene may have thousands of lights. As long as light sources have limited range, and at each particular shape you don’t have too many lights, then there’s no problem.

That’s why I mention it’s not applicable for complex scene :slight_smile: (for example multiple shapes on screen, overlap with each other i.e. midnight forest scenery with fairies cast light around). For scene with shapes that doesn’t overlap with each other much it’s obvious work fine.

I will try to play around with the renderer in my free time to see if I can work on this feature.

1 Like