What is the overhead in the engine of dynamically adding and removing TCastleScenes?

In a hypothetical system which implements a large world by logically dividing it into cells - what approach to dynamically adding and removing TCastleScene’s would minimise variation in the frame rate? (I do appreciate this will probably require answering by experimentation - but someone may already have experience, or know a best approach :slight_smile:

If such a system were to use a combination of occlusion query, distance from viewer etc, to decide which parts of the terrain to dispose of and which parts to load, then would it be better from the engine perspective if Scenes were added in batches - or restricting to just one per frame? I.e. Does each add of a scene incur overhead which might be more efficient if they are done in batches? Or would it be smoother to slowly add them - possibly obscured in fog in the distance?

The data could come from different sources: procedurally generated locally, loaded from local files, or loaded from internet server. I’m assuming there are no problems with getting the data by either of these methods in other threads - but that the actual creation of the TCastleScenes and inclusion into the scene must be done in the main thread? Or, is it possible to construct each TCastleScene in the other threads - but then only insert them in the scene in the main thread?

I’m imagining that a thread safe queue would be required so that “producers” of the data can push into the queue, and then the main thread can pop from the queue and insert the TCastleScene’s…

But at what point can the TCastleScene’s actually be constructed? Can they be constructed in other threads and then inserted into the queue so that the main thread only has to retrieve them?

Or (as I suspect), must another representation of the data be inserted in the queue - and then the main thread must retrieve this and construct each TCastleScene before inserting in the view?

All the CGE operations must be done from the same thread, see Threads usage | Manual | Castle Game Engine . This is similar to e.g. Unity or Lazarus LCL.

So if you want to utilize threads, you have to operate only on your own structures in threads, and, as you describe, the main thread has to “pop” them from some queue.

I want to add asynchronous loading capability to CGE, see my plans about demo of “huge city” on Slides, movies and thoughts from my GIC 2022 presentation – Castle Game Engine . But this is not done yet. Also, the asynchronous API will still mean you always call CGE from a single thread – it will just be able to load and initialize as much as possible resources in a thread, but that will be hidden from you. (The reason is that coding using threads is notoriously hard for users, and allowing parts of API to be used from multiple threads is also hard for engine development – so we prefer a more limited approach.)

As for the overhead of adding scenes: it very depends on what the scene contains, so indeed some experimentation will be necessary. In general, the scene can reuse e.g. shaders from other scenes, this happens automatically under the hood. Since adding the scenes is synchronous for now, I would indeed recommend to add them “slowly”, not a big number of scenes at once.

Thank you, Michalis. I think I’m on the right track then :slight_smile: