Continuing on my terrain client/server, I was finding that if 1000s of tiles are sent all at once to the client, it would choke up as those loaded (amazingly fast when the data is already generated on the server). The client had been reading the data in its client thread and then passing that to build into graphics on the main thread, the same way it handles strings.
As an experiment I thought would be dangerous from your warnings about CGE and threads… I build the graphics in the client thread (A TCastleScene that is an indexed triangle set) and pass the built TCastleScene to the main thread where the old one is removed and the new one is added to the Viewport.items. I have been unable to produce any unwanted behavior. And now the tiles dump to the scene without even causing a hiccup in the framerate. Like 10,000 variable resolution tiles in second. Am I playing with fire? Or is this usecase of threads safe with cge?
The image below shows variable resolution tiles. I now use 120 as a tile’s internal base grid size as it is divisible by more numbers than 100 so reduces resolution more smoothly. I still haven’t fixed the seams because it makes the tile grid easy to see.
To avoid thread problems, if any, you can use a method like:
You have a protected variable. For example, in Delphi, there is TMultiReadExclusiveWriteSynchronizer.
To protect a variable that is used by multiple threads,
when data is received, it is stored in the variable protected by TMultiReadExclusiveWriteSynchronizer.
On the other hand, CGE components read this protected variable in the OnUpdate event and update the interface.
Any access (read, write) to CGE components from multiple threads just works by accident. So yes, I’m afraid you “play with fire”
As long as we don’t access any cache (shared by other scenes in other threads) the code may work, but you really should not do this. See Threads | Manual | Castle Game Engine - “All access to the Castle Game Engine API must happen from one and the same thread.”.
You can of course use other threads to build data. Things like TVector3 or TVector3List are thread-safe. But then
to pass this data to CGE components, like TCastleScene or X3D nodes, you really need to be always in the one, the same (usually called “main”) thread, otherwise undefined things may happen.
It is unfortunately easy with threads to get a code that works in a test environment, but then crashes in production usage when the code in other threads happens to touch the shared memory by doing other intensive work, without the necessary synchronization.
It is also possible that something works now, but will be broken by future engine versions – we are explicitly not thread-safe, we don’t test or think about “what will happen if multiple threads execute this at the same time”, because it would make developing the engine many times harder. In particular, we have some caches that are used by multiple TCastleScene instances, so any access to them is a potential crash if you have scenes in multiple threads.
So if you calculate something in a different thread, you have to use FPC / Delphi threads API to make sure that you interact with CGE only in the main thread.
I realize this is limiting – but other libraries, like VCL, LCL, or even Unity, have the same limitation. Some of our reasons and plans are on Threads | Manual | Castle Game Engine , in short:
We will likely have more ways to use threads “under the hood”, e.g. do asynchronous loading. The engine already uses internal threads in some ways.
We will likely never have an API that is “just thread-safe, use this from multiple threads”. Even something like “TCastleScene in a separate thread” would be an effort, since we’d have to make sure it is disconnected (no caching) from the other scenes, which is quite an extra effort (and hard to test).
Again, I realize this is limiting, some valid use-cases in effect cannot be optimized using threads, even though the potential gain could be huge, like in your case. But we really cannot guarantee that we support it.
I can build just the vertex and index lists in the thread and pass those to the main thread to create the tcastlescene indexed triangle mesh. It won’t make much difference in performance since all the work is creating those lists. I might even move to building those lists on the server though it would more than triple the amount of data in the stream it would make the work in the client almost nothing.
What about TCastleNoise? Is that tight enough that the same object could be accessed from different threads at the same time? The biggest bottleneck of all are still calls to that and I was thinking of moving the tile building into parallel threads on the server.
(ps, I can now save and load the full resolution tiles from disk after they are built from the noise… loading from disk is Fast!). My world is gonna be bigger!
This sounds like a solution then, cool. If you just build things like TVector3List and TInt32List in a thread, and then pass them to TCastleScene in main thread – this is guaranteed to work.
That is, using TVector3List and other simple lists (lists of simple types, like float or int, or our records like TVector3) can be safely done in any thread, no problem, as long as these lists are not accessed from multiple threads (so passing data between threads has to be properly secured to never change the data in one thread when some other thread may read it).
You mean TCastleTerrainNoise I guess. No, this one is not safe to be used from multiple threads.
It looks “self-contained” but it may be connected to other things, e.g. it descends from TCastleTerrainData that has ChangeNotifications. We don’t guarantee it will work from multiple threads. We could make this guarantee in some circumstances, but that’s really opening a “pandora’s box” of problems IMHO, i.e. this guarantee may cause us problems in the future.
If you need it, I recommend you just fork it into your own code. You can take TCastleTerrainNoise code, cut it down (remove TCastleTerrainData.ChangeNotifications, maybe remove whole hierarchy of TCastleTerrainData) and get something that is simpler and definitely thread-safe.
