Castle-data URIs under MacOS launched from IDE

Today I converted a test app to use the castle-data: URI scheme

When running the program from the IDE the data files can’t be loaded when using my Mac.

Running the program from the command line works if I start it by starting the program in the source directory or using <application>.app/Contents/MacOS/<application> (the latter is simply a symlink back to the real executable)

I found a work-around here…

https://forum.lazarus.freepascal.org/index.php/topic,34244.msg223818.html#msg223818

Just wondering if there’s an in-built way to make everything work transparently. I want the same source-tree to function properly regardless of deployment OS.

Cross-Platform is important to me so I’ve re-tasked a (very bad) Laptop as an Ubuntu test system with main development on Win 10. When I get iOS + Android build facilities working I’ve got tablet + phones for both OSes to hand as well.

By accident, I answered a very similar question 3 days ago, see http://disq.us/p/29cl7tc . Pasting my answer below – it contains 3 possible solutions.

But I also just realized how to trivially modify the ApplicationData detection to handle it better automatically. I’ll do it tomorrow (have to test on mac).


That’s because you run from Lazarus through the “Application Bundle” (see the checkbox “Use Application Bundle for running and debugging” in the “Project Options”).

This means that Lazarus creates a special directory called MyApplication.app that user can double-click to run the program. It is actually a directory. When you use “castle-data:/” protocol, and CGE detects that we are inside the “application bundle”, the engine assumes that the data files are in the Contents/Resources/data/ subdirectory inside the application bundle.

Because that’s how it has to work when the application is released to the user. The data files should then be part of the “Application Bundle”. This allows the user to just drag-and-drop the MyApplication.app to move it / install / trash etc.

But it is admittedly burdensome when developing.

Solutions:

  1. Do not run through “Application Bundle”. Uncheck “Use Application Bundle for running and debugging” in project settings in Lazarus. I’m not sure about the possible bad side effects – quickly testing it seems everything is OK. The new form is just not focused at start, but that’s a no-problem.

  2. Copy your data files each time before running the application. Like rm -Rf MyApplication.app/Contents/Resources/data && cp -R data/ MyApplication.app/Contents/Resources/

  3. Add in your application a debug code that sets ApplicationDataOverride. Like

{ When debugging on desktop macOS,
and you are inside an "Application Bundle",
override castle-data:/ meaning to look at "data/" outside of the bundle. }
{$if defined(DEBUG) and defined(DARWIN) and (not defined(CASTLE_IOS))}
if BundlePath <> '' then
ApplicationDataOverride := FilenameToURISafe(BundlePath) + '../data/';
{$endif}

Actually I made a fix already, in https://github.com/castle-engine/castle-engine/commit/ba07da9ae862596186628d122f8be4081f120c32 .

Please test it and let me know – I did it blindly now, I’ll actually test it tomorrow.

Note that this only makes things running “out of the box” in development, when the “data” subdirectory is actually a sibling of the temporary application bundle created by Lazarus.

When distributing the application to end users, you should still manually copy the “data” subdirectory inside the bunde (to Xxx.App/Contents/Resources/data). In case of compiling and packaging using our build tool we could do it automatically (although it’s not implemented for now), in case of using Lazarus to create the bundle – you will have to do it manually always. I recommend writing a simple shell script for this.

I’ll document it on https://castle-engine.io/macosx_requirements.php tomorrow.

( We actually have an internal shell script in CGE to help with macOS bundles, but it doesn’t help with copying “data” now. It’s in https://github.com/castle-engine/cge-scripts/blob/master/create_macosx_bundle.sh , used by https://github.com/castle-engine/view3dscene/blob/master/macosx/pack_macosx.sh .)

https://castle-engine.io/macosx_requirements.php has been updated. At the bottom I mention the “data” directory stuff: https://castle-engine.io/macosx_requirements.php#section_data .

Sorry for lots of messages, I really should go to sleep instead of coding :slight_smile:

Tried it out and it works nicely

One small addendum, how to I get the actual location of a castle-data: location

i.e. I want to be able to do something like…

if not FileExists(CastleFileLocation('castle-data:/manifest.json')) then
  begin
    ShowMessage('Missing castle-data:/manifest.json');
  end;

The best answer is that you should not use FileExists, instead you should use URIExists or URIFileExists for such checks. They automatically handle URLs (with protocol castle-data:/, file:/ and some others) correctly.

If you really want to go more low-level,

  • you can use ResolveCastleDataURL to convert the URL to another URL without castle-data:/ protocol. But beware – the result may use file:/ protocol, or it may use something else. On some platforms (Android and Nintendo Switch, right now), the “data files” are not regular files at all. They are being accessed through special API functions on these platforms. E.g. on Android, resolving URL with castle-data:/ protocol will result in an URL with castle-android-assets:/ protocol. The proper way to read data files there is to use Android Assets API.

    ( Although Android also features a regular file system, e.g. to read SD card files, but data files “packaged” inside an application are accessed through a special Android Assets API. )

  • you can also always use URIToFilenameSafe to convert URL to a regular file path (not URL). It will work for file:/ URLs, it will also work for castle-data:/ URLs but only on platforms where castle-data maps to regular files. It will return empty string otherwise.

IOW,

  • checking FileExists(URIToFilenameSafe('castle-data:/manifest.json')) … will work, but only on some platforms.

  • Checking URIFileExists('castle-data:/manifest.json') is simpler, and will work always.

(We even do special trick to make it work on platforms that don’t expose an API to check file existence in an efficient way. E.g. on Android, when you package a game, we additionally create and package a special XML file that acts like a “table of contents” for the data directory. It will be completely automatically created and used by URIExists and URIFileExists routines.)