Indoor Navigation (IGS) | with or w.o. Augmented Reallity (AR)

1

Hello dear CGE users. Hello dear CGE devs.

Since now, i didn’t use CGE that much. Mostly observing the CGE-UI and browsing examples.

A possible new project will start soon for me, which needs more capabilities than just displaying visual things. I need to make an indoor-navigation App or a PWA. Tasks that need to be accomplished are:

a) taking Photo (mobile phone) of an QR-Code
b) measuring distances / position from phone to WiFi / Beacon
c) sync 2D Map with current position & direction of phone
d) 3D map incl. AR function is optional (maybe for a vers. 2)

I am evaluating some software, that i could possibly use. For example, there are some solutions available for Unity3D. But i want to avoid using Unity. To heavy binaries f.e. and so on. The general question is: is it possible within “Castle Game Engine”? (i guess it has more to do with “free Pascal” / Lazarus IDE)

Somehow i believe it could be possible with Pascal / Lazarus / CGE. It would be great, if anyone here has some answers to my questions. Or if there are some links / infos or any other possibilities to realize such a project. Thx.

Greetings & cheers :beers:

2

Hi! Sorry for the long delay in reply.

Note that PWA means Progressive web app, Progressive web app - Wikipedia . It’s basically a regular webpage, plus some metadata that allows WWW browsers to “install” it on a user system. Once installed, PWA application “looks” a bit like a native application (it can have a regular icon on your phone etc.) but underneath it is just a webpage that runs in a web browser.

Creating web applications (including PWAs) with Castle Game Engine will be possible once we realize our plans: Web target – progress and plans – Castle Game Engine . I want to focus on it heavily after 7.0 release, and hope to have something by the end of 2023. But it is not something available already.

Do you want to create a web application, or a mobile application? With CGE, you can already create mobile applications (that are native on Android or iOS).

Admittedly, none of these features are readily available in Castle Game Engine, but they could be made available by writing your own services that access Android / iPhone APIs. Taking photos, GPS position, map display, AR — all these features are available in phones APIs. In CGE, we’ll want to “wrap” the existing Android / iOS functionalities and expose them.

In some cases, this should be easy. E.g. implementing Android and iOS services that query position and direction using your phone sensors should be straightforward. See about adding new services: Android Services | Manual | Castle Game Engine , iOS Services | Manual | Castle Game Engine . Getting photos should also be easy, that’s a common need on mobile.

Some of the other things you mention are not so trivial :slight_smile: AR is a big thing – Android and iOS both offer their AR frameworks (ARKit, ARCore), and we’d like to integrate with them, but this is admittedly a big job. I hope to do this someday, but we’ll not have time to do it e.g. in 2023. (Of course, if someone wants to contribute this work, I would be happy to review and help. Or if some company has money to sponsor it, I would be happy to organize a developer(s) to make it happen.)

I’m not sure about “measuring distances / position from phone to WiFi / Beacon”. If it’s something measured already by phone sensors, and available in Android / iOS APIs, then getting it should be easy.

3

Hi Michalis.

Thanks for your reply. (No worries, i am not that fast :slight_smile: )

PWA´s: i meant, it is an option if its possible. But nothing that is needed. A regular App (APK) would be fine.
And if i export some html5 stuff, via some web editors, its not a big deal to make it working like an PWA (even if its exported as a regular web page, but UI-designed as an app). So if CGE one day can export webGL, PWA is peanuts for CGE. From my point of view.

Do you want to create a web application, or a mobile application? With CGE, you can already create mobile applications (that are native on Android or iOS).

So as described above, i would be very happy if i can accomplish my tasks via exporting a regular App (APK).

a) Like you mentioned for Android Services: photo_service is already available inside CGE?! Thats great ! Point a) is doable via CGE. :+1:

b)

I’m not sure about “measuring distances / position from phone to WiFi / Beacon”

By my research, i can tell you that this measuring is running permanent. You dont need to be logged in to an WiFi or Bluetooth Beacon. When your phone is registering a WiFi signal or Bluetooth (mostly your phone spit out the name of such a possible connection), it is calculating already the signal strength, and the distance, if i understood it correctly.

d) AR. Lets cancel the AR goal. Just a simple 2D map from top would be great. This should be possible, i guess ?

My problem is, i dont know where to start. The most difficult thing for me is point b) getting such measuring data from your phone with CGE / Pascal / Lazarus. Like:

  • signal strength
  • distance
4

The photo_service (as it is now implemented, castle-engine/tools/build-tool/data/android/integrated-services/photo_service/README.adoc at master · castle-engine/castle-engine · GitHub , Castle Game Engine: CastlePhotoService: Class TPhotoService ) just allows to store an image into the “system” pictures gallery on Android and iOS.

It doesn’t allow to integrate with camera on the device.

But it is possible to add such integration, as a new service. If you know your way around Android or iOS APIs, you can add it (or submit a feature request). I emphasize: this integration is doable, but not available already.

I would say you need to search the Android and iOS native APIs documentation for whether and how they expose such parameters.

If anything else (like some Unity asset, or React Native package) is doing this, then surely it is exposed by some Android / iOS APIs. You need to find out which ones, and then we can utilize it from CGE too :slight_smile:

So it’s not really something to search inside Pascal or Lazarus. Whatever Android APIs (Java) or iOS (in Objective-C or Swift) expose, then we can query it from CGE. But in case of such things, it may be an adventure to find out how exactly Android / iOS expose what you need.

If you have anything doing it, with source code (like open-source Unity plugin or React Native plugin) then you can just look at how they’re doing it.

5

Hi Michalis.

Ah okay. Now i got it, i think. The “photo_service” is just handling picture files, like *.jpg. A possible task could be, a screenshot of your phone screen which is than stored inside the gallery folder (just my guess)? By the name “Photo_” i assumed it can handle the camera. My misstake, because i interprate it this way.

…you can add it (or submit a feature request).

Is this the correct channel to submit feature requests ? :
https://github.com/castle-engine/castle-engine/pulls

If you have anything doing it, with source code (like open-source Unity plugin or React Native plugin) then you can just look at how they’re doing it.

Thats a good hint ! Just take a look at other addons which are available out there. This could save some research time with Android API´s. If i understand it correctly, CGE can execute, or access f.e. JAVA code, which of course works with Android OS.Am i right here ? So if the API describes wich code is necessary for accessing the camera, or the wifi-module, i can access it via Pascal / CGE ?

6

Better use GitHub issues: Issues · castle-engine/castle-engine · GitHub . Add “enhancement” label to it, like Tiled maps: detect the tile indicated by mouse, taking into account texture tiles with any shape, and with any alpha channel · Issue #448 · castle-engine/castle-engine · GitHub .

Yes, exactly.

CGE has a concept of “services” that on Android are just written using Java. So they can do everything possible from Java on Android. And then Pascal code just communicates with Java – we have mechanism to do this communication (CastleMessaging or JNI) done and well tested :slight_smile:

As an example, you can take a look at Java sources of google_play_games service in Castle Game Engine . The core of this service is just a Java file ( castle-engine/tools/build-tool/data/android/integrated-services/google_play_games/app/src/main/java/net/sourceforge/castleengine/ServiceGooglePlayGames.java at master · castle-engine/castle-engine · GitHub ) that uses the Play Games services as documented by Google (independent from Pascal). Then our Pascal code (TGameService class in Pascal) just communicates with it.

7

Ah okay.

Thx for all these explanations. I will read more about the Pascal to Java communication. :+1: