Non-ascii UI and fonts management

1

Tried to make the button having Russian caption and… wow…
It seems to be a trove of issues, at least from a newb perspective, so i can not make it well-aimed and specific…

The starter was Windows and non-Latin text · Issue #360 · castle-engine/castle-engine · GitHub

Your button is an unrelated issue. By default, we don’t load non-ASCII characters from fonts. Extend data/CastleSettings.xml (see Customize look by CastleSettings.xml | Manual | Castle Game Engine ) to load non-ASCII characters in the default font (see property sample_text there). You could also create other fonts, and assign there LoadCharacters to load non-ASCII characters. See Font classes upgraded and available in CGE editor – Castle Game Engine (the docs on Text and fonts | Manual | Castle Game Engine should be updated with this information too, right now they are a little outdated).

Ooookay… First ideas:

It is really not comfy to just monkey-type ALL the alphabet and twice, and for every font out there.

Using unicode ranges, or regexp/pascal-like spans would be beter, like "A".."Z" or A-Z, at very least for sample_code

Also i think font glyphs span better be a global property, applied to all fonts (unless some specific font would override it and narrow down), not to eveyr font.

The blog posts says GCE can add non-default fonts, and allegedly edit it in Properties Inspector way. If so - there to be the same possibility to edit the default font by GUI means too.

Okay, i tried to edit settings XML, fearing GCE UI would not actually pay any attention to it.

<castle_settings>
  <ui_scaling
    mode="EncloseReferenceSize"
    reference_width="1600"
    reference_height="900"
  />
  <default_font
sample_text="АБВГДЕЁЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдеёжзийклмопрстуфхцчшщъыьэюя"
  />
</castle_settings>

Try to open it in GCE and - i have infinite loop of error windows that are spawning faster than user can close them. Basically i only have a minute or two to kill the process from TaskManager before all WIndows GDI resources would be exhausted.

[Window Title]
Error

[Content]
An error occurred when reading the CastleSettings.xml file in your project:

Exception "EInvalidSettingsXml":
The <default_font> specified in CastleSettings.xml does not have a <regular> variant

[OK]

I do not think requirement to fill regular was mentioned above, and if it was - that i know HOW to fill it for default project.

A show stopper :frowning:

P.S. also i wonder why lazy loading of missed glyphs could not be done… I imagine some game/app where localizations would be made and published long after main program releases, as plugins/DLCs… IOW it is not always possible to know all the possible characters in advance.

2

As for infinite error box:

Thanks! I fixed the infinite errors in this case.

I also improved that message. It now says The <default_font> specified in CastleSettings.xml does not have “url” attribute (for simple fonts, like TCastleFont, that are not TCastleFontFamily) nor a variant (for TCastleFontFamily fonts).

As for usage:

Indeed, it is not enough to just set sample_text. You have to also provide some font file (by url at <default_font> or, if you want to have TCastleFontFamily then in <regular>). You cannot just customize the default font – as that has character set already “fixed” because it is generated in CGE.

As for controlling default font visually:

This is planned, by an upgrade to CastleSettings. They are right now in XML, and not editable from editor. The plan is simple: make them a design, CastleSettings.castle-component, that you will be able to double-click and edit inside the editor. Then the DefaultFont (along with additional non-ASCII chars to load) will be configurable, using editor.

As for specifying the character set: I agree it could be more comfortable, but I also don’t want to go overboard with options here. We already have 3 ways to do this in Customize look by CastleSettings.xml | Manual | Castle Game Engine . And in case of localization using GetText, we provide ready sample_get_text_mo, so you don’t need to actually list all the chars – you just have to point CGE at the localization files you’re using, and custom font that handles the language. (We used it for Japanese localization of “Escape from the Universe” game with CGE.)

The issue still exists, though, if you want to just make your game in some language with non-ASCII chars, then you need to provide your own fonts and specify sample_text too (or TCastleFont.LoadCharacters when setting up TCastleFont in editor).

As for lazy-loading:

As loading font causes now a need to regenerate texture, this is not something I want to do lightly. It will inevitably cause a slowdown. Even worse, if it happens many times – one UI uses this non-ASCII character, another UI uses another non-ASCII character… Listing them explicitly as “characters to load”, at design-time, is most efficient.

I’m not saying we will never change it. I do agree this is a confusing thing for a newcomer, as non-ASCII characters don’t work “out of the box” and there’s no UI leading you to a solution. And there is possibility to improve it.

My way forward:

  • First of all, I really need to update the manual Text and fonts | Manual | Castle Game Engine to show screenshots “how to do it in editor” and reflect all the new classes/properties available after Font classes upgraded and available in CGE editor – Castle Game Engine .

  • We need to do “reloading font file at demand”. Both the default and the custom fonts. If we would do it, and make a warning about it (“we reload font texture to get new characters, this is not efficient, better specify characters using LoadCharacters”) it would be better than current behavior that shows “?” and doesn’t guide the user how to solve it.

3

Can there be some URL to just take a font from operating system ?

As loading font causes now a need to regenerate texture

Not sure what you mean here, lazy loading is about adding new (unused yet) glyphs to the already loaded font, so the font’s “clients” should not had see somethign changed.

Well, maybe one day i would dive into sources there and see what you mean and why it is so.

We already have 3 ways to do this

Maybe you then can add font loading hook, so a developer would be able to transform sample_text as he sees fit in the Pascal? Adding there ranges, or links to global constants, etc.

Frankly, current scheme to me looks as “mathematician’s answer” - correct yet remote from practical uses".

And that is why i was talking about Unicode planes or Windows codepages or other ranges.

I can hardly imagine someone hires “literature” tasks to a writer, like plot, dialogs, and tells the writer "You can use letters “A,B,C,E,F and also a,b,d,e; You can not use letters D, c, f until further contract update”. If writer makes a literature work in French - he would expect all letters of French alphabete available, same with Japanese alphabetes, etc.

Okay, there may be some special font like WebDings, like Awesome, like pseudo-graphics. Standing on their separate own.
But for most fonts their roles would be shared. Some fonts use to render any text, available to user (his language alphabete, that is), some fonts would perhaps be used to only render digits, Arabic or Roman or both.
Fonts in plural here.

The need to repeat two copies of one’s alphabete for every next font from scratch… Using GetText as a macro expander inside XML parser is an interesting hack, but matka boska, a hack it is…

I believe there can be explicit or implicit (via inheritance, CSS-like) font groups, that would clearly mapped to their use in the program.

one UI uses this non-ASCII character, another UI uses another non-ASCII character

…this can be cached in %AppData%, so it would only be slow for the first time :-DDD

One a more serious note, such situation would again ask for thinking about character subsets, not single characters. UI control then should have a way to warm-up font with ALL the text line to be rendered, not one letter a time. Which would require a way to “channel” it through all the object layers, from “button” to “font”, which is why i said above until i deep-dive into the source i can hardly grasp details. And i probably would not deep-dive as i would probably just use LCL for UI.

Or, again, to think about character groups, not disjoint characters. If a “font” was asked to render unknown glyph non-caps “7” - the font might reasonably expect all nine other non-caps digits would soon be requested too. And even if not - there would be not so much memory overuse to load 10 digits at once. Same with national alphabets, etc. Assuming disjoint characters seems to be way too fine-grained to me.

4

I don’t consider it a hack. It means we can load font with support for exactly the characters that are actually used by the localization. It solves the issue, without the need to define anything else from anyone – if you use GetText for locailzation.

As for the rest of your notes: I agree that making it more customizable, using ranges and/or developer callback, would make it better. But it would also complicate things, and right now I don’t see enough gain from this complication. If you want to implement any fancy way of determining the allowed characters, you can do this on top of CGE in a particular application – calculate LoadCharacters string and just set it for all your fonts ( Castle Game Engine: CastleFonts: Class TCastleFont ).

And we have to implement “reloading of font when some glyph is necessary” anyway. I agree that current solution is not intuitive (newcomes just see “?” without obvious solution), but introducing more ways to configure LoadCharacters is a secondary solution. The primary solution is to just automatically reload the font texture, thus fixing the problem (though with small performance loss, but we can emit clear warning about it) even for new users.