TextSubtext() issue

[DesLandysh]

I'm testing the text functions in python wrapper of raylib, and found this issue I couldn't resolve:

1. I have a string that contains ASCII and Cyrillic letters, I even ensure that it's 'utf-8' encoding.
2. I use drawTextEx() function with textSubtext() function with the parameters from tutorial.
   And it goes fine until it meets the Cyrillic glyph.

I tried different fonts
I even ensure that codepoints contains need glyphs and print it's copy using loadUTF-8() function, so Pycharm Terminal sees these utf-8 glyphs, but textSubtext() is not.

I guess, it's wide characters issue in C (and locale), that's why it also screened what returns and error (ffi goes to None with remark that it's not true address of __cffi...)

(OS: Windows10CORP 22H2, Pycharm2023.2.3, Python 3.12)

Have anyone thoughts how to solve this?

[orcmid]

I can't tell from your code. However, it is important to know that codepoints are not UTF-8, they are essentially UTF-32. Put another way, codepoints are exactly the way glyphs are assigned in the Unicode specification (e.g., л has codepoint 0x043B). It happens, of course, that the first 127 of them correspond to the single-byte UTF-8 encodings and also the ISO-646 (Internationalized ASCII) codes.

This is different than the wide-character business on a platform such as Windows. Those are typically (historically?) UTF-16 when used for Unicode. Part of the difficulty here is that wchar_t is not the same on all platforms and libraries. It's possible for wchar_t variables to hold full-Unicode code points. This gets murky when creating portable code for multiple-platforms.

[raysan5]

@DesLandysh I'm afraid TextSubtext() does not consider UTF-8 strings but a plain byte array of data, so, Cyrillic characters are actually processed byte by byte, getting the equivalent character for every individual byte (or just crashing in the process)...

The solution would be getting the subtext character by character, considering that on UTF-8 some characters could imply more than one byte. raylib provides the following function for that:

int GetCodepointNext(const char *text, int *codepointSize);  // Get next codepoint in a UTF-8 encoded string

Here some code sample to move along a text buffer one character at a time, considering Unicode codepoints codified as UTF-8:

int currentCodepoint = -1;
int nextCharacterIndex = 0;

while (currentCodepoint != 0)   // Check for '\0', usually EOL character
{
    int codepointSize = 0;
    int currentCodepoint = GetCodepointNext(my_text + nextCharacterIndex, &codepointSize);
    nextCharacterIndex += codepointSize;  // It could be 1 byte or more
}