Jme3 freetype font

A freetype bitmap font plugin.

CI Maven Central binding javadoc

Downloads: Release v0.2.0 · jmecn/jme3-freetype-font · GitHub

I am currently working on a new font project. It is a rewrite of gdx-freetype and expected to have these core features:

  • Load *.ttf fonts and render them with the support of freetype. The backend now is lwjgl-freetype.
  • Generate a BitmapFont of your desired size on the fly.
  • Seamless integration with the original BitmapFont and BitmapText. This is mainly to allow Lemur to use the new font directly, and other jME3 user projects can also easily replace the font.
  • build an font editor tool like hiero, in pure jME3 way. User can preview the font, save and load font presets with it.

Other important features:

  • Emoji :monkey:!!
  • Support generating SDF fonts and rendering them correctly.
  • Support font fallback. Users can set the priority of fonts like CSS style sheets.
  • With the support of harfbuzz, correctly handle glyphs and accurately recognize languages with different writing script.

Are there any other features do you need?

16 Likes

I am very eager to have the new font applied soon. Even if the result is not visually appealing, or if the line spacing and height are calculated incorrectly, or if BitmapText cannot dynamically apply the new BitmapTextPage.

Anyway, let’s take a look at the effect first.


import com.jme3.app.SimpleApplication;
import com.jme3.asset.AssetKey;
import com.jme3.font.BitmapFont;
import com.jme3.font.BitmapText;
import com.jme3.material.MaterialDef;
import com.jme3.math.ColorRGBA;
import com.jme3.texture.Texture;
import io.github.jmecn.font.generator.FtFontGenerator;
import io.github.jmecn.font.generator.FtFontParameter;

import java.io.File;

public class TestBitmapText3D extends SimpleApplication {

    final private String TEXT = "《将进酒·君不见》\n" +
            "作者:李白 朝代:唐\n" +
            "君不见,黄河之水天上来,奔流到海不复回。\n" +
            "君不见,高堂明镜悲白发,朝如青丝暮成雪。\n" +
            "人生得意须尽欢,莫使金樽空对月。\n" +
            "天生我材必有用,千金散尽还复来。\n" +
            "烹羊宰牛且为乐,会须一饮三百杯。\n" +
            "岑夫子,丹丘生,将进酒,杯莫停。\n" +
            "与君歌一曲,请君为我倾耳听。\n" +
            "钟鼓馔玉不足贵,但愿长醉不愿醒。\n" +
            "古来圣贤皆寂寞,惟有饮者留其名。\n" +
            "陈王昔时宴平乐,斗酒十千恣欢谑。\n" +
            "主人何为言少钱,径须沽取对君酌。\n" +
            "五花马,千金裘,呼儿将出换美酒,与尔同销万古愁。";

    public static void main(String[] args){
        TestBitmapText3D app = new TestBitmapText3D();
        app.start();
    }

    @Override
    public void simpleInitApp() {
        viewPort.setBackgroundColor(ColorRGBA.DarkGray);

        MaterialDef matDef = assetManager.loadAsset(new AssetKey<>("Common/MatDefs/Misc/Unshaded.j3md"));

        FtFontGenerator generator = new FtFontGenerator(new File("font/Noto_Serif_SC/NotoSerifSC-Regular.otf"));
        FtFontParameter parameter = new FtFontParameter();
        parameter.setSize(16);
        parameter.setMatDef(matDef);
        parameter.setMagFilter(Texture.MagFilter.Bilinear);
        parameter.setMinFilter(Texture.MinFilter.BilinearNoMipMaps);
        parameter.setBorderWidth(1);
        parameter.setCharacters(TEXT);

        BitmapFont fnt = generator.generateFont(parameter);

        BitmapText txt = new BitmapText(fnt);
        txt.setSize(0.5f);
        txt.setText(TEXT);
        rootNode.attachChild(txt);
    }

}

This is a compare between two fonts.

Left is freetype with font = FreeSerif.ttf, size = 32, textureAtlas size = 256x256, generate in runtime.
Right is the Default.fnt binding inside jme3-core.

Add SDF compare

Left: freetype sdf font = FreeSerif.ttf, size = 32, spread = 4, textureAtlas size = 256x256
Center: freetype normal font = FreeSerif.ttf, size = 32, textureAtlas size = 256x256
Right is the Default.fnt binding inside jme3-core.

8 Likes

I have a problem with the BitmapText class.

As each instance of BitmapText holds a private filed textPages, they can only be updated by assemble() method. The newly generated font page won’t display.

public class BitmapText extends Node {

    private BitmapFont font;
    private StringBlock block;
    private boolean needRefresh = true;
    private BitmapTextPage[] textPages;

    public BitmapText(BitmapFont font, boolean rightToLeft, boolean arrayBased) {
        textPages = new BitmapTextPage[font.getPageSize()];
        for (int page = 0; page < textPages.length; page++) {
            textPages[page] = new BitmapTextPage(font, arrayBased, page);
            attachChild(textPages[page]);// <--- only old font pages, no new generated font pages.
        }

        this.font = font;
        this.block = new StringBlock();
        block.setSize(font.getPreferredSize());
        letters = new Letters(font, block, rightToLeft);
    }

    @Override
    public void updateLogicalState(float tpf) {
        super.updateLogicalState(tpf);
        if (needRefresh) {
            assemble();
        }
    }

    private void assemble() {
        letters.update();
        for (int i = 0; i < textPages.length; i++) {
            textPages[i].assemble(letters);// update the textPages.
        }
        needRefresh = false;
    }
}

I need to modify the assemble() so it could add new page to textPages like this:

    private void assemble() {
        if (font.getPageSize() > textPages.length) {
            // add new page to textPages
        }

        letters.update();
        for (int i = 0; i < textPages.length; i++) {
            textPages[i].assemble(letters);// update the textPages.
        }
        needRefresh = false;
    }

The first way come out to me was define a listener to observe the page add event. As the class BitmapTextPage are package private, I can’t access it without reflection. The code may looks like this.

BitmapFont font = generator.generateFont(parameter);
BitmapText text = new BitmapText(font);
generator.addListener(new BitmapTextPageListener(text));
    @Override
    public void onPageAdded(Packer packer, PackStrategy strategy, Page page) {
        int index = page.getIndex();
        logger.debug("An new page is added to BitmapText:{}", index);
        try {
            if (textPagesField.get(text) != null) {
                Object[] old = (Object[]) textPagesField.get(text);
                Object array = Array.newInstance(clazzBitmapTextPage, packer.getPages().size());
                for (int i = 0; i < old.length; i++) {
                    Array.set(array, i, old[i]);
                }
                Object newPage = constructor.newInstance(text.getFont(), true, index);
                Array.set(array, index, newPage);

                textPagesField.set(text, array);
                text.attachChild((Spatial) newPage);
                logger.info("page {} is added", index);
            }
        } catch (ReflectiveOperationException e) {
            logger.error("error", e);
        }
    }

It works, but still have some disadvantages.

  • User must create Listeners for every BitmapText object, even the Text maybe never changed.
  • Lemur code have to change a lot to work with the new font.

I need it to be automatic. Maybe I can override BitmapText’s constructor, so it can add Listener to the font by itself?

I tried to use AspectJ, adding some aspect to constructor, but I failed.
I tried to use javassist, adding some interceptor to private method assemble(), it tell me the jme3.core module is not open for my app, I can’t do that.

Finally I tried byte-buddy to redefine the BitmapText class, it works! :open_mouth:

Here is an video about dynamic font generation.

In the generator class I use ByteBuddy to redefined assemble() method.

    static {
        // The dark magic to override private method BitmapText#assemble()
        ByteBuddyAgent.install();

        new ByteBuddy().redefine(BitmapText.class)
                .method(ElementMatchers.named("assemble"))
                .intercept(MethodDelegation.to(BitmapTextDelegate.class))
                .make()
                .load(BitmapText.class.getClassLoader(), fromInstalledAgent());
    }

Then I create a delegate class to BitmapText.

package io.github.jmecn.font.delegate;

import com.jme3.font.BitmapCharacterSet;
import com.jme3.font.BitmapFont;
import com.jme3.font.BitmapText;
import com.jme3.scene.Spatial;
import io.github.jmecn.font.FtBitmapCharacterSet;
import io.github.jmecn.font.exception.FtRuntimeException;
import net.bytebuddy.implementation.bind.annotation.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.lang.reflect.Array;
import java.lang.reflect.Constructor;
import java.lang.reflect.Field;
import java.lang.reflect.Method;

/**
 * This class is a delegate to BitmapText, it overrides the private method assemble() to support dynamic generated BitmapTextPage.
 *
 * @author yanmaoyuan
 */
public class BitmapTextDelegate {
    static Logger logger = LoggerFactory.getLogger(BitmapTextDelegate.class);

    static Field textPagesField;
    static Field needRefreshField;
    static Field lettersField;
    static Field fontField;

    // Letters#update()
    static Method lettersUpdateMethod;

    // BitmapTextPage#assemble(Letters)
    static Method bitmapTextPageAssembleMethod;

    static Class<?> clazzBitmapTextPage;
    static Constructor<?> constructor;

    static {
        try {
            textPagesField = BitmapText.class.getDeclaredField("textPages");
            textPagesField.setAccessible(true);
            needRefreshField = BitmapText.class.getDeclaredField("needRefresh");
            needRefreshField.setAccessible(true);
            lettersField = BitmapText.class.getDeclaredField("letters");
            lettersField.setAccessible(true);
            fontField = BitmapText.class.getDeclaredField("font");
            fontField.setAccessible(true);

            Class<?> letters = Class.forName("com.jme3.font.Letters");
            lettersUpdateMethod = letters.getDeclaredMethod("update");
            lettersUpdateMethod.setAccessible(true);

            clazzBitmapTextPage = Class.forName("com.jme3.font.BitmapTextPage");
            constructor = clazzBitmapTextPage.getDeclaredConstructor(BitmapFont.class, boolean.class, int.class);
            constructor.setAccessible(true);// allow access to package-private constructor

            bitmapTextPageAssembleMethod = clazzBitmapTextPage.getDeclaredMethod("assemble", letters);
            bitmapTextPageAssembleMethod.setAccessible(true);
        } catch (Exception e) {
            throw new FtRuntimeException("Failed to init BitmapTextDelegate", e);
        }
    }

    public static void assemble(@This Object obj) throws Throwable {
        BitmapText text = (BitmapText) obj;
        BitmapCharacterSet charSet = text.getFont().getCharSet();

        Object letters = lettersField.get(obj);
        Object[] textPages = (Object[]) textPagesField.get(obj);

        if (charSet instanceof FtBitmapCharacterSet) {
            FtBitmapCharacterSet ftCharSet = (FtBitmapCharacterSet) charSet;
            int pageSize = ftCharSet.getPageSize();
            if (pageSize > textPages.length) {
                logger.debug("page size:{}, current:{}", pageSize, textPages.length);
                Object array = Array.newInstance(clazzBitmapTextPage, pageSize);
                for (int i = 0; i < textPages.length; i++) {
                    Array.set(array, i, textPages[i]);
                }
                for (int index = textPages.length; index < pageSize; index++) {
                    Object newPage = constructor.newInstance(text.getFont(), true, index);
                    Array.set(array, index, newPage);
                    text.attachChild((Spatial) newPage);
                    logger.debug("create new page: {}", index);
                }
                textPagesField.set(obj, array);
                textPages = (Object[]) array;
            }
        }

        // first generate quad list
        lettersUpdateMethod.invoke(letters);
        for (int i = 0; i < textPages.length; i++) {
            bitmapTextPageAssembleMethod.invoke(textPages[i], letters);
        }
        needRefreshField.set(obj, false);
    }
}

So far it works fine. But I still want some advices about better solutions.

I know I can define a new FtBitmapText, but it can’t work with lemur together.

Eventually this will be fixed as I have already started adding the ability to have pluggable text implementations to Lemur. I just don’t know when I will get back to it.

…or by defining a class in the same package to give you access.

BitmapText (mostly its underlying infrastructure) is really awful in hindsight and due for a rewrite. As is, it seems like every time someone touches it, they break five other things. It’s very fragile. (Edit: note that I have personally started such a rewrite twice before giving up for one reason or another… matching functionality ‘feature’ for ‘feature’ while also improving things is very tricky.)

ByteBuddy is a very clever solution that may serve you well in the short term.

I’d personally also be willing to support SMALL extensibility tweaks (like making assemble() protected instead of private, for example, and providing a providing a protected getter.) You’d still have to extend BitmapText with your own class… and thus extend Lemur classes with your own versions, too… though at least it’s easier in the case of a BitmapText subclass.

1 Like

Can’t agree more, the BitmapText definitely needs to be redesigned.

I found it is really hard to do the following thing with current class:

  • Do correct text-shaping by harfbuzz. String.chatAt(i) is absolutely wrong to get the real glyph.
  • Rich text tag is hard to implements with “style” thing.

Currently I’m learning from:

Trying to make it better.

2 Likes

Another big frustration for me is that there are two different pieces of code that calculate glyph position. BitmapFont does it one way and the “make the mesh” stuff does it in its own way.

In my redesign, I did it in one place… which also made it easier to find the glyph clicked on… which is essentially impossible now without some deeply invasive code.

Trying to run with lemur, here is another problem.

Here is how I load an set default font to lemur.

    assetManager.registerLoader(FtFontLoader.class, "otf");
    FtBitmapFont fnt = assetManager.loadAsset(new FtFontKey("Font/NotoSerifSC-Regular.otf", 14, true));

    GuiGlobals.initialize(app);
    BaseStyles.loadStyleResources(DarkStyle.RESOURCE);
    GuiGlobals.getInstance().getStyles().setDefaultStyle(DarkStyle.STYLE);
    GuiGlobals.getInstance().getStyles().setDefault(fnt);// set default font

It dose not work for me, because lemur saves the default value by a strongly type sensitive data structure.

    private Map<Class, Object> defaults = new HashMap<Class, Object>();

    public void setDefault( Object value ) {
        defaults.put(value.getClass(), value);
    }

    @SuppressWarnings("unchecked")
    public <T> T getDefault( Class<T> type ) {
        return (T)defaults.get(type);
    }

and how the Label component use it.

    Styles styles = GuiGlobals.getInstance().getStyles();
    BitmapFont font = styles.getAttributes(elementId.getId(), style).get("font", BitmapFont.class);// <----- type sensitive
    this.text = new TextComponent(s, font);

Finally I understand what dose “pluggable text implementations” feature means.

I need more black magic power to tinker with BitmapFont class, to allow the FtFontLoader to generate a modified BitmapFont directly.

    assetManager.registerLoader(FtFontLoader.class, "otf");
    BitmapFont fnt = assetManager.loadAsset(new FtFontKey("Font/NotoSerifSC-Regular.otf", 14, true));
    GuiGlobals.getInstance().getStyles().setDefault(fnt);// set default font

Now it works. :joy: I achieved the 3rd main goal. Then I need to make a new editor .

1 Like

Yeah, Lemur really needs a second “setDefault” that takes the class so that it’s possible to use subclass instances to override regular types.

Note that usually,

getSelector("label").set("font", someBitmapFont);

Is enough to make Lemur use a different font even if you don’t mess with a default value. Most of the text elements will key off of “label” for the text portions. (Not all of them but they can also usually be targeted in a similar way.)

…would be nice to have a second setDefault(class, obj) method, though.

1 Like

I plan to do an early release so that those interested in this project can give it a try.

This project is currently very immature, and I hope to receive more criticism and suggestions.

Note that all interfaces are not guaranteed to be stable until reaching version v1.0.0.

How to use

dependency

maven:

<dependencies>

    <dependency>
        <groupId>io.github.jmecn</groupId>
        <artifactId>jme3-freetype-font</artifactId>
        <version>0.1.2</version>
    </dependency>

    <dependency>
        <groupId>org.lwjgl</groupId>
        <artifactId>lwjgl-freetype</artifactId>
        <version>3.3.2</version>
        <classifier>natives-windows</classifier>
        <scope>runtime</scope>
    </dependency>
</dependencies>

gradle:

repositories {
    mavenCentral()
}

dependencies {
    implementation "io.github.jmecn:jme3-freetype-font:0.1.2"
    implementation "org.lwjgl:lwjgl-freetype::natives-windows:3.3.2"
}

usage

Use FtFontKey and FtFontLoader, you can specify many parameters to load a font.

public void simpleInitApp() {
    assetManager.registerLoader(FtFontLoader.class, "ttf", "otf");
    FtFontKey key = new FtFontKey("fonts/NotoSans-Regular.ttf", 16);
    // set the payload characters. It's optional.
    key.setCharacters("abcdefghijklmnopqrstuvwxyz");
    // enable real-time glyph generation.
    key.setIncremental(true);
    BitmapFont font = assetManager.loadFont(key);
    BitmapText text = new BitmapText(font);
}

Use FtBitmapFont to create a font quickly. All parameters ara default.

public void simpleInitApp() {
    FtBitmapFont font = new FtBitmapFont(assetManager, "fonts/NotoSans-Regular.ttf", 16);
    BitmapText text = new BitmapText(font);
}

Use FtFontGenerater and FtFontParameter to create a font manually.

public void simpleInitApp() {
    FtFontGenerater generater = new FtFontGenerater(new File("fonts/NotoSans-Regular.ttf"));

    FtFontParameter params = new FtFontParameter();
    params.setSize(16);
    params.setIncremental(true);
    params.setRenderMode(RenderMode.MONO);
    params.setMagFilter(Texture.MagFilter.Nearest);
    params.setMinFilter(Texture.MinFilter.NearestNoMipMap);

    params.setBorderWidth(1);
    params.setBorderColor(ColorRGBA.Black);
    
    params.setShadowOffsetX(3);
    params.setShadowOffsetY(3);
    params.setShadowColor(ColorRGBA.DaryGray);

    FtBitmapCharacterSet charSet = generater.generateData(params);

    BitmapFont font = new BitmapFont();
    font.setCharSet(charSet);
}

Use Distance-field font. You need to change the default MatDef to any other SDF font support shader. I copied one from freetype-gl as a default MatDef.

DON’T set BorderWidth or ShadowOffsetX/Y with SDF font.

public void simpleInitApp() {
    assetManager.registerLoader(FtFontLoader.class, "otf");
    FtFontKey key = new FtFontKey("Font/NotoSerifSC-Regular.otf", 64, true);
    key.setRenderMode(RenderMode.SDF);// specify the render mode
    key.setSpread(8);// specify the spread, range in [2, 32]
    key.setMatDefName("Shaders/Font/SdFont.j3md");// specify the Shader
    key.setColorMapParamName("ColorMap");
    key.setUseVertexColor(false);// SdFont currently doesn't support vertex color, so turn it off
    BitmapFont fnt = assetManager.loadAsset(key);
}

Hope you enjoy it.

5 Likes

Now working on emoji, I’ve been a bit confused.

🧑 = \uD83E\uDDD1
🧑🏽 = \uD83E\uDDD1 + \uD83C\uDFFD (🧑 + dark skin)
🧑🏻 = \uD83E\uDDD1 + \uD83C\uDFFB (🧑 + white skin)
String text = "\uD83E\uDDD1\uD83E\uDDD1\uD83C\uDFFD\uD83E\uDDD1\uD83C\uDFFB";
System.out.println(text);

the result should be:

🧑🧑🏽🧑🏻

But what I got.

https://youtrack.jetbrains.com/issue/JBR-2523

The editor is nearly done. I’m currently using it to tweak font parameters.

4 Likes

Great job! !!!
Now we can use custom fonts ~~~~

BTW
how do you handle the text input aka: IME?

I didn’t do anything, imgui handles it very nice.

I release v0.2.0 library, this version is mainly about the font tool. You can download it from github
release page.

5 Likes

Finally I figured out how to process emoji with ZWJ and Fitzpatrick modifier.

The problem now is how to split a text into different text runs by the mixed languages.

7 Likes

The icu4j seems to be a good solution for this issue.

UScriptRun can split text into different script, Bidi can split text into different writing direction.

import com.ibm.icu.lang.UScript;
import com.ibm.icu.lang.UScriptRun;
import com.ibm.icu.text.Bidi;

class TestIcu4j {

    @Test void testUScriptRun() {
        String text = "Love and peace" +// latin
                "爱与和平" +// Han
                "الحب والسلام" + // Arabic
                "사랑과 평화" // Hangul
                ;

        UScriptRun run = new UScriptRun(text);
        while (run.next()) {
            int start = run.getScriptStart();
            int limit = run.getScriptLimit();
            int script = run.getScriptCode();
            System.out.printf("Script %s from %d to %d\n", UScript.getName(script), start, limit);
        }
// output:
// Script Latin from 0 to 14
// Script Han from 14 to 18
// Script Arabic from 18 to 30
// Script Hangul from 30 to 36
    }

    @Test void testBidi() {
        String text = "Love and peace" +// latin
                "爱与和平" +// Han
                "الحب والسلام" + // Arabic
                "사랑과 평화" // Hangul
                ;

        Bidi bidi = new Bidi(text, Bidi.DIRECTION_DEFAULT_LEFT_TO_RIGHT);
        System.out.printf("isMixed:%b, runCount:%d\n", bidi.isMixed(), bidi.getRunCount());

        for (int i = 0; i < bidi.getRunCount(); i++) {
            int start = bidi.getRunStart(i);
            int limit = bidi.getRunLimit(i);
            System.out.printf("start=%d, limit=%d, level=%d\n", start, limit, bidi.getRunLevel(i));
        }

// 0-left_to_right, 1-right_to_left
// output:
// isMixed:true, runCount:3
// start=0, limit=18, level=0
// start=18, limit=30, level=1
// start=30, limit=36, level=0
    }
}

ICU4j do not process emojis, so something like emoji-java or emoji-segmenter should be used first.

After dig into pango/pango-emoji.c for one week, I’m so excited that emoji can be recognized correctly.

>>>> 😊 <<<<
unicode count:1, character count:2
[id]:  unicode,   string, emoji, text
[ 0]: [ 0,  1), [ 0,  2),  true, 😊

>>>> 👨‍👩‍👧‍👦 <<<<
unicode count:7, character count:11
[id]:  unicode,   string, emoji, text
[ 0]: [ 0,  7), [ 0, 11),  true, 👨‍👩‍👧‍👦

>>>> ✋🏻 <<<<
unicode count:2, character count:3
[id]:  unicode,   string, emoji, text
[ 0]: [ 0,  2), [ 0,  3),  true, ✋🏻

>>>> 👩🏽‍🚒 <<<<
unicode count:4, character count:7
[id]:  unicode,   string, emoji, text
[ 0]: [ 0,  4), [ 0,  7),  true, 👩🏽‍🚒

>>>> 🆒 <<<<
unicode count:1, character count:2
[id]:  unicode,   string, emoji, text
[ 0]: [ 0,  1), [ 0,  2),  true, 🆒

>>>> 🇨🇳 <<<<
unicode count:2, character count:4
[id]:  unicode,   string, emoji, text
[ 0]: [ 0,  2), [ 0,  4),  true, 🇨🇳

>>>> 🏴‍☠️ <<<<
unicode count:4, character count:5
[id]:  unicode,   string, emoji, text
[ 0]: [ 0,  4), [ 0,  5),  true, 🏴‍☠️

>>>> #️⃣*️⃣0️⃣1️⃣2️⃣3️⃣4️⃣5️⃣6️⃣7️⃣8️⃣9️⃣ <<<<
unicode count:36, character count:36
[id]:  unicode,   string, emoji, text
[ 0]: [ 0, 36), [ 0, 36),  true, #️⃣*️⃣0️⃣1️⃣2️⃣3️⃣4️⃣5️⃣6️⃣7️⃣8️⃣9️⃣

>>>> Hello, 你好,🌍世界! <<<<
unicode count:14, character count:15
[id]:  unicode,   string, emoji, text
[ 0]: [ 0, 10), [ 0, 10), false, Hello, 你好,
[ 1]: [10, 11), [10, 12),  true, 🌍
[ 2]: [11, 14), [12, 15), false, 世界!

>>>> Hello🙋🧑🧑🏻🧑🏼🧑🏽🧑🏾🧑🏿world🍰🐒家庭👨‍👩‍👧‍👦 <<<<
unicode count:33, character count:51
[id]:  unicode,   string, emoji, text
[ 0]: [ 0,  5), [ 0,  5), false, Hello
[ 1]: [ 5, 17), [ 5, 29),  true, 🙋🧑🧑🏻🧑🏼🧑🏽🧑🏾🧑🏿
[ 2]: [17, 22), [29, 34), false, world
[ 3]: [22, 24), [34, 38),  true, 🍰🐒
[ 4]: [24, 26), [38, 40), false, 家庭
[ 5]: [26, 33), [40, 51),  true, 👨‍👩‍👧‍👦
4 Likes