Detecting mouse over Text specific Word

General Discussion
1

Hello,

I would like to ask you for good idea for detecting when mouse is over specific word in all text (Whatever if its Default JME one or TTF lib).

For example i would like to have tooltip when mouse is over specific word. Any nice ideas/help?

If idea would even allow having custom icons within text, would be even better.

I thought on splitting text on lines, and where there is word in middle of line then split even for smaller areas of text, but this idea sounds nasty.

Another idea i had was creating AppState to pre-render texture with text split like above idea, but as one texture with additional params of mouse uv coords for words on that texture.

Could also just count letter size and try predict it, idk.

2

Once you get past basic rendering of Latin (or similar) alphabets, text gets very complicated very fast. Generally to detect a word you’d have to calculate the beginning/end visual coordinates of the word and compare the cursor’s position.

Emoji/other icons are similarly difficult. For what you’re after I’d probably just pull in LWJGL’s new FreeType/HarfBuzz bindings, render text to a texture, and see what API they have available to answer queries about what word/letter a specific coordinate is visually over.

1 Like
3

My two cents

One idea could be to underline, use a different color (afaik JME supports styling parts of texts via special tags), or add * at the end of those words you want to provide information on a page, then add an info button at the end that when clicked shows all tips for those words in a single popup panel.

Another idea could be to create a small tooltip button (a button with just a small tooltip icon on it) using a GUI library like Lemur and place them over the words you want (with a very small Z offset), so when clicked it will popup the tooltip panel for that word.

2 Likes
4

I mean im already using:

Where it can render to Texture. Maybe i should ask @Tryder if he is still arround here and there is some API for that. There are a lot of methods about line widths for every line or font height/etc.

And i thought to make calculation, tho it might be very not precise if there could be any bolded text or different things.

But ye, this was one of idea. Maybe FreeType/HarfBuzz could already have it in their API, then maybe i could switch library, thats true.

True, it could be, tho its not designed way ofc. Might be easier for Pad this way, so might think about this way too or just make this way as alternative. But i still would like have normal mouse tooltip over word. Anyway thanks for alternative way in total different view :slight_smile:

1 Like
5

You can handle this accurately if you have enough API to interact with the layout engine. I developed an in-house UI framework (Obsidian) that I planned to announce here and formally begin offering support for soon (once I polish off one last open question around some of the component API). Here is an example of some minimal API for interacting with the layout engine, and here is an example showing cursor placement. I have some more work to do with advanced text styling (the above only work in spans of text that share the same font size/weight/overall style), but with some minimal information from the layout engine you can get what you want. You really need the offsets from the layout engine rather than guessing it based on glyph widths since different writing systems may have rather different layout rules - if you guess it you’ll probably be right most of the time in most common languages, but it’ll break in other cases. Plus, it’s much easier to let the layout engine correctly handle all the kerning and other layout features that layout engines are made to handle.

1 Like
6

Note that Bitmap text is just a bunch of glyph quads. In theory, you could use the intersected triangle index to figure out which quad you hit and thus the letter.

…the TTF stuff must work in a similar way, I would think.

1 Like
7

Oh, i thought bitmap generate texture not mesh, seems like my memory is weakening :smiley:

Yes, TTF have 2 modes. One generate Texture and one generate Mesh. Idk if mesh have anything that i can decide which part of text it is, tho this is some kind of idea.

I thought of using just bitmap, but then i seen ttf have no problem with chineese/etc languages too.

After even more investigation some languages are written from right to left or even bottom to up.
While bitmap or ttf can align vertically and horizontally as i remember, i still would like to put icon for a words. You know, like you have <metal_icon>Iron for a strategy games.

So currently just thought to use ttf, but make offscreen rendering per word, so can add even icons there.

It could work even for bitmap if bitmap support more than i see on .fnt textures. Like chineese/etc letters. I seen fonts didnt had much on them, just basic letters, im not specialist in this topic, but i guess i would need load different fonts per language or something to have chineese for a bitmap from .fnt?

With word by word offscreen render idea could just render each word as “quad” positioning them one by one with align proper for language.
And then pre-render one single texture with textArea size, with embeded provided information of position for keywords on it. And also possibility to include icons.

Your idea sounds cool if i would not like icons/styles-per-word, and if i can determine word/char by mesh data.

I were thinking to use JavaFX or WEB engine in game itself. But i think it work much faster and less riscy to just use internal rendering system to manage UI.

Anyway if i would want use lib other than i got here, i would use Lemur anyway for same reasons.

Also its not like TTF have no API, since it have a lot of params, im just confused and could not find proper coorelation between them. But anyway once i had now new idea after you helped me, stopped looking for it, since i belive new idea might work.

I were testing ttf params, but even parameters like “getFont().getScaledPointSize()” were not maching line width, so i were unable to find proper coorelation here.

While i know all width or line with. For example ttc.getWidth() or ttc.getTextWidth(), so i can use it to know for example total width. Tho i were unable to find correct width per letter.

Also in TTF lib there are some “margins” that can be setup (or are setup idk).
Were trying to find coorelations of many values like here to calculate:

        System.out.println("!!!!!!!!!!textDataBEGIN!!!!!!!!!!!");
        System.out.println("text " + text);
        System.out.println("ttc.getHeight() " + ttc.getHeight());
        System.out.println("ttc.getWidth() " + ttc.getWidth());
        System.out.println("ttc.getTextHeight() " + ttc.getTextHeight());
        System.out.println("ttc.getTextWidth() " + ttc.getTextWidth());
        System.out.println("ttc.getStringContainer().getLineCount() " + ttc.getStringContainer().getLineCount());
        System.out.println("ttc.getStringContainer().getFont().getActualPointSize() " + ttc.getStringContainer().getFont().getActualPointSize());
        System.out.println("ttc.getStringContainer().getFont().getLineWidth(getText(), 0) " + ttc.getStringContainer().getFont().getLineWidth(text, 0));
        System.out.println("ttc.getStringContainer().getFont().getScaledPointSize() " + ttc.getStringContainer().getFont().getScaledPointSize());
        float totalLinesLength = 0;
        for (int i = 0; i < ttc.getStringContainer().getLineWidths().length; i++) {
            totalLinesLength += ttc.getStringContainer().getLineWidths()[i];
            System.out.println("ttc.getStringContainer().getLineWidths()["+i+"] " + ttc.getStringContainer().getLineWidths()[i]);
        }
        System.out.println("totalLength (total like length)" + totalLinesLength);
        System.out.println("getText().indexOf(\"labore\")" + text.indexOf("labore"));
        System.out.println("getText().length()" + text.length());
        System.out.println("!!!!!!!!!!textDataEND!!!!!!!!!!!");

Yes i would need api access to know word positions, tho i got cursor position that no issue, it even work on 3d GUI elements in game i needed. But i see there are some apis in your UI for char positions.

But instead changing all GUI code/files now, its still more easy for me to just make own Text rendering word by word or char by char using ttf or bitmap or whatever. It would be much work it would be now to change all GUI library. The only thing i need rly is word position on rendered text texture. So here i might only use your idea about FreeType/HarfBuzz if they really have good API for that.
And if first idea described in beginning of post will not work as expected i will proably try LWJGL libs that you suggested and see.

TTF lib also i seen need new area(texture generation) if i would like to make onw word different color/etc. I forgot about that, so now it even more make me choose to just make offscreen renderer connecting word by word generated by other lib (currently ttf lib) where i could at least color/etc this words differently in the middle of the text too.

Yes, in general seems we think about the same, but your idea to to use lower level library API capabilities, while now i had idea about just make own offscreen renderer word by word generated by this libs where i would also know direct position + icons + inner text different styles.

Because if i would use lower API now, and lets say i will know real offset of letter/word, still probably i could make space for icon to put over, but i would not be able easly make single word in much different style.

In general thanks for help, every opinion gave me a lot of ideas. For now i will try just make text texture generation from smaller word-text texture generations from lower api ttf lib. and this way add icon capability and style per word capability (i forgot about this is not possible for lower libs, thanks for letting me know :slight_smile: )

Im not sure how they do this in different games, but if single word of all text is in different style, then i guess they also render them word by word or letter by letter determining style per them. and then connect all into single mesh or texture or etc.

Thanks for different ideas and view on topic, it helped me a lot! :slight_smile:

1 Like
8

I added a few custom icons to my BitmapText “back in the day” by editing the font png file directly and overwriting characters that I wouldn’t be using. BitmapText also supports multiple font ‘pages’ for different styles (like bold, italic, etc.) which might also be leveraged for icons. I don’t remember if they have to be monochrome or not… but probably.

…but you are right about ‘infinite character’ languages like chinese being a problem. Trying to generate a prebuilt font png for that gets tricky. I think the TTF library adds glyphs to the texture only as needed.

Personally, if I wanted to do all of the things you are doing then I’d probably just write my own text handling from scratch using per-character glyphs and a packed texture (built on the fly)… but with the benefit of having a way to go back and forth and have custom glyphs and stuff.

2 Likes
9

Thx, so Bitmap indeed have problem with ‘infinite character’ languages.

What i try now seems to be middle-ground between handling from scratch and just using libs.

Icons i thought i might just use png/etc scaled down as quad(same as text words as quads), but idk what quality or scaling i should use to maintain good quality. Mipmapping should not be issue i guess if all correctly setup. Anyway it do not need to be perfect.

10

I think there are some methods in jme-ttf that return glyps, the widths of the glyphs and the x-advance of each glyph. If you’re looking for the position of a specific word within a sentence or something you could add up, if memory serves, the x-advances for each glyph preceeding the word in question, plus double the outline width if you’re using outlines, then add in the position of the text block and any padding that may also be present and you have the position of your word.

Add the x-advances for each glyph in the word, plus double the outline width if any, and you have the width of your word.

2 Likes
11

Thx for help, but what is x-advance vs glyph width? For me x-advance is even bigger value than glyph width.

((TrueTypeContainer) element).getStringContainer().getLines()[0][0].getWidth();

this one is even smaller than

((TrueTypeContainer) element).getStringContainer().getLines()[0][0].getXAdvance()

((TrueTypeContainer) element).getStringContainer().getLines()[0][0].getWidth(): 33
((TrueTypeContainer) element).getStringContainer().getLines()[0][0].getXAdvance() : 42

Also what do you mean with outline?

edit:
I made offscreen rendering to be able put icons and use different style per word.

Im using

whitespaceWidth = ((TrueTypeContainer) element).getStringContainer().getFont().getLineWidth(" ", 0);

For a whitespace. here question if its proper for calculating whitespace width? i just random used it and it seems working, tho i do not know how this method work exactly.

nextPositionX += ((TrueTypeContainer) element).getTextWidth();

for a word width

maxLineHeight = ((TrueTypeContainer) element).getTextHeight();

for calculate line height

and the result in Test is like (at least icon work nice):

image
image1544×791 133 KB

12

The width is the absolute width of the glyph, from the left most point on the curve to the right most point on the curve. X-advance is the width of the glyph plus the amount of spacing that should appear between the right most point of the glyph curve, or left most for right to left directionality text such as arabic, and the next glyph in a string.

So if you’re only displaying a single character then your display area need only account for the width of the displayed glyph, but in a string you need the x-advance to determine where to begin drawing the next glyph in the string. In this case you’re looking for the position of a word within a string so you want x-advance. Jme-ttf gives you the option, if memory serves, to set a text kerning value which is just extra space between characters so if you’ve set a kerning value you want to add that for each character as well.

Jme-ttf gives the option to add an outline to bitmap text. The value set for this option is the absolute width of the outline in pixels, I believe. Since the outline covers both sides of a character you need to double this value and add it to the width of the glyph to get the actual display width, but only if you have the outline option set above the default value of zero.

This is all from memory so I may be a little off here and there. I’m sitting on my bike outside a stripmall right now. Absolutely stunning day for a ride by the way. Best wishes.

1 Like
13

Thanks, i could probably manage calculate word position with this explaination.

But please note im currently using word by word rendering, so topic changed from “find-word-position” into “do i use proper whitespace width and line height and etc”

Initially i thought to calculate position, but now i got it on hand myself anyway.

Thanks again for more info on this.

I really like your library, tho it wasnt updated for 4 years, but it works perfectly so no need :wink:

On site there is jME-TTF-v2.2.2.zip while Github have v2.2.1 Release. I guess i am currently using Github master version, is there some out-of-github newer version?

1 Like
14

Good to hear someone likes something I made. Almost 20k views on the jme-ttf thread, maybe a few more people are using it. I’m surprised it still works out of the box after four years without maintenance.

I would stick with the one from my website. I stopped working with git repositories some years ago. Maybe I’ll start toolin’ around with jme again someday, Godot is, shall we say… buggy.

2 Likes
15

Oh, so its real that non-git one is more current version.

I will look later at this zip file if it have more current version.

Why not git tho? You have something better for versioning/storing?

16

Git makes more sense to me if I’m collaborating with a group of people. Working by myself it just adds extra work with no benefit that I am aware of.

17

Hmm, ok. But you no need rly create branches / merge releases / pull requests / code-checks / etc.

You can always just commit+push (seconds) into Master without even proper description. Thats all then.

Benefits: you got “additional code store place” + “commit history” + “auto-publish for people like me”

18

I’m not sure I see the need for a commit history for a lone artist. Auto-publish isn’t so great if you don’t always have an internet connection and it’s not as if my works need immediate updates, usually I just work on something periodically for months or even years then bundle it up once and upload it.

Preview for the link isn’t working on my end:
" Sand mandala is a [Tibetan Buddhist] tradition involving the creation and destruction of [mandalas] made from colored [sand]. Once complete, the sand mandala’s ritualistic [dismantling] is accompanied by ceremonies and viewing to symbolize Buddhist doctrinal belief in the transitory nature of material life"

19

I cannot count the number of times I’ve looked through the commit history of my own private repositories (SVN in my case).

Sometimes people think they are just a lone developer but then they forget that they are actually a team of “past-me”, “present-me”, and “future-me”… and those are very different people.

20

I just forgot to tell.

Topic is solved. Thanks everyone for suggestions :slight_smile:

Here is Demo result of what i needed:

That was what i really needed.

For different platforms they might appear in different way ofc.

2 Likes