I know the font… but it looks like hiero makes you put in a character set for it to export and I don’t know what to put for japanese. For the screen shot above, I just pasted in the short line of Japanese text you had in your test program.
Incidentally, I had a sneaking suspicion that the padding issue you were seeing was actually a bug in Hiero. This image seems to confirm that:
The padding is supposed to control how much space is given around a character in the texture… but the texture coordinates in the .font file should be adjusted accordingly. Something about how Hiero writes the .fnt file is incorrect. Padding is either incorrectly included in the character advances or something else… I haven’t looked in the .fnt files to see.
Either way, it makes fixing this issue tricky on JME’s end as BM Fonts will be doing the correct thing with respect to padding.
Edit: note that another annoying thing seems to be how the two tools use “font size”. It seems like BM font is using the size as “pixel size” (as we always expect) where as Hiero is just letting the font render at that particular point size and then slicing it up… which is usually not exactly pixel size. I have to do further testing to confirm this.
So, this exercise has been VERY enlightening. I’m so glad we are doing it and maybe will finally get a handle on font rendering… but man, what a pain.
Here is a screen shot showing various fonts from Hiero and comparing them to Angel Code’s tool:
The issue with “font size” is pretty serious with respect to how JME uses fonts. (Edit: not serious after I looked at the JME code.)
(forgive the remedial font-related teaching if this is known to the reader)
In the normal world of crazy typesetting type folks… a font size of 16 does not mean 16 pixels high. It means 16 points high. To make a long story short, this is rarely the same as pixel height. In fact, “I want to render this font 16 pixels high” can be a very tricky question to ask and may require binary searching through available point sizes, and so on… (I know because I did it last weekend to try to get output to match but there was a fatal flaw in my approach…)
Hiero renders fonts at point size. So even though your exported FNT file may say it’s size 16… it is not actually size 16. It may have font glyphs that are as much as 17 or 18 pixels high.
When we render fonts to a bitmap for screen, we have a target pixel size in mind because we want good looking fonts and we know that slightly scaling them will make them look like crap. (Halving can be ok… but odd font scales will often have artifacts). Unfortunately, Hiero exported 16 point fonts rendered at 16 pixels seem like they may have scaling issues. I’ve confirmed this by looking in the fnt file at the glyph sizes… I have not confirmed that there are actually pixels in all of those sizes, though. (And actually, looking at the bitmap font code I guess it’s ok… the fonts will be rendered bigger than 16 pixels high but they shouldn’t scale when we don’t want them to.)
At any rate, it’s clear from the above pictures that the two tools treat that size very differently… and the size of the same character will vary quite a bit between the different size=16 FNT fonts that I’ve generated.
Let’s take a look at some of them:
char 33 “!”
Angel code:
char id=33 x=36 y=45 width=5 height=10 xoffset=1 yoffset=2 xadvance=4 page=0 chnl=15
Hiero Java:
char id=33 x=294 y=106 width=6 height=14 xoffset=1 yoffset=4 xadvance=5 page=0 chnl=0
Hiero FreeType:
char id=33 x=235 y=84 width=5 height=10 xoffset=0 yoffset=4 xadvance=5 page=0 chnl=0
Hiero native:
char id=33 x=17 y=0 width=10 height=22 xoffset=0 yoffset=0 xadvance=5 page=0 chnl=0
…this last one really bugs me. 22 pixels high seems completely ridiculous so there is no way that’s what has actually been rendered into the PNG.
Looking directly at the image confirms this:
Lots of extra space in that layout and the height is including all of that white space.
Anyway, as inserted above, I just looked at the bitmap font code and I guess it doesn’t cause artifacts in scaling. JME scales the actual geometry by:
sizeScale = block.getSize() / font.getCharSet().getRenderedSize();
…so if you set the BitmapText size to 16 then it will be rendered at 1:1 scale from what’s in the texture. It just won’t be rendered at 16 pixels high… which only matters if you are trying to manually layout your stuff instead of using something like Lemur, I guess.
Regardless, this has been an educational deep dive.
Hiero is definitely wrong about padding, though. Looking at the FNT files, it is incorrectly including it in the advance for the glyphs.
Here is character 33 again from the AngelCode with 5555 padding:
char id=33 x=76 y=271 width=15 height=20 xoffset=-4 yoffset=-3 xadvance=4 page=0 chnl=15
Here is the same from Hiero, similar settings:
char id=33 x=220 y=252 width=14 height=22 xoffset=-3 yoffset=0 xadvance=13 page=0 chnl=0
Notice that the xadvance is waaay too big. xadvance is about where the glyph should be rendered and so should not include padding. I also think that the yoffset is probably wrong, too… but it’s hard to say without a detailed pixel-by-pixel look in the PNG. And since I already know that the xadvance is wrong then I don’t feel particularly included to research it further.
Fixing this for loaded Hiero fonts should be as simple as iterating over the glyphs and subtracting padding from their xadvance… and probably the y offset, too. xoffset seems to be correct. We can maybe provide a utility method or something but it’s really a user-code thing. There seems to be nothing in the font file that indicates which tool it came from.
2 Likes
That padding space is used by the “distance field” usually:
[Home · libgdx/libgdx Wiki · GitHub]
So, maybe they include padding always to make sure that distance-field-based shaders will always work. And originally the padding was only intended to have some extra space for mip-mapping, since Angelcode did not plan for using distance field. Or in short:
- mip-mapping: glyph doesn’t include padding, just need extra space between
- distance-fields: glyph includes padding (for the shader), must subtract padding during layout
Yes, that would be a nice addition - especially after what you discovered about the differing sizes (px versus pt). Another thing that Angelcode didn’t plan for. But on the other hand: If every tool would generate “correctly” such distinguishing would not be necessary, since there is only one “right way” to interpret a .fnt file.
To fix the signed distance fields: Maybe change the shader so that it will add the padding when looking up the texture coordinates. This way, padding could be ignored when layout is done (like Angelcode BMFont was designed for). However, we would need a new tool to generate those distance fields then, since Hiero does things a little bit different. Or teach Hiero to do .fnt files like BMFont, but still use the padding for the SDF information like it does now.
I’m not sure your explanation covers why it’s “wrong”. I think they have a padding bug and probably just dealt with it down stream. No information is lost for the shader either way.
The issue is how the “cursor” position is tracked as the letters are ‘generated’. If you imagine some ‘xCursorPosition’ that starts at zero, each character should xadvance that cursor position. The quad coordinates are found with the xoffset, yoffset, and width and height.
Hiero incorrectly includes the padding in the xadvance. This is unnecessary and messes up the layout. Similarly, they’ve neglected to properly set the yoffset. (I could see an argument for some weird rendering structure that would require 0 offsets… but then that doesn’t explain why the x offset is calculated correctly.)
I suspect that few people use padding and when they added it for distance fields they just dealt with the bugs on the rendering side in libgdx. It doesn’t make it right, though.
Anyway, a comparison between the output of BM Font and Hiero shows that in JME all you need to do for a Hiero font with padding is to iterate over the glyphs and subtrace x left/right padding from the xadvance and ytop padding from the y offset. I haven’t tried it but it should ‘fix’ the font for JME.
So, the solution might be a combination of these two:
and
This way, we could use the Hiero-pipeline to generate .fnt with SDF in it -and- have a shader that uses the space between the glyphs as lookup for the SDF during rendering. Problem solved (hopefully).
I haven’t looked at the shader but I feel like it shouldn’t matter. xadvance should not be passed down to anything. It’s only used for placement (and used incorrectly, I guess).
And the offsets are mixed. Even assuming that’s needed, then either xoffset is wrong or yoffset is wrong, or for some crazy reason the shader does different things with x texture coordinates versus y.
But I fully admit that I don’t know anything about distance fields in this case.
The area around the glyphs (“padding zone”) is needed to store the SDF information.
The inventor of the .fnt format most likely never planned for this kind of usage.
The .fnt format was designed for:
- black/white glyphs
- black/white glyphs with outline around them
- colored glyphs (e.g. smileys with yellow, black and red in them)*
→ * - needs manual editing of the .fnt file -or- my new tool (which I stopped working on, sadly)
EDIT: But you’re right, that does not fully explain why Hiero does things so differently. My guess is that they simply didn’t know how .fnt was supposed to be used -or- because they hand over glyphs from a different renderer (Java2D, nativeOS) they had to do things a little bit different.
Good work on the font system.
I think a good way to debug spacing issues, would be start with the kernings. Create a font with zero padding & zero spacing so that only kerning issues will pop up.
Regarding Hiero and padding. Hiero adds padding to xadvance and lineHeight. It’s likely to be as you say, that padding is incorrent.
In DFontLoader, I subtracted left & right padding from xadvance, and up and down padding from lineHeight.
Which would break if there were ever a tool that output correct values and the distance field values… but it makes sense given what we have.
I think now that I’ve reloaded all of this information about fnt files into my brain that I should probably write something up that describes what all of the values do.
Hint to anyone ever creating their own file format: include at least a format version in it. 
On master there is now a new TestBitmapFontLayout that is a large subset of what was tested above. I only limited it a bit to try to save on git binary size for uploaded TTFs… so I just use the FreeSerif font.
It looks like this:
2 Likes