How to make an Interactive texture (like computer screen)?

@pspeed @zarch Thanks for the reply.



Dim me finally noticed that BitmapText extends Node therefore can be attached. -.-

I setup a jme3.font.rect, called setSize(0.05f), then attaced to RooNode.

I see beautiful Lorem Ipsum all over my screen.

Fantastic! :slight_smile:



Now for the grim part, I think I found a bug:



Things I checked:

The font file I’m using was created with JME’s built-in font generator.

The font .png is transparent with white characters.

So far it appears to be okay and the .fnt file is next to it.



What I did:

If I start my app while the BitmapText object is in view of the camera then the background is of the BitmapText object is transparent (which is great, that’s what I want)

However, if I start the app with the BitmapText object out of view of the camera then it’s background is opaque (black).



Any ideas what could be causing this?

@raxar said:
@pspeed @zarch Thanks for the reply.

Dim me finally noticed that BitmapText extends Node therefore can be attached. -.-
I setup a jme3.font.rect, called setSize(0.05f), then attaced to RooNode.
I see beautiful Lorem Ipsum all over my screen.
Fantastic! :)

Now for the grim part, I think I found a bug:

Things I checked:
The font file I'm using was created with JME's built-in font generator.
The font .png is transparent with white characters.
So far it appears to be okay and the .fnt file is next to it.

What I did:
If I start my app while the BitmapText object is in view of the camera then the background is of the BitmapText object is transparent (which is great, that's what I want)
However, if I start the app with the BitmapText object out of view of the camera then it's background is opaque (black).

Any ideas what could be causing this?


Did you put it in the transparent bucket?
1 Like

@pspeed Ha! yes that did the trick, thanks :slight_smile:

@pspeed The transparency thing just isn’t working right :frowning: I read this post http://hub.jmonkeyengine.org/groups/graphics/forum/topic/i-set-blend-mode-to-alpha-and-queue-bucket-to-transparent-but-still-i-have-a-strange-problem-with-transparent-textures/ and it appears that there’s no solution for the problem I’m getting. Anyway, thanks for the help and tips thus far :slight_smile:



@zarch Looks like I’m going to have to do the slower texture painting route after-all. Please tell me how I can use Image.java and ImagePainter.java in my project? I don’t really know how to include them since Image.java conflicts with Image.java from the core. What step do I need to take to get it up and running?

Sometime this week (if all goes to plan) I’m going to be packaging up ImagePainter as a plugin - that will be adapted to be compatible with current core.



Your best bet will be to use it then as it will just be a checkbox on the sdk to add it.

2 Likes
@raxar said:
@pspeed The transparency thing just isn't working right :( I read this post http://hub.jmonkeyengine.org/groups/graphics/forum/topic/i-set-blend-mode-to-alpha-and-queue-bucket-to-transparent-but-still-i-have-a-strange-problem-with-transparent-textures/ and it appears that there's no solution for the problem I'm getting. Anyway, thanks for the help and tips thus far :)


Well, without knowing the problem it would be impossible to help further.
1 Like

@zarch ok thanks, I will wait for your plugin then :slight_smile: oh, and I don’t want to appear ungrateful by delaying use of your plugin, it’s just me viewing this task as a “give a man a fish” scenario :slight_smile:



@pspeed hehe yes that is a good point isn’t it, here is my setup in a nutshell:

I have a ConsoleEntity object that extends BitmapText

The ConsoleEnity is added to the Bucket with ‘this.setQueueBucket(RenderQueue.Bucket.Transparent);’

Then I attachChild a new Quad to ConsoleEntity to act as the back panel.

I’ve tested both default and modiefied settings for ConsoleEnity’s material and in both cases the transparency bug appeared.

The Quad is using a material with a slight transparency and no texture.

I found that when I move the quad further away in the Z direction, the transparency bug/glitch gets better but then the whole thing looks like crap with text floating far above it.



Here’s where the transparency works, with a head-on view:

http://imageshack.us/a/img38/6972/20121105180425.png

and a little further away:

http://imageshack.us/a/img100/5789/20121105180452.png



Here is where it fails, viewing it at a side angle:

http://imageshack.us/a/img94/2712/20121105180515.png

another side angle and a bit closer

http://imageshack.us/a/img543/3938/20121105180559.png



I’d be happy to PM you the two java files for a closer look if need be, juz let me know :slight_smile:

Don’t worry, use the right tools for the job :slight_smile:



So long as you only need text then if you can get it working bitmap text is probably your best bet.



Having said that something like that that only updates occasionally should have no problems through ImagePainter so it’s all good.

1 Like
@raxar said:
@zarch ok thanks, I will wait for your plugin then :) oh, and I don't want to appear ungrateful by delaying use of your plugin, it's just me viewing this task as a "give a man a fish" scenario :)

@pspeed hehe yes that is a good point isn't it, here is my setup in a nutshell:
I have a ConsoleEntity object that extends BitmapText
The ConsoleEnity is added to the Bucket with 'this.setQueueBucket(RenderQueue.Bucket.Transparent);'
Then I attachChild a new Quad to ConsoleEntity to act as the back panel.
I've tested both default and modiefied settings for ConsoleEnity's material and in both cases the transparency bug appeared.
The Quad is using a material with a slight transparency and no texture.
I found that when I move the quad further away in the Z direction, the transparency bug/glitch gets better but then the whole thing looks like crap with text floating far above it.

Here's where the transparency works, with a head-on view:
http://imageshack.us/a/img38/6972/20121105180425.png
and a little further away:
http://imageshack.us/a/img100/5789/20121105180452.png

Here is where it fails, viewing it at a side angle:
http://imageshack.us/a/img94/2712/20121105180515.png
another side angle and a bit closer
http://imageshack.us/a/img543/3938/20121105180559.png

I'd be happy to PM you the two java files for a closer look if need be, juz let me know :)


The easiest way to solve this is with a custom GeometryComparator for the transparent bucket that will always sort the panel behind (earlier in the list) than the text. You still need a bit of offset to prevent Z-fighting but proper sorting will prevent the problem you see.

Another thing that will minimize it is to grab the materials from the font and set alpha test on. JME, for whatever reason, hard codes BitmapFont to use the unshaded.j3md file directly instead of a font.j3m or something... so this has to be fixed in code. The font loader doesn't do it but you can.

Here is the method I use to make fonts work right in my own code:
[java]
void fixFont( BitmapFont font ) {
for( int i = 0; i < font.getPageSize(); i++ ) {
Material m = font.getPage(i);
m.getAdditionalRenderState().setAlphaTest(true);
m.getAdditionalRenderState().setAlphaFallOff(0.1f);
}
}
[/java]

I've been too lazy and worried about unintended side-effects to fix that in the font loader.

Anyway, you will still need to fix the sorting issue as outlined above... but the alpha test stuff will minimize the issue if it still crops up... and more importantly, it keeps the individual letters from z-fighting where they overlap.
1 Like

@pspeed Thank you for the detailed reply.



I was on a similar track using pages[] to modify the Font material. I’ve tried a bunch of options and settings, including the ones you provided above, but still the alpha doesn’t look right.



I have not tried the GeometryComparator yet but I’ll give it a bash.



Though it looks like doing it via texture-painting is going to win out in this scenario, since it should in theory eliminate all this z-fighting and depth-sorting nonsense in one fowl swoop. Also, considering that the console will only update once every 500ms or so when not in use, will probably deliver an acceptable performance hit.

Yep, depending on the size of your texture you should be fine. You can also do some clever tricks with scrolling as well. (set wrap mode to repeat and then scroll by moving the uv co-ordinates - then you just need to render the new line each time you scroll up one line…you will need to blank the line first but that’s still better than a full redraw on the whole Image).

1 Like

To build on the previous suggestion. You could also do “smooth” scrolling this way as well by having a line worth of the texture off screen at the bottom. You then write into this off-screen line and then scroll the uv-coordinates to smoothly scroll it on.

1 Like

@zarch @pspeed



Thank for the help so far, here’s what I’ve been looking into.



I’ve been reading across the web on how text-only terminals actually work, you know the lower level stuff. I was trying to pin down why they are so fast compared to what we do in games. Anyway, it turns out that the display buffer is often only 80x25*2 bytes big. So on a 640x480 display (in this case an Image buffer) you only need to loop over on 80x25 = 2000 elements to fill the entire screen with characters. (and that’s if you actually fill the entire screen, usually you’ll only blink the caret which is 1 element)



Each character on screen is represented by 2 bytes (16bits).

The first 8 bits are used for it’s foreground and background color etc.

The second 8 bits are the actual character byte. (‘a’, ‘b’, ‘c’ etc)



I saw in JME3 that an Image can be constructed using

Image(Image.Format format, int width, int height, int depth, java.util.ArrayList<java.nio.ByteBuffer> data).



As I understand it, for Alpha enabled textures you need each pixel to be represented by RGBA (which is 4 bytes per pixel r, g, b, a).



Now here’s what I’d like to try.

  1. A character is 9x16 pixels big.
  2. I want to define an array/list fontPage[] to hold each character as a bytearray (possibly loaded by loadFont())
  3. Each element (character) in fontPage thus holds 9x16*4 bytes
  4. Then using this fontPage I write a copy of each character as I need it to the ByteBuffer of the Image.
  5. Then when the ByteBufre has been written to, I call Image() or something to create a new image and send it off to the be displayed as a texture



    I realize that I’ll have to carefully manage overflows etc, but I’m okay with this, it will be fun to figure that part out.



    To me, my logic seems sound, the one thing I’m kinda stuck on though is 5. I’m not sure how to go about that one

    What do you guys think? can it be done in jme3?

I think you are confusing a few different approaches to things. For example if you go back to the 80s/90s people were trying to fit things into 8k. They had special screen modes where you would give the id of a character for a position and it would then render it. You don’t need that sort of thing here though.



Your easiest approach will be to leverage the existing font rendering and suchlike code. ImagePainter already has code to render out a font file into a texture for you.



If not doing that then (especially if you want monospaced fonts) another approach would be to define your monitor space as a grid with a panel for each letter position. Set the UV co-ordinates to display the correct character within that cell of the grid.



What you are doing is not particularly resource intensive though - there are lots of ways to do it and none of them will tax any modern system unless you really go out of your way to make it non-performant.

1 Like

To add to that, in real terminal, the display is directly probing the memory and translating that into characters. It’s all done locally right on the hardware.



In your case, you are really manipulating a big image and copying images into it… then copying that whole image over to the GPU. It will always be slower than a real terminal.



The BitmapText approach is actually closer to a real terminal since the data is minimal in comparison… but even it is doing more work (sending a quad instead of just poking a byte) than the old terminals would. It’s a closer analogy though since the shader is what is actually taking the letter quad information and rendering to the screen. You would be moving a lot less data around than the image-based approach, though… by a large factor.

1 Like

@zarch @pspeed Thanks for the feedback guys, you both have given me good ideas and approaches.

So I guess now it’s just up to me to get this thing into a working prototype :slight_smile: