Lemur smooth scrolling using off-screen render

User Code & Projects lemur
1

Hi @pspeed,
I finally decided to try this out and I somewhat succeeded.
I have only two issues.
First, the gui rendered in separate viewport is a little darker and the font seems distorted.
Second, TextField doesn’t work. It doesn’t show anything and seems not to react to input.

The code is a mess as it is a raw experiment but it should work out of the box.
Could please take look and point me in the right direction?


import java.util.ArrayList;
import java.util.function.Consumer;

import com.jme3.app.SimpleApplication;
import com.jme3.material.Material;
import com.jme3.math.Matrix4f;
import com.jme3.math.Vector2f;
import com.jme3.math.Vector3f;
import com.jme3.renderer.Camera;
import com.jme3.renderer.RenderManager;
import com.jme3.renderer.ViewPort;
import com.jme3.scene.Geometry;
import com.jme3.scene.Node;
import com.jme3.scene.Spatial;
import com.jme3.scene.control.AbstractControl;
import com.jme3.scene.shape.Box;
import com.jme3.texture.FrameBuffer;
import com.jme3.texture.Image.Format;
import com.jme3.texture.Texture2D;
import com.simsilica.lemur.Axis;
import com.simsilica.lemur.Button;
import com.simsilica.lemur.Container;
import com.simsilica.lemur.FillMode;
import com.simsilica.lemur.GuiGlobals;
import com.simsilica.lemur.Label;
import com.simsilica.lemur.ListBox;
import com.simsilica.lemur.Panel;
import com.simsilica.lemur.RangedValueModel;
import com.simsilica.lemur.Slider;
import com.simsilica.lemur.TextField;
import com.simsilica.lemur.component.QuadBackgroundComponent;
import com.simsilica.lemur.component.SpringGridLayout;
import com.simsilica.lemur.core.VersionedList;
import com.simsilica.lemur.core.VersionedReference;
import com.simsilica.lemur.event.CursorButtonEvent;
import com.simsilica.lemur.event.CursorEventControl;
import com.simsilica.lemur.event.CursorMotionEvent;
import com.simsilica.lemur.event.DefaultCursorListener;
import com.simsilica.lemur.event.MouseAppState;
import com.simsilica.lemur.style.BaseStyles;

public class LemurPanelWithOffscreenTextureTest extends SimpleApplication {
	
	private Node myRoot;
	private Texture2D target;
	private Camera cam;
	private Container mainCont;
	private Panel imgPanel;
	
	public static void main(String[] args) {
		LemurPanelWithOffscreenTextureTest app = new LemurPanelWithOffscreenTextureTest();
		app.start();
	}

	@Override
	public void simpleInitApp() {
		initLemur();
		createLemurGUI();
		setupOffscreenRender();
		bindTexture();
	}
	
	@Override
	public void simpleUpdate(float tpf) {
		super.simpleUpdate(tpf);
		myRoot.updateLogicalState(tpf);
		myRoot.updateGeometricState();
	}
	
	private void createLemurGUI() {
		mainCont = new Container(new SpringGridLayout(Axis.X, Axis.Y, FillMode.None, FillMode.None));
		mainCont.setBackground(null);
		
		imgPanel = mainCont.addChild(new Panel(), 0, 0);
		imgPanel.setPreferredSize(new Vector3f(400, 400, 0));
		QuadBackgroundComponentCustom bckg = new QuadBackgroundComponentCustom(target);
		imgPanel.setBackground(bckg);
		
		ArrayList<String> items = new ArrayList<String>();
		for(int i = 0; i < 50; i++) {
			items.add("Item in some large list " + i);
		}
		VersionedList<String> model = new VersionedList<>(items);
		ListBox<String> listBox = mainCont.addChild(new ListBox<>(model), 2, 0);
		listBox.setVisibleItems(20);
		
		Slider slider = mainCont.addChild(new Slider(Axis.Y), 1, 0);
		
		slider.addControl(new RangedValueWatcher(slider.getModel(), value -> {
			System.out.println(value);
			Vector3f location = cam.getLocation();
			location.setY((value.floatValue() * 4) + 200);
			cam.setLocation(location);
		}));
		
		slider.getModel().setValue(100);
		
		mainCont.setLocalTranslation(calculateCenteredPosition(mainCont, getCamera()));
		getGuiNode().attachChild(mainCont);
	}
	
	private void bindTexture() {
		imgPanel.setBackground(new QuadBackgroundComponent(target));
	}
	
	private void setupOffscreenRender() {
		int x = 400;
		int y = 400;
		
		float camX = imgPanel.getWorldTranslation().x;
		float camY = imgPanel.getWorldTranslation().y - imgPanel.getPreferredSize().y;
		
		System.out.println("CamX: " + camX);
		System.out.println("CamY: " + camY);
		
		cam = new OffscreenCamera(x, y, new Vector2f(camX, camY));
        cam.setFrustum(1.0f, 1000.0f, -200, 200, 200, -200);
        cam.setLocation(new Vector3f(200f, 200f, 10f));
        cam.lookAt(new Vector3f(200f, 200f, 1f), Vector3f.UNIT_Y);
        cam.update();
		
        System.out.println(cam.getDirection());
		System.out.println(cam.getLocation().toString());
        
        ViewPort preView = getRenderManager().createMainView("test", cam);
		preView.setClearFlags(false, false, false);
		
		target = new Texture2D(x, y, Format.ARGB8);
		FrameBuffer fb = new FrameBuffer(x, y, 1);
		fb.addColorTexture(target);
		preView.setOutputFrameBuffer(fb);
		
		myRoot = new Node();
		
		Container container = new Container(new SpringGridLayout(Axis.Y, Axis.X, FillMode.None, FillMode.None));
		container.addControl(new CursorEventControl( new PanelListener()));
		container.setPreferredSize(new Vector3f(400, 800, 0));
		container.addChild(new Label("test"), 0, 0);
		container.addChild(new Label("test"), 1, 0);
		container.addChild(new Label("test"), 2, 0);
		container.addChild(new Label("test"), 3, 0);
		container.addChild(new Label("test"), 4, 0);
		container.addChild(new Label("test"), 5, 0);
		container.addChild(new Label("test"), 6, 0);
		container.addChild(new TextField("Editaceeeeeeeeee"), 7, 0);
		container.addChild(new Button("Button"), 7, 1);
		container.addChild(new Label("test"), 8, 0);
		container.addChild(new Label("test"), 9, 0);
		container.addChild(new Label("test"), 10, 0);
		container.addChild(new Label("test"), 11, 0);
		container.addChild(new Label("test"), 12, 0);
		container.addChild(new Label("test"), 13, 0);
		container.addChild(new Label("test"), 14, 0);
		container.addChild(new Label("test"), 15, 0);
		ArrayList<String> items = new ArrayList<String>();
		for(int i = 0; i < 50; i++) {
			items.add("Item in some large list " + i);
		}
		VersionedList<String> model = new VersionedList<>(items);
		ListBox<String> listBox = container.addChild(new ListBox<>(model), 16, 0);
		listBox.setVisibleItems(50);
		listBox.getSlider().removeFromParent();
		container.setLocalTranslation(0, 800, 0);
		myRoot.attachChild(container);
		
		myRoot.updateLogicalState(0);
		myRoot.updateGeometricState();
        
        preView.attachScene(myRoot);
        
        stateManager.getState(MouseAppState.class).addCollisionRoot(preView);
	}
	
	public Geometry createBox1() {
		Box box1 = new Box(10f, 10f, 10f);
		Geometry geom = new Geometry("box1", box1);
		Material box1mat = new Material(getAssetManager(), "Common/MatDefs/Misc/Unshaded.j3md");
		geom.setMaterial(box1mat);
		return geom;
	}
    
    public void initLemur() {
        GuiGlobals.initialize(this);
        BaseStyles.loadGlassStyle();
        GuiGlobals.getInstance().getStyles().setDefaultStyle("glass");
    }
	
	public static Vector3f calculateCenteredPosition(Panel panel, Camera camera) {
		int width = camera.getWidth();
		int height = camera.getHeight();
		Vector3f preferredSize = panel.getPreferredSize();
		float x = (width / 2f) - (preferredSize.x / 2f);
		float y = (height / 2f) + (preferredSize.y / 2f);
		return new Vector3f(x, y, 0);
	}
    
    public static class PanelListener extends DefaultCursorListener {
    	
    	private Vector3f contactPoint;
    	
    	@Override
    	public void cursorMoved(CursorMotionEvent event, Spatial target, Spatial capture) {
    		if(event.getCollision() != null) {
    			contactPoint = event.getCollision().getContactPoint();
    		}
    	}
    	
    	@Override
    	protected void click(CursorButtonEvent event, Spatial target, Spatial capture) {
    		System.out.println(event.getX());
    		System.out.println(event.getY());
    		System.out.println(target);
    		System.out.println(target.getLocalTranslation());
    		System.out.println(contactPoint);
    	}
    	
    }
    

    public static class OffscreenCamera extends Camera {
	
		private Vector2f lowerLeftCorner;
		
		public OffscreenCamera(int width, int height) {
			super(width, height);
		}
		
		public OffscreenCamera(int width, int height, Vector2f lowerLeftCorner) {
			super(width, height);
			this.lowerLeftCorner = lowerLeftCorner;
		}
	
		@Override
		public Vector3f getWorldCoordinates(Vector2f screenPosition, float projectionZPos, Vector3f store) {
	        if (store == null) {
	            store = new Vector3f();
	        }
	 
	        Matrix4f inverseMat = new Matrix4f(viewProjectionMatrix);
	        inverseMat.invertLocal();
	        
	        Vector2f subtract = screenPosition.subtract(lowerLeftCorner);
	        
	        if(subtract.x > getWidth() || subtract.y > getHeight()) {
	        	subtract = new Vector2f(-1,  -1);
	        }
	
	        store.set(
	                (subtract.x / getWidth() - viewPortLeft) / (viewPortRight - viewPortLeft) * 2 - 1,
	                (subtract.y / getHeight() - viewPortBottom) / (viewPortTop - viewPortBottom) * 2 - 1,
	                projectionZPos * 2 - 1);
	
	        float w = inverseMat.multProj(store, store);      
	        store.multLocal(1f / w);
	
	        return store;
		}
	
    }
    
    public static class RangedValueWatcher extends AbstractControl {

    	private VersionedReference<Double> reference;
    	private Consumer<Double> callback;

    	public RangedValueWatcher(RangedValueModel model, Consumer<Double> callback) {
    		reference = model.createReference();
    		this.callback = callback;
    	}

    	@Override
    	protected void controlUpdate(float tpf) {
    		if(reference.update()) {
    			callback.accept(reference.get());
    		}
    	}

    	@Override
    	protected void controlRender(RenderManager rm, ViewPort vp) {}

    }

}
2

I have limited time to drill into the code right this second so in the mean time I will just ask if you rolled this viewport stuff yourself or based it on the Lemur Demo that has a “UI in a ViewPort” example?

You say that text fields don’t work but are buttons at least clickable or is that also not working?

3

I wasn’t aware of the demo but as far as I can tell you don’t do anything special when setting up the viewport and the gui node. The only difference is in using the transparent bucket which I tried and also clearing the buffers of framebuffer which I tried as well.

There is also one more difference and that is I provide my own framebuffer with my own render target which I then use as a background of the panel. This way I render the GUI inside a panel.

To answer your other question, yes, buttons work and so does a list box for example. I did this by tweaking Camera’s getWorldCoordinates a little. The clipping there is a little messy but does the trick for the purpose of the experiment.

One other note. When I put a simple quad with image texture instead of the GUI in the custom viewport, I don’t see any color difference between standard GUI node and the off-screen one. Using standard unshaded material.

4

It’s ok if you don’t have time, I’ll keep playing with it and posting here just for reference :slight_smile:

5

Assuming the above mentioned issues can be fixed, this solution still only partially covers my requirement.

The reason why I have it rendered to a texture is to be able to cover part of it by another dialog for example. But even when covered it would still receive input events. Is it possible to trigger the PickSession to perform input handling for a specific collision root? For example the panel with the rendered texture would be listening for mouse events and somehow triggering the PickSession for the specific collision root?

6

Usually in those cases, you wouldn’t register the viewport as a picking root at all. You’d add a listener to the panel displaying that UI and then forward those events to your real viewport root.

Somewhere I have such a listener but I don’t remember it being very tricky. I think it just creates a PickSession and calls that.

7

Hmm… creating a pick session for that purpose seems as a easy way out. Thanks.

1 Like