FillterPostPocessor and Bloom filter in AWTPanel

First off I would like to congratulate the JMonkey creators and contributors on an excellent product. That said, I think it’s time to get the broken AWTPanel to work with FilterPostProcessor and BloomFilter. I am working on a space MMORPG and absolutely require 3D panels on multiple windows / tabs at the same time with post processing filters. I understand that it’s a output buffer issue … specifically around how AWTPanel deals with reshape but isn’t it time to hunker down and fix AWTPanel? I understand it would have to be redesigned but don’t we have enough people that need this feature already? A single screen game design will be going the way of the dinosaur soon and we still don’t have a viable solution for multi-screen post processing. With over 27 years of experience behind me, I am more than willing to contribute my time to help getting this issue resolved but would require a bit more detail around the way the back end works. I certainly can walk through the code base and reverse engineer everything, but it would be faster for someone to either point me to some docs, or spend some time with me to get me up to speed. Of course, someone with more knowledge on the back end could also possibly help us all by creating a working version of AWTPanel that does not have the buffer shortfalls of the current one when it comes to post processing.

I hope there is a solution to this issue.

Zissis

Hm im not sure I can follow in detail. But the issue itself is that you cannot have mutliple AWTPanels simoutlanously while using prosprocessing in them?

No the issue is apparently an old one. Use case is as follows:

Instantiate an AWTPanel
Create a new ViewPort
Attach the Viewport to the AWTPanel (Use true for overwrite buffer because otherwise the system blows up past reshape)
Atach a FilterPostProsessor with a bloom filter to a new ViewPort created for the AWTPanel
Manually run initialize on the FilterPostProcessor to get around the exception on reshape.
Display the AWTPanel.

Results: The FilterPostProcessor will run, it will render an overlayed scene that is zoomed out because of it’s use of Camera width and height, the mouse and key inputs will not work. You must also overload the PostProcessor’s reshape method to constantly call initialize because it is not synched up with the AWT panel’s reshape.

Here are the issues that I have observed:

AWTPanel can not deal with the new output buffer that post processing creates from. The AWT output buffer and the post processing output buffer confuse the AWTPanel.

There is no clean way to synchronize the AWTPanel’s reshape component listener with the filter post processor.

There might be other issues involved as well. I did do my homework before posting and did try the jme_ext_swing alternative, but it also has issues with multiple panels that have post processing enabled.

I have included sample code below that gets fired when creating an AWTPanel within the application. There are alot of proprietary methods in it but you should be able to read through the JME components. If you derive a test case through this code you will see that it will blow up regardless of the bloom state you use.

private AwtPanel createPanel()
{
	panel = SystemFactory.getAwtPanelsContext().createPanel(PaintMode.Accelerated);
	panel.setPreferredSize(new Dimension(400, 300));

	SwingUtilities.invokeLater(new Runnable()
	{

		@Override
		public void run()
		{
			SystemFactory.getJmeApplication().enqueue(new Callable<Object>()
			{

				@Override
				public Object call() throws Exception
				{
					rootNode = new Node("root-" + ((BaseModel) getParent()).getGuidField().getValue());
					if(node != null)
						rootNode.attachChild(node);
					camera = new Camera(SystemFactory.getJmeApplication().getCamera().getWidth(), SystemFactory.getJmeApplication().getCamera().getHeight());
					camera.copyFrom(SystemFactory.getJmeApplication().getCamera());

					flyCamera = new FlyByCamera(camera);
					flyCamera.setMoveSpeed(5);
					flyCamera.registerWithInput(SystemFactory.getJmeApplication().getInputManager());
					flyCamera.setDragToRotate(true);
					flyCamera.setEnabled(false);
					SystemFactory.getJmeApplication().getRootNode().attachChild(rootNode);
					viewPort = SystemFactory.getJmeApplication().getRenderManager()
							.createMainView("v" + ((BaseModel) getParent()).getGuidField().getValue(), camera);
					viewPort.setClearFlags(true, true, true);
					viewPort.attachScene(rootNode);

					panel.attachTo(false, viewPort);
					rootNode.updateLogicalState(60);
					rootNode.updateGeometricState();
					FilterPostProcessor fpp = new FilterPostProcessor(SpatialUtils.getInstance().getAssetManager());
					// fpp.setNumSamples(4);
					fpp.initialize(SystemFactory.getJmeApplication().getRenderManager(), getViewPort());
					BloomFilter bloom = new BloomFilter();
					bloom.setDownSamplingFactor(2);
					bloom.setBlurScale(1.37f);
					bloom.setExposurePower(3.30f);
					bloom.setExposureCutOff(0.2f);
					bloom.setBloomIntensity(2.45f);

					fpp.addFilter(bloom);
					viewPort.addProcessor(fpp);
					firePropertyChanged(VIEWPORT_INITIALIZED_PROPERTY, 0, 1);

					return null;
				}
			});

		}
	});

The result of this so-posed happy path code is:

May 09, 2015 1:59:37 PM com.jme3.app.Application handleError
SEVERE: Uncaught exception thrown in Thread[LWJGL Renderer Thread,5,main]
java.lang.IllegalStateException: Why did you change the output framebuffer?
at com.jme3.system.awt.AwtPanel.postFrame(AwtPanel.java:293)
at com.jme3.renderer.RenderManager.renderViewPort(RenderManager.java:987)
at com.jme3.renderer.RenderManager.render(RenderManager.java:1029)
at com.jme3.app.SimpleApplication.update(SimpleApplication.java:252)
at com.jme3.system.awt.AwtPanelsContext.updateInThread(AwtPanelsContext.java:188)
at com.jme3.system.awt.AwtPanelsContext.access$100(AwtPanelsContext.java:44)
at com.jme3.system.awt.AwtPanelsContext$AwtPanelsListener.update(AwtPanelsContext.java:68)
at com.jme3.system.lwjgl.LwjglOffscreenBuffer.runLoop(LwjglOffscreenBuffer.java:125)
at com.jme3.system.lwjgl.LwjglOffscreenBuffer.run(LwjglOffscreenBuffer.java:151)
at java.lang.Thread.run(Thread.java:745)

Bump. Nobody??

I think that exception was actually removed in 3.1. Its worth a try in any case …

Game is going into beta in a couple of months. 3.1 will not be ready by then. I really need to get filerpostprosessing working with AWTPanels under a stable release.

Well feel free to make a PR.

1 Like

This. And also don’t be shy to try 3.1; I’ve made the switch and never looked back.

2 Likes

So 3.1 is stable enough for me to use with the beta testers? Game will be in beta until the end of the summer.

Wait. If I read it correctly, do your question imply that your beta testers actually WRITE the code of your game?

Believe it or not yes. I write tools that allow them to visually create content and in some cases the tools auto generate late binding code … Yes I am one of those ex IBM AI Labs geeks lol

OK so I switched the the latest 3.1 release and it still has the same problem. Is anyone going to respond to this bug or will we never be able to use a FilterPostProcessor in an AWTPanel?

It was worth trying I guess… I am still not very clear about what the issue is, but I will admit this: The AWT panel system has a lot of issues that require fixing.

So basically there is no way to use a filter post processor with bloom filter in an AWTPanel and I need to have multiple panels in separate windows for my game. Anyone know of an alternate solution that allows us to have multiple panels on multiple windows that will also allow the use of a bloom filter? Or, it there any way of creating bloom effects WITHOUT running through the post processor? A shader maybe?

Create an app without awt panels, make GUI using Nifty or anything else, use viewports or custom rendering pipeline to have multiple ‘windows’.
For GUI there is an option to make something that is almost-awt, based on Picture objects which handles mouse events, with window+controls hierarchy etc. You just need to write some smart code and optimize the texture’s loading process - to fill the existing image instead of creating new one.

Here is an example:

public class GuiBaseWindow extends Picture implements GuiInterface
{

    
    protected static final int TITLE_TOP = 60;
    
    public static final float HEIGHT_HD = 1080f;
    
    private boolean _visible;
    private boolean _normalPaint;
    
    private Texture2D _texture;
    protected BufferedImage _image;
    
    protected final int _width;
    protected final int _height;
    protected float _scale;
    protected final int _group;
    protected final int _zOrder;
    private final boolean _paintStdBack;
    
    protected ArrayList<GuiInterface> _children;
    
    private GuiInterface _underMouse;
    //protected GuiInterface _focusedChild;
    
    //Trzeba tu zrobic funkcje repaint, paint component(g), liste komponentów i ogolnie ogarnąć jak w AWT
    
    public GuiBaseWindow(AssetManager manager, String name, int width, int height, int group, int z)
    {
        this(manager, name, width, height, group, z, false);
    }
    
    
    public GuiBaseWindow(AssetManager manager, String name, int width, int height, int group, int z, boolean painted)
    {
        this(manager, name, width, height, group, z, painted, BufferedImage.TYPE_INT_ARGB, Format.RGBA8);

    }
    
    public GuiBaseWindow(AssetManager manager, String name, int width, int height, int group, int z, 
                         boolean painted, int imageType, Format imageFormat)
    {
        super(name);
        
        _children = new ArrayList<GuiInterface>();
        
        setWidth(width);
        setHeight(height);
        setLocalTranslation(0, 0, (float)z);
        
        _paintStdBack = painted;
        
        _width = width;
        _height = height;
        _group = group;
        _zOrder = z;
        
        _image = new BufferedImage(width, height, imageType); //BufferedImage.TYPE_INT_ARGB);
        
        _texture = new Texture2D(width, height, imageFormat);

        setTexture(manager, _texture, true);
        //applyImage();
        
        _visible = true;
        _normalPaint = true;
    }
    
    protected BufferedImage getMainImage()
    {
        return _image;
    }
    
    public boolean isVisible()
    {
        return _visible;
    }
    
    public void setVisible(boolean val)
    {
        if (val == _visible) return;
        
        _visible = val;
        if (val) repaint();
        
        if (val) setCullHint(CullHint.Inherit);
        else setCullHint(CullHint.Always);
    }
    
    //public void onFocus(GuiInterface child)
    //{
        //if (_focusedChild != child)
        //{
        //    if (_focusedChild != null) _focusedChild.setFocused(false);
        //    
        //    _focusedChild = child;
        //    _focusedChild.setFocused(true);
        //    repaint();
        //}
    //}
    
    //@Override
    //public void setPosition(float x, float y)
    //{
    //    super.setPosition(x, y);
    //    setTransformRefresh();    
    //}
    
    public void setFocused(boolean value)
    {
        if (value == false) return;
        
        //if (_focusedChild != null)
        //{
        //    _focusedChild.setFocused(false);
        //    _focusedChild = null;
        //    repaint();
        //}
    }
    
    public boolean isFocused()
    {
        return false; // Okno zawsze to zwraca
    }
    
    public void setParent(GuiInterface parent)
    {
        //Okna nie maja parenta
    }
    
    public void addChild(GuiInterface child)
    {
        _children.add(child);
        child.setParent(this);
    }
    
    public int getGroup()
    {
        return _group;
    }
    
    public float getScale()
    {
        return _scale;
    }
    
    public int getZOrder()
    {
        return _zOrder;
    }
    
    protected void setNormalPaint(boolean val)
    {
        _normalPaint = val;
    }
    
    public void repaint()
    {
        
        if (!_normalPaint)
        {
            if (_visible) repaint(null);
            
            Image fimg = Util.load(_image, true, _texture.getImage()); //_awtLoader.load(_image, true); 
            //Image fimg = _awtLoader.load(_image, true); 
            _texture.setImage(fimg);
            return;
        }
        
        Graphics2D g = (Graphics2D)_image.getGraphics();
        Util.activateAntiAliasing(g);
        //prepareTextDraw(g);
        
        //czyszczenie
        if (_paintStdBack)
        {
            //g.setColor(Color.GRAY);
            //g.fillRect(0, 0, _width, _height); 
            
            g.setComposite(AlphaComposite.Clear); 
            g.fillRect(0, 0, _width, _height); 
            g.setComposite(AlphaComposite.SrcOver);
            
            //BufferedImage img = ImageFactory.getInstance().BIMG_GUI_CHRCARD;
            //g.drawImage(img, 0, 0, _width, _height, 
            //            0, 48, img.getWidth(), img.getHeight(), null);
            
            int border = 23;
            BufferedImage til = ImageFactory.getInstance().BIMG_WND_TILEABLE;
            
            
            for (int y = border; y <= _height - border; y += til.getHeight())
            {
                int h = til.getHeight();
                if (y + h > _height - border) h = _height - border - y;
                
                for (int x = border; x < _width - border; x += til.getWidth()) 
                { 
                    int r = til.getWidth();
                    if (x + r > _width - border) r = _width - border - x;
                    
                    g.drawImage(til, x, y, x + r, y + h, 0, 0, r, h, null);
                }
                
            }            

            BufferedImage img1 = ImageFactory.getInstance().BIMG_WND_FRAME_T;
            BufferedImage img2 = ImageFactory.getInstance().BIMG_WND_FRAME_B;
            
            for (int x = border; x < _width - border; x += img1.getWidth()) 
            {  
                int r = img1.getWidth();
                if (x + r > _width - border) r = _width - border - x;
                g.drawImage(img1, x, 0, x + r, img1.getHeight(), 0, 0, r, img1.getHeight(), null);
                
                g.drawImage(img2, x, _height - img2.getHeight(), x + r, _height, 
                            0, 0, r, img2.getHeight(), null);
            } 
            
            img1 = ImageFactory.getInstance().BIMG_WND_FRAME_L;
            img2 = ImageFactory.getInstance().BIMG_WND_FRAME_R;
            
            for (int y = border; y < _height - border; y += img1.getHeight()) 
            {  
                int r = img1.getHeight();
                if (y + r > _height - border) r = _height - border - y;
                g.drawImage(img1, 0, y, img1.getWidth(), y + r, 0, 0, img1.getWidth(), r, null);
                
                g.drawImage(img2, _width - img2.getWidth(), y, _width, y + r,  
                            0, 0, img2.getWidth(), r, null);
            }
            
            img1 = ImageFactory.getInstance().BIMG_WND_CORNER_LT;
            g.drawImage(img1, 0, 0, null);
            
            img1 = ImageFactory.getInstance().BIMG_WND_CORNER_RT;
            g.drawImage(img1, _width - img1.getWidth(), 0, null);
            
            
            img1 = ImageFactory.getInstance().BIMG_WND_CORNER_LB;
            g.drawImage(img1, 0, _height - img1.getHeight(), null);
            
            img1 = ImageFactory.getInstance().BIMG_WND_CORNER_RB;
            g.drawImage(img1, _width - img1.getWidth(), _height - img1.getHeight(), null);
        }
        else
        {
            g.setComposite(AlphaComposite.Clear); 
            g.fillRect(0, 0, _width, _height); 
            g.setComposite(AlphaComposite.SrcOver);
        }
        

        
        if (_visible) repaint(g);
        
        g.dispose();
        
        Image fimg = Util.load(_image, true, _texture.getImage()); //_awtLoader.load(_image, true); 
        //Image fimg = _awtLoader.load(_image, true); 
        _texture.setImage(fimg);

    }
    
    public void repaint(Graphics2D g)
    {
        paintComponent(g);
        //GuiInterface foc = null;

        if (g == null) return;
        
        for (GuiInterface gui : _children)
        {
            //if (gui == _focusedChild) continue; //Na koncu!
            if (gui.isFocused()) continue;
            Graphics co = g.create(gui.getPosX(), gui.getPosY(), 
                                   gui.getBaseWidth(), gui.getBaseHeight());
            gui.repaint((Graphics2D)co);
            co.dispose();
        }
        
        for (GuiInterface gui : _children)
        {
            //if (gui == _focusedChild) continue; //Na koncu!
            if (!gui.isFocused()) continue;
            Graphics co = g.create(gui.getPosX(), gui.getPosY(), 
                                   gui.getBaseWidth(), gui.getBaseHeight());
            gui.repaint((Graphics2D)co);
            co.dispose();
        }
        
        //if (_focusedChild != null)
        //{
        //    Graphics co = g.create(_focusedChild.getPosX(), _focusedChild.getPosY(), 
        //                           _focusedChild.getWidth(), _focusedChild.getHeight());
        //    _focusedChild.repaint((Graphics2D)co);
        //    co.dispose();
        //}
    }
    
    public void paintComponent(Graphics2D g)
    {
        
    }
    
    public boolean isActiveArea(float parX, float parY)
    {
        if (!_visible) return false;
        
        Vector3f pos = getLocalTranslation();
        if (parX < pos.x) return false;
        if (parY < pos.y) return false;
        
        if (parX > pos.x + getLocalScale().x) return false;
        if (parY > pos.y + getLocalScale().y) return false;
        
        //TODO: test pixeli, czy klik na transparentnym
        
        return true;
    }
    
    public boolean isAcceptDrag(int locX, int locY)
    {
        return false;
    }    
    
    public void mouseExit()
    {
        if (_underMouse != null) _underMouse.mouseExit();
        _underMouse = null;
    }
    
    public void mouseEnter()
    {
        if (_underMouse != null) _underMouse.mouseExit();
        _underMouse = null;
    }

    public boolean mouseMove(int locX, int locY)
    {
        if (!_visible) return false;
        
        GuiInterface mostUp = null;
        
        for (GuiInterface gui : _children)
        {
            if (locX < gui.getPosX()) continue;
            if (locY < gui.getPosY()) continue;
            if (locX > gui.getPosX() + gui.getBaseWidth()) continue;
            if (locY > gui.getPosY() + gui.getBaseHeight()) continue;
            
            if (mostUp == null) mostUp = gui;
            //if (mostUp == _focusedChild) break;
            if (mostUp.isFocused()) break;
        }
        
        if (mostUp != _underMouse)
        {
            if (_underMouse != null) _underMouse.mouseExit();
            
            _underMouse = mostUp;
            if (_underMouse != null) 
            {
                GuiManager.getInstance().setHint(null);
                _underMouse.mouseEnter();
            }
            
        }
        
        // To prawdopodobnie psuło
        //if (mostUp == null) 
        //{
        //    if (_focusedChild != null)
        //    {
        //        _focusedChild.setFocused(false);
        //        _focusedChild = null;
        //    }
        //    return;
        //}
        
        if (mostUp != null) 
        {
            return mostUp.mouseMove(locX - mostUp.getPosX(), locY - mostUp.getPosY());
            
        }

        return false;
    }
    
    public boolean mouseUp(int locX, int locY, int mb)
    {
        if (!_visible) return false;
        
        GuiInterface mostUp = null;
        
        for (GuiInterface gui : _children)
        {
            if (locX < gui.getPosX()) continue;
            if (locY < gui.getPosY()) continue;
            if (locX > gui.getPosX() + gui.getBaseWidth()) continue;
            if (locY > gui.getPosY() + gui.getBaseHeight()) continue;
            
            //if (mostUp == null) 
            mostUp = gui;
            //if (mostUp == _focusedChild) break;
            if (mostUp.isFocused()) break;
        }
        
        //if (mb == GuiManager.MB_LEFT)
        {
            for (GuiInterface gui : _children)
            {
                if (gui == mostUp) continue;
            
                gui.setFocused(false);
            }            
        }

        if (mostUp == null) return false;
        //{
        //    if (_focusedChild != null)
        //    {
        //        _focusedChild.setFocused(false);
        //        _focusedChild = null;
        //    }
        //    return false;
        //}
            
        return mostUp.mouseUp(locX - mostUp.getPosX(), locY - mostUp.getPosY(), mb);
    }
    
    public int getBaseWidth()
    {
        return _width;
    }
    
    public int getBaseHeight()
    {
        return  _height;
    }
    
    public int getPosX()
    {
        return (int) getLocalTranslation().x;
    }
    
    public int getPosY()
    {
        return (int) getLocalTranslation().y;
    }
    
    /**
     * Perform scale and positioning operations.
     * @param w width
     * @param h height
     */
    public void screenSizeChanged(int w, int h)
    {
        _scale = h / HEIGHT_HD;
        setWidth(_width * _scale);
        setHeight(_height * _scale);    
        
        Vector3f pos = getLocalTranslation();
        if (pos.x + _width > w) pos.x = w - _width;
        //if (pos.y + _height > h) pos.y = h - _height;
        if (pos.y > h) pos.y = h;
    }
    
    public void applyState(int guiState, boolean isGameScene)
    {
        for (GuiInterface g : _children)
        {
            g.applyState(guiState, isGameScene);
        }
    }

        
    // Przerobiony AWT Loader

}

Unfortunately I need real separate windows so that the user can drag them to different monitors. I also need separate panels so that scenes can be rendered in separate tabs in the MDI based gui that can also be floatable and turned into windows. In order to fully understand you can take a look at this video: Massive Frontier planets created by the alpha testers. - YouTube

bump