My next program is supposed to be a fullscreen application that basically captures the previous contents of my monitor and renders it as texture to an object. For this project, I am currently reviewing various engines and trying to select the most appropriate.
It is not supposed to be a 3D Desktop, more like a simulation enviroment. For example, imagine I am playing for example full-screen tetris on my PC, then I start my program which then displays my tetris game on three floating spheres in some 3D enviroment in real time. Playing the tetris works as normal. If I terminate my program, its simply back to normal full screen tetris.
Obviously, taking screenshots and using those as textures isn’t an option performance wise and I want to use arbitrary applications with this (talyormade programs would be not that difficult, they could simply provide a texture). My desktop, a browser, a game, you name it.
Is jmonkey appropriate? Can this be done with reasonable performance here?
I have a question…how is the tetris going to continue to play (and generate screenshots for you to map) when JME3 has taken over the screen so the tetris has nowhere to render?
If you have OpenGL running in both applications you could try and pass the reference to the texture… Theres a library called “Syphon” that does this on OSX and it works very efficiently. Dunno about a windows equivalent though.
Thats exactly the issue, OpenGL (or whatever) is not necessarily running the displayed application. I simply want to capture the current screen and use it as a texture in my program. This could also be for example my desktop, with firefox and some windows on it.
To answer Zarch’s question, tetris or whatever should be rendered as if it would have the screen but then be used as a texture.
I am looking for a 3D engine that makes this possible in a performant and plattform-independent way.
Look, you cannot expect one application to do stuff to others or even extend the OS’s functionality in such a broad way. These apps have to be able to tell what texture id they use etc. With syphon you can plug it into Java apps or Quartz Composer files or get video out of some video processing applications. Else you have to go with screen-capturing and while I know this works on OSX with hidden windows too (got a nice capturing tool that works like that) I again don’t know how easily thats possible on windows. Since windows 7 now also has an OpenGL context for the UI it might just be that it works there as well.
@vidmaster said:
Thats exactly the issue, OpenGL (or whatever) is not necessarily running the displayed application. I simply want to capture the current screen and use it as a texture in my program. This could also be for example my desktop, with firefox and some windows on it.
To answer Zarch's question, tetris or whatever should be rendered as if it would have the screen but then be used as a texture.
I am looking for a 3D engine that makes this possible in a performant and plattform-independent way.
"should be rendered as if it would have the screen"
How are you going to do that? I can tell you now that it is impossible to do it in a platform independent way. So forget JME3, you need to solve that problem before you can do anything - no matter whether you use JME3 or any other system.
@zarch said:
no matter whether you use JME3 or any other system.
Edit: Oh, you said "platform independent" :D
Capturing a screen in Java:
http://docs.oracle.com/javase/1.5.0/docs/api/java/awt/Robot.html#createScreenCapture(java.awt.Rectangle)
@normen said:
eherm.. :)
Edit: Oh, you said "platform independent" :D
Yep, I'm saying that before he worries about what graphics engine to use he needs to work out how he's going to pipe the output of the other application into his own program. I think he's going to find that very hard unless the program being used is designed to be output like that...
@zarch said:
Yep, I'm saying that before he worries about what graphics engine to use he needs to work out how he's going to pipe the output of the other application into his own program. I think he's going to find that very hard unless the program being used is designed to be output like that...
In OSX its all GL quads, getting the ones for a window and draw them elsewhere works pretty well in the capture tools I have. So on OS level its possible.
Windows 7 as well, you just dont have acces to this. (And linux as well under most distirbutions, they usually redirect the main framebuffer of an application to a own system)
@vidmaster said:
I am looking for a 3D engine that makes this possible in a performant and plattform-independent way.
My other misguided attempts at humor aside, I will pile on to the bandwagon and add that there is no such 3D engine... not only that, it has nothing to do with a 3D engine. You are talking about tapping into the display drivers at a level that is not really available and (as others stated) certainly not available in any cross-platform way (to include each different display card potentially requiring a different way of doing this).
Getting the live image of your monitor is going to be the hardest part. Second hardest part? Getting the live image of your monitor without including your application that is turning it into spheres.
You might be able to trick the OS into rendering the desktop and windows to a separate buffer using a similar approach to what multi-desktop setups use to render a preview window. This will be slow and have many limitations... and be completely OS-specific.
…that said you can try how far you get with the screen capture built into java (nice catch @pspeed)… that would at least cover the “platform independent” part but I doubt its very performant ^^ depending on what you want to do that doesn’t have to be a concern though (say a static web page). Or you have to go the platform-dependent way (at least possible on linux and osx, idk about windows, it changes api’s like I change my underwear).
@normen said:
..that said you can try how far you get with the screen capture built into java (nice catch @pspeed).. that would at least cover the "platform independent" part but I doubt its very performant ^^ depending on what you want to do that doesn't have to be a concern though (say a static web page). Or you have to go the platform-dependent way (at least possible on linux and osx, idk about windows, it changes api's like I change my underwear).
Yeah, it's the "capture the screen without my app that's displaying the screen" part that would be problematic if you wanted live updates.
No live updates, no problem.
My thoughts exactly @pspeed.
It could maybe be done on a dual monitor setup if you ran the capture against one monitor and JME on the other.
Dual monitors would be a possibility but once again, I’d like to avoid that. Live updates on the other hand are necessary.
Currently thinking about using an LWJGL context for the other app, than using this as the texture. Not yet sure if this is possible and it would limit the app to be a Java one, but it is better than nothing.
Make sure you read back and understand everything that was said, if necessary with the help of :google: