Android -- (Issue Solving In Progress)

Dardzull · April 3, 2013, 1:33am

Anyone get this to work on Android? I can run in Windows without issue, but Android no.

I keep getting:
[java]
E/AndroidHarness( 869): Exception thrown in Thread[GLThread 93,5,main]
E/AndroidHarness( 869): java.lang.NoClassDefFoundError: tonegod.gui.core.Screen
E/AndroidHarness( 869): at app.Menu.simpleInitApp(Menu.java:97)
E/AndroidHarness( 869): at com.jme3.app.SimpleApplication.initialize(SimpleApplication.java:225)
E/AndroidHarness( 869): at com.jme3.app.AndroidHarness.initialize(AndroidHarness.java:473)
E/AndroidHarness( 869): at com.jme3.system.android.OGLESContext.initInThread(OGLESContext.java:216)
E/AndroidHarness( 869): at com.jme3.system.android.OGLESContext.onSurfaceCreated(OGLESContext.java:187)
E/AndroidHarness( 869): at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1494)
E/AndroidHarness( 869): at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1240)
[/java]

The line that is throwing the error:
[java]
Screen screen = new Screen(app, “Styles/style_map.xml”);
[/java]

Dardzull · April 3, 2013, 3:16am

Ok, I downloaded the source and poked around a little. It seems that the clipboard code is the issue. After commenting out the java.awt.datatransfer.* and code needing it, I was able to run my app on Android.

Not sure how this can be later addressed for android integration.

Dardzull · April 3, 2013, 8:54pm

Ok, for anyone that wants to use this on Android at the moment I have a couple diff patches you might be interested in (and maybe @t0neg0d to apply somehow or in a branch maybe). The first solved my crashing on the emulator (I was using 4.1.2 at the time), and the second one fixed the 1280 error I was getting because the Android TV box i was using does not support Int values for the Index Buffers.

tonegod.gui.core.Screen
[java]

This patch file was generated by NetBeans IDE

It uses platform neutral UTF-8 encoding and \n newlines.

— Screen.java (Yesterday 7:47:22 PM)
+++ Current File
@@ -46,11 +46,11 @@
import com.jme3.texture.Texture;
import java.awt.Cursor;
import java.awt.Toolkit;
-import java.awt.datatransfer.Clipboard;
-import java.awt.datatransfer.ClipboardOwner;
-import java.awt.datatransfer.DataFlavor;
-import java.awt.datatransfer.StringSelection;
-import java.awt.datatransfer.Transferable;
+//import java.awt.datatransfer.Clipboard;
+//import java.awt.datatransfer.ClipboardOwner;
+//import java.awt.datatransfer.DataFlavor;
+//import java.awt.datatransfer.StringSelection;
+//import java.awt.datatransfer.Transferable;
import java.io.IOException;
import java.io.InputStream;
import java.text.ParseException;
@@ -85,7 +85,8 @@
*

@author t0neg0d
*/
-public class Screen implements Control, RawInputListener, ClipboardOwner {
+public class Screen implements Control, RawInputListener//, ClipboardOwner
+{

public static enum CursorType {
POINTER,
@@ -170,7 +171,7 @@
private Node cursorEmitterPlaneNode = new Node(“cursorEmitterPlaneNode”);
private boolean useCursorEffects = false;

private Clipboard clipboard;
+// private Clipboard clipboard;
private boolean clipboardActive = false;

private boolean useTextureAtlas = false;
@@ -840,9 +841,9 @@
*/
public void setClipboardText(String text) {
try {

  	clipboard = Toolkit.getDefaultToolkit().getSystemClipboard();

  	StringSelection stringSelection = new StringSelection( text );

  	clipboard.setContents(stringSelection, this);

+// clipboard = Toolkit.getDefaultToolkit().getSystemClipboard();
+// StringSelection stringSelection = new StringSelection( text );
+// clipboard.setContents(stringSelection, this);
} catch (Exception ex) {
this.clipboardText = text;
}
@@ -854,18 +855,18 @@
*/
public String getClipboardText() {
String ret = “”;

  clipboard = Toolkit.getDefaultToolkit().getSystemClipboard();

  Transferable text = clipboard.getContents(null);

  boolean isText = (text != null &amp;&amp; text.isDataFlavorSupported(DataFlavor.stringFlavor));

```
  if (isText) {
```
```
  	try {
```

  		ret = (String)text.getTransferData(DataFlavor.stringFlavor);

```
  	} catch (Exception ex) {
```
```
  		ret = this.clipboardText;
```
```
  		if (ret == null)
```
```
  			ret = "";
```
```
  	}
```
```
  }
```

+// clipboard = Toolkit.getDefaultToolkit().getSystemClipboard();
+// Transferable text = clipboard.getContents(null);
+// boolean isText = (text != null && text.isDataFlavorSupported(DataFlavor.stringFlavor));
+// if (isText) {
+// try {
+// ret = (String)text.getTransferData(DataFlavor.stringFlavor);
+// } catch (Exception ex) {
+// ret = this.clipboardText;
+// if (ret == null)
+// ret = “”;
+// }
+// }
return ret;
}

@@ -1615,11 +1616,11 @@
keyboardElement = element;
}

@Override
public void lostOwnership(Clipboard clipboard, Transferable contents) {
// System.out.println(“Clipboard failed, switching to internal clipboard.”);
// this.clipboardActive = false;
}
+// @Override
+// public void lostOwnership(Clipboard clipboard, Transferable contents) {
+// // System.out.println(“Clipboard failed, switching to internal clipboard.”);
+// // this.clipboardActive = false;
+// }

// Layout Parser
public void parseLayout(String path, AbstractAppState state) {

[/java]

tonegod.gui.core.ElementQuadGrid.java
[java]

This patch file was generated by NetBeans IDE

It uses platform neutral UTF-8 encoding and \n newlines.

— ElementQuadGrid.java (Yesterday 7:47:22 PM)
+++ Current File
@@ -11,7 +11,7 @@
import com.jme3.scene.VertexBuffer.Type;
import com.jme3.util.BufferUtils;
import java.nio.FloatBuffer;
-import java.nio.IntBuffer;
+import java.nio.ShortBuffer;

/**
*
@@ -20,7 +20,7 @@
public class ElementQuadGrid extends Mesh {
private FloatBuffer verts = BufferUtils.createFloatBuffer(163);
private FloatBuffer coords = BufferUtils.createFloatBuffer(162);

private IntBuffer indexes = BufferUtils.createIntBuffer(336);

private ShortBuffer indexes = BufferUtils.createShortBuffer(336);
private FloatBuffer normals = BufferUtils.createFloatBuffer(16*3);
private FloatBuffer colors = null;

@@ -32,7 +32,7 @@
private float templateZ = 0;
private float imgWidth, imgHeight, pixelWidth, pixelHeight, atlasX, atlasY, atlasW, atlasH;

private int[] templateIndexes = new int[] {

private short[] templateIndexes = new short[] {
0,1,5,5,4,0
};

@@ -93,24 +93,24 @@
// Populate indexes from temple
indexX = 0;
for (int x = 0; x < indexes.capacity(); x += 18) {

  	indexes.put(x, templateIndexes[0]+indexX);

  	indexes.put(x+1, templateIndexes[1]+indexX);

  	indexes.put(x+2, templateIndexes[2]+indexX);

  	indexes.put(x+3, templateIndexes[3]+indexX);

  	indexes.put(x+4, templateIndexes[4]+indexX);

  	indexes.put(x+5, templateIndexes[5]+indexX);

  	indexes.put(x+6, templateIndexes[0]+indexX+1);

  	indexes.put(x+7, templateIndexes[1]+indexX+1);

  	indexes.put(x+8, templateIndexes[2]+indexX+1);

  	indexes.put(x+9, templateIndexes[3]+indexX+1);

  	indexes.put(x+10, templateIndexes[4]+indexX+1);

  	indexes.put(x+11, templateIndexes[5]+indexX+1);

  	indexes.put(x+12, templateIndexes[0]+indexX+2);

  	indexes.put(x+13, templateIndexes[1]+indexX+2);

  	indexes.put(x+14, templateIndexes[2]+indexX+2);

  	indexes.put(x+15, templateIndexes[3]+indexX+2);

  	indexes.put(x+16, templateIndexes[4]+indexX+2);

  	indexes.put(x+17, templateIndexes[5]+indexX+2);

  	indexes.put(x,(short) (templateIndexes[0]+indexX));

  	indexes.put(x+1, (short) (templateIndexes[1]+indexX));

  	indexes.put(x+2, (short) (templateIndexes[2]+indexX));

  	indexes.put(x+3, (short) (templateIndexes[3]+indexX));

  	indexes.put(x+4, (short) (templateIndexes[4]+indexX));

  	indexes.put(x+5, (short) (templateIndexes[5]+indexX));

  	indexes.put(x+6, (short) (templateIndexes[0]+indexX+1));

  	indexes.put(x+7, (short) (templateIndexes[1]+indexX+1));

  	indexes.put(x+8, (short) (templateIndexes[2]+indexX+1));

  	indexes.put(x+9, (short) (templateIndexes[3]+indexX+1));

  	indexes.put(x+10, (short) (templateIndexes[4]+indexX+1));

  	indexes.put(x+11, (short) (templateIndexes[5]+indexX+1));

  	indexes.put(x+12, (short) (templateIndexes[0]+indexX+2));

  	indexes.put(x+13, (short) (templateIndexes[1]+indexX+2));

  	indexes.put(x+14, (short) (templateIndexes[2]+indexX+2));

  	indexes.put(x+15, (short) (templateIndexes[3]+indexX+2));

  	indexes.put(x+16, (short) (templateIndexes[4]+indexX+2));

  	indexes.put(x+17, (short) (templateIndexes[5]+indexX+2));
  	indexX += 4;
  }

[/java]

wezrule · April 3, 2013, 9:24pm

No IntBuffer or Awt stuff naughty t0neg0d!!

nice investigation Dardzull. How do you find the GUI on Android? does it seem to respond correctly to events?

Dardzull · April 3, 2013, 10:07pm

I’m slowly working my way through it. Panels and Labels are working. I’m looking into why dragging a window isn’t working. I can resize, but the dragging of an entire window is not working. After that I was going to work on buttons. Once I get to TextBoxes, I can look at the clipboard (if no one has by then).

Overall, I haven’t gone to far into making a GUI on Android. One step at a time

t0neg0d · April 4, 2013, 5:54am

How did I miss this topic?

This is good stuff. Any way to detect android usage and disable the clipboard? Or does this just need to be removed?

I guess alternatively a branch for Android could be created.

Suggestions?

I haven’t even begun to think about Android usage yet. So… no testing of any sort =( yet.

t0neg0d · April 4, 2013, 6:07am

ElementQuadGrid.java has been updated to use a ShortBuffer.

Downloading the emulator to see if I can help this effort along.

Dardzull · April 4, 2013, 10:21pm

@t0neg0d

Personally I’m not sure how useful the clipboard would be on Android (except for maybe a tablet), but could be very useful for a PC based game. An idea might be to possibly move the clipboard code into a sub-class. Then add the option to enable the clipboard, so if the sub-class is not null then return the value else return a blank (like my dinky patch).

t0neg0d · April 5, 2013, 4:15am

@Dardzull said: @t0neg0d
Personally I’m not sure how useful the clipboard would be on Android (except for maybe a tablet), but could be very useful for a PC based game. An idea might be to possibly move the clipboard code into a sub-class. Then add the option to enable the clipboard, so if the sub-class is not null then return the value else return a blank (like my dinky patch).

This sounds like a valid option to me. I’ll work on this tonight

nehon · April 5, 2013, 9:10am

there is a clipboard support for android
http://developer.android.com/guide/topics/text/copy-paste.html

Maybe you could use the awt clipboard for desktop and delegate to the android clipboard system for android.

wezrule · April 5, 2013, 11:01am

says it requires API level 11, which is Android 3.0, are we still trying to make it Android 2.2 Compatible?

nehon · April 5, 2013, 11:44am

@wezrule said: says it requires API level 11, which is Android 3.0, are we still trying to make it Android 2.2 Compatible?

mhh right. Well yes, our target is android 2.2 and later, but considering android 2.2 is about 4% of the market, I guess we can target 2.3 (i think it's API level 9).

Dardzull · April 5, 2013, 3:46pm

Maybe the Android Gurus (@nehon, @wezrule) can point me in the right direction for this.

I hit a little wall. I have a feeling that the issue is with the AndroidHarness (stock RC2 version) vs the GUI, but when using the Menu element I get the following (works fine in an Applet and raw JME3):

[java]
W/dalvikvm( 713): threadid=12: thread exiting with uncaught exception (group=0x40a13300)
E/AndroidHarness( 713): Exception thrown in Thread[GLThread 85,5,main]
E/AndroidHarness( 713): java.lang.ClassCastException: com.jme3.math.Vector4f cannot be cast to com.jme3.math.Quaternion
E/AndroidHarness( 713): at com.jme3.renderer.android.OGLESShaderRenderer.updateUniform(OGLESShaderRenderer.java:657)
E/AndroidHarness( 713): at com.jme3.renderer.android.OGLESShaderRenderer.updateShaderUniforms(OGLESShaderRenderer.java:709)
E/AndroidHarness( 713): at com.jme3.renderer.android.OGLESShaderRenderer.setShader(OGLESShaderRenderer.java:890)
E/AndroidHarness( 713): at com.jme3.material.Material.render(Material.java:1058)
E/AndroidHarness( 713): at com.jme3.renderer.RenderManager.renderGeometry(RenderManager.java:525)
E/AndroidHarness( 713): at com.jme3.renderer.queue.RenderQueue.renderGeometryList(RenderQueue.java:301)
E/AndroidHarness( 713): at com.jme3.renderer.queue.RenderQueue.renderQueue(RenderQueue.java:350)
E/AndroidHarness( 713): at com.jme3.renderer.RenderManager.renderViewPortQueues(RenderManager.java:790)
E/AndroidHarness( 713): at com.jme3.renderer.RenderManager.flushQueue(RenderManager.java:721)
E/AndroidHarness( 713): at com.jme3.renderer.RenderManager.renderViewPort(RenderManager.java:985)
E/AndroidHarness( 713): at com.jme3.renderer.RenderManager.render(RenderManager.java:1037)
E/AndroidHarness( 713): at com.jme3.app.SimpleApplication.update(SimpleApplication.java:251)
E/AndroidHarness( 713): at com.jme3.app.AndroidHarness.update(AndroidHarness.java:485)
E/AndroidHarness( 713): at com.jme3.system.android.OGLESContext.onDrawFrame(OGLESContext.java:332)
E/AndroidHarness( 713): at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1516)
E/AndroidHarness( 713): at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1240)
[/java]

Test Code:
[java]
import com.jme3.app.SimpleApplication;
import com.jme3.input.event.MouseButtonEvent;
import com.jme3.math.ColorRGBA;
import com.jme3.math.Vector2f;
import com.jme3.system.AppSettings;
import tonegod.gui.controls.buttons.ButtonAdapter;
import tonegod.gui.core.Screen;
import tonegod.gui.controls.menuing.Menu;

public class AppletDisplay extends SimpleApplication
{

static AppletDisplay app;
Screen screen;
static Menu menu;

public static void main(String[] args)
{
AppSettings settings = new AppSettings(true);
settings.setFrameRate(60);

settings.setHeight(720);
settings.setWidth(1280);

app = new AppletDisplay();

app.setShowSettings(false);
app.setDisplayStatView(false);
app.setDisplayFps(true);

app.setPauseOnLostFocus(false);
app.setSettings(settings);
app.start();
}

@Override
public void simpleInitApp()
{
this.getViewPort().setBackgroundColor(ColorRGBA.Black);

flyCam.setEnabled(false);
inputManager.setCursorVisible(true);

screen = new Screen(this);
guiNode.addControl(screen);

Menu subMenu = new Menu(
	screen,
	"sub",
	new Vector2f(0, 0),
	false)
{
    @Override
    public void onMenuItemClicked(int index, Object value, boolean isToggled)
    {
    }
};
    // Add a menu item
subMenu.addMenuItem("Some string caption 1", null, null);
    // Add a toggle-able menu item (checkbox)

// subMenu.addMenuItem(“Some string caption 2”, null, null, true);
// Add a toggle-able menu item and set the default state of the checkbox to checked
// subMenu.addMenuItem(“Some string caption 3”, null, null, true, true);
screen.addElement(subMenu);

menu = new Menu(
	screen,
	"main",
	new Vector2f(0, 0),
	false)
{
    @Override
    public void onMenuItemClicked(int index, Object value, boolean isToggled)
    {
    }
};
    
    // Add subMenu as a sub-menu to this menu item
menu.addMenuItem("Some caption", null, subMenu);
screen.addElement(menu);

ButtonAdapter b = new ButtonAdapter(screen, "button", new Vector2f(50, 50))
{
    @Override
    public void onButtonMouseLeftUp(MouseButtonEvent evt, boolean isToggled)
    {
            menu.showMenu(null, getAbsoluteX(), getAbsoluteY() - menu.getHeight());
    }
};
b.setText("Show Menu");
screen.addElement(b);
 
}

}
[/java]

t0neg0d · April 5, 2013, 5:27pm

Hmmm… this is odd as there is no use of Quaternions (rotation in parallel projection only works along the z axis --in the case of UI’s anways-- and rotation against BitmapText doesn’t work at all due to the way BitmapText is rendered)

@nehon Is there any way to update the Material expections to say what material and what parameter is the problem area? This Exception’s stack trace is largely useless without those bits of info in the message.

EDIT: Oh crap… I just read the exception again. I need to check the source, but it looks like the android specific material calls are assuming any Vector4 is a Quaternion. This may be an issue =)

nehon · April 5, 2013, 6:31pm

This issue has already been fixed in nightly.

Erebos3D · September 2, 2013, 11:02am

Although this topic is already 5 months old, I prefer to continue it instead of opening a new one.

I am trying to get the GUI running on Android.

I get no error messages, but only texts are displayed. Not background images, even if I specify them myself.

I am using the recent source from googlecode and only adapted the change about the Clipboard.
@dardzull, you say panel and windows would be working, at least partially. Have you changed anything else in tonegod’s code?

In general I like the idea of a seperate branch for Android, and I would like to contribute to it.

Dardzull · September 3, 2013, 7:13am

@Erebos3D

I have had to put this on the back burner for the last several months (annoying life stuff), but the changes I posted helped fix some of the issues I ran into running on 4.1.2 of Android. I was able to get a background to show up on a Android-TV box, but I was not able to get the drag-drop or the menus to work at that point. I have been waiting for the downloaded lib to be updated before I started doing any major development currently, but I’m getting the feeling that I’m going to have to breakdown and just start raiding the SVN vs. using the plugin.

Are you using the emulator for testing or is this on a specific phone/table/etc? I have run across a lot of devices that use the ARM chips liking to take short cuts on the video by forcing 2X textures (i.e. the textures need to be square 4x4px, 8x8px, 16x16px, etc etc ). Although I have not played with any other chip sets, I would guess that the others will have this issue at times also.

Erebos3D · September 3, 2013, 10:38am

Okay, now with 2X textures the display works. Seems like I have to change the size of every standard background texture.
But I can see a strange pattern in the textures which does not appear on my PC.

Screenshot

I am compiling with Android 2.3.3 and testing on the old Nexus 7 with Android 4.3. Did you see anything similar @Dardzull ?

As you said, resizing works, but not moving. I will have a look into that.

EDIT:
I found the reason for not moving, played a bit around and now moving works, too. But it is still buggy.

Dardzull · September 3, 2013, 5:49pm

@Erebos3D

I would advise you to look into using the Texture Atlas feature that @t0neg0d put in. It should (i have not tested this yet) allow you to make a single square texture then pull off the parts you need, bypassing the need to resize all your textures.

I have not tried using the lib on anything older than 4.1.2, but on my Android-TV box everything looked the same as my PC.

t0neg0d · September 3, 2013, 8:12pm

<cite>@Dardzull said:</cite> @Erebos3D
I would advise you to look into using the Texture Atlas feature that @t0neg0d put in. It should (i have not tested this yet) allow you to make a single square texture then pull off the parts you need, bypassing the need to resize all your textures.

I have not tried using the lib on anything older than 4.1.2, but on my Android-TV box everything looked the same as my PC.

Definitely… I’ll be replacing the default texture for atlasing soon with a 128x128 image excluding the color wheel atlas image… this will be a separate atlas for optional use. I have made quite a few changes to atlasing accounting for overriding the default atlas with secondary atlases… so, button icons can be pulled from a complete different source than… etc, etc, etc. (I hope that made sense).

At the moment, I am still unable to set up the emulator for testing, so I have yet to update the library with any of these changes aside from the IntBuffer issue.

EDIT: Guess I should explain what I have been up to with the library. Putting it to practical use! This has helped in tracking down issues I probably would not have found otherwise. Example: Setting the visual state of a Button or AbstractButtonvia the EffectManager will no longer crash your application when atlasing is enabled Most other updates are a bit more subtle. lol