Splitting Examples from Tests

Development
#1

Referring to the task documented here:
https://github.com/jMonkeyEngine/jmonkeyengine/issues/1290

Since this is a pretty big reorganization… I felt it’d be worth running the plan by everyone before starting anything, to ensure our understanding of the goals are in agreement, and answer a few questions I’ve come up with.

Steps:

  1. Attempt to determine whether each “Test/Example” in jme3-examples was truly intended to be a test, or an example to be referenced by “end-developers”.

Most of the tests seem to have the word “test” in them somewhere (shocking, I know) so that’ll be a good starting point. If any “tests” seem to have been intended more as an example (or as a test and an example), I’ll bring those to everyone’s attention and we can decide what to do with those.

  1. Reorganize project:
  • Demos, tutorial code, etc. remain in jme3-examples under a new package named jme3example
  • Tests move to new project jme3-tests under the (existing) package jme3test
  • Dependencies which aren’t tests (but are used by tests) get moved into sub-packages (for example, RayTrace.java in the collision package, PhysicsTestHelper, or MovingAverage.java inside network)
  • Cleanup/remove any junk (empty “Main.java” class in there, so that’s weird)
  1. Since examples is being split into two different projects, something would need to be done with the TestChooser class.

Do we modify it to support both tests and examples somehow? (They’re in different sub-projects…) My feeling is that since there seems to be far more “tests” than examples, it should continue to only focus on tests. Examples would be standalone applications with main() methods which can run independently (ex. from IDE).

So basically if we go that route, TestChooser would be moved to jme3-tests, executing tests in the jme3test package. Since it relies on reflection and the package name remains the same for “tests”, I’m hoping it won’t require any real code updates. I could also do some cleanup of TestChooser’s code and bring the formatting more to the informal(?) standard, with a NetBeans auto-format.

Not sure if I’ll get to test automation or not, but whoever works on it, that can come after.

How does this sound? Awaiting concerns, feedback, modifications, requests, hate mail, etc.

3 Likes
#2

First of all: Thank your for helping us cleaning up the engine :smiley:

Step 2:
Personally, I don’t like this “non canonical” package names like jme3test, jme3tools and jme3examples, they could also be com.jme3.tests (testing?) and com.jme3.examples.

Also take a note at PR #1298, you can run it to see all the issues with the current examples, currently it’s a few bad uses of loggers, "possible null pointer dereference of heightmap in x on exception path, toString instead of Arrays.toString on TestCloner$ArrayHolder and lots of unused fields etc.

Step 3:
Why not have both in the Chooser? Maybe with a TabbedPanel?
If not, I’d go for examples, because tests would be run automatically using gradlew check or gradlew test.

Note: If you want to go the extra mile, you could mark/note a few tests/examples that look like their output could be compared very well and report it to me at MeFisto94/jme3-testing, see for instance here, that’s how PBR Rendering is supposed to look like and we compare the output.
Having such things for Parallax and whatnot would be awesome to have :slight_smile:

And again, thanks for helping out :slight_smile:

1 Like
#3

And remember, a lot of these have “Test” in the name but they are not tests or they straddle the line between a test and an example. Or like TestJoystick are not a test but an example+diagnostic tool.

Personally, I’d err on the side of “this is probably an example named test” because a large percentage of those test classes were written that way. Dev thought “I’m writing this new feature and I need some code to try it out and be a minimal example”.

We may want to consider a third name in these “Demo” for a lot of the cases like described above. As in, it’s demonstrating a particular function but not an automated test and not really a ‘best practices’ style example… but someone could still use it as such.

Names are weird.

2 Likes
#4

Thank you for taking the time to do this!

I have to agree with @Darkchaos on the “non canonical” naming.

1 Like
#5

Totally agree on the package naming using com.jme3, I should have caught that!

So then maybe com.jme3.test and com.jme3.example. Singular package naming looks like the more common convention. Once I start going through things I’ll keep everyone posted, and we can decide if a 3rd demos project/package is warranted (or they could share the jme3-examples project), for those examples that are very focused on a particular feature.

Tabs work too, but I was going to suggest a radio button which acts as a filter between tests and examples. Then I realized it might be very lopsided (ex. 90% tests) but more importantly, which project would the chooser belong in? Would it still be called “TestChooser”? So it kind of opened up more questions. Open to whatever suggestions.

It will take some time going through these and trying to divine the intent; let me know if I’m holding up any planned “Spring cleaning” etc.

1 Like
#6

Any suggestions on what we should do with tests that also work as examples, or what basis should be used to differentiate between the two?

For a concrete example, consider jme3test.animation.TestCinematic. Appears to have been used as a “test” during development for the Cinematic class, but it also looks useful as an example, to quickly understand how to use the Cinematic class. Maybe we should default to “example”, especially since we want to do test automation (but some of these more visual ones may not be possible to fully automate).

#7

To me “test” means automated… but I guess I’m the minority there.

Almost all of the current “tests” are also examples. See my earlier comment.

3 Likes
#8

Oh, you beat my edit where I added that :slight_smile: : “Maybe we should default to “example”, especially since we want to do test automation (but some of these more visual ones may not be possible to fully automate).”

Yeah, so I guess we’re going to end up with many more examples than tests, after all.

#9

More of them than not are demonstrations of some feature, sometimes in as bare-bones a way as possible. We may have used them to test that feature but it doesn’t change the fact that they are examples (maybe not a full game, best-practice example) of using that feature.

As a core developer, even I, when trying something new, look at the “tests” to see how I should do it.

#10

I’d like to see jME get some honest to goodness tests, although outside of core classes unrelated to rendering that’s tough to do.

#11

It might be beneficial to make some of these more “example” type tests into real automated tests. Ogre3d has a visual testing system for this. They do image comparisons to see whats changed between versions. If anything, it might be helpful to see a quick verification that “yeah… everything seems to be rendering correctly” before a new major release. It’s just opening each test then taking a screen shot, but automated.

If anyone’s interested in this contact me, I’d be willing to do the web viewer part.

2 Likes