No javadoc in com.jme3.terrain.noise.* or documentation about noise or filters

SQLek · May 5, 2014, 12:18am

Hi.

Where can i find documentation for noise filters? I can guess basic principle behind noise bases but i have no idea about Modulator and Filter classes and methods. Javadoc is almost all blank in all classes inside package com.jme3.terrain.noise

Thanks

Sploreg · May 5, 2014, 4:53pm

The guy who made the noise code is gone, and didn’t document it. Sorry =(
I would add docs if I understood noise algorithms more, but as it is I really don’t know much of anything about noise.

SQLek · May 6, 2014, 4:43pm

@Sploreg said: The guy who made the noise code is gone, and didn't document it. Sorry =( I would add docs if I understood noise algorithms more, but as it is I really don't know much of anything about noise.

This sounds like job for me. I'll do a documentation and debug for this subsystem.

Most problematic thing is a range of the noise. Sometimes it is (-1,+1) and sometimes <0,+1).

I think that <0,+1) should be used because it is also native to OpenGl, Blender and java.util.Random.

Diff generated by command git diff – jme3-terrain/src/main/java/com/jme3/terrain/noise/Basis.java. jme3_terrain_noise_Basis_java_0.path - Google Drive . I don’t know anything about gradle and I’m git noob.

Sploreg · May 6, 2014, 9:38pm

Yea the parameters are a nightmare, I really wish I knew what they were. Maybe I will download some white papers on a weekend and read up on it.

I can apply your patch. But for changing the parameter ranges I will also have to change the terrain editor as it uses the noise for the rough tool and I won’t be able to do that until this weekend.

pspeed · May 6, 2014, 11:33pm

@SQLek said: I think that <0,+1) should be used because it is also native to OpenGl, Blender and java.util.Random.

…on the other hand, in cases of terrain generation, 90% of the noise usage will then look something like: (noise - 0.5) * 2 since you actually do want to deflect the value +/-.

I agree pick one or the other… or at least document the heck out of why they are different. But personally, I like my noise functions to return +/- (though I don’t use any of these anyway).

SQLek · May 7, 2014, 12:47am

@pspeed said: ...on the other hand, in cases of terrain generation, 90% of the noise usage will then look something like: (noise - 0.5) * 2 since you actually do want to deflect the value +/-.
I agree pick one or the other… or at least document the heck out of why they are different. But personally, I like my noise functions to return +/- (though I don’t use any of these anyway).

This is main purpose of NoiseModulator. In many places there is return ShaderUtils.clamp(in[0] * 0.5f + 0.5f, 0, 1); and in others return (in[0] - 0.5) * 2;.

I’m considering rewriting terrain subsystem as a 3th party library, and when it will be redy move it to jME. What do You think about it?

– Edit –
I forgot posting whitepapers.

libnoise: Generating coherent noise - how to made noise samples coherent.

http://www.noisemachine.com/talk1/ - presentation of some noise parameters and results.

How to Use Perlin Noise in Your Games – Dev.Mag - explanation how fractal noise works.

This is all i have on hand.

pspeed · May 7, 2014, 1:26am

@SQLek said: This is main purpose of NoiseModulator. In many places there is return ShaderUtils.clamp(in[0] * 0.5f + 0.5f, 0, 1); and in others return (in[0] - 0.5) * 2;.
I’m considering rewriting terrain subsystem as a 3th party library, and when it will be redy move it to jME. What do You think about it?

– Edit –
I forgot posting whitepapers.

libnoise: Generating coherent noise - how to made noise samples coherent.

http://www.noisemachine.com/talk1/ - presentation of some noise parameters and results.

How to Use Perlin Noise in Your Games – Dev.Mag - explanation how fractal noise works.

This is all i have on hand.

I’m all for more plugins. I’m one of the few core members who feels like more of what’s included in “core” should have been a plugin anyway. Terrain is a prime example of something that should/could iterate more often as a plugin. Whether we include it in the download or not is a separate question, in my opinion but I like the idea of more plugins.

jmaasing · May 7, 2014, 8:01am

I’ll chime in with a +1 for ripping terrain and especially the noise stuff out of core (I agree with pspeed, even more things should be plugins).

I remember how many hours I wasted trying to understand the noise lib before giving up and just getting a Perlin implementation from the internet.
So I have my self to blame for those wasted evenings? No I don’t think so, including something in core signalled to me as a newbie that this is a special library that fit jME well for some reason and I should try hard to understand why that is. Turns out that wasn’t the case at all.

normen · May 7, 2014, 10:26am

I don’t think terrain should be taken out of core, the point of jme is that you get a toolkit to make games with. You can remove the terrain from your projects no problem but making it a completely separate plugin isn’t a good idea imo. That said, I do think that the terrain implementation should be much more modular and reusable for other terrain purposes than the currently implemented one.

pspeed · May 7, 2014, 10:37am

@normen said: I don't think terrain should be taken out of core, the point of jme is that you get a toolkit to make games with. You can remove the terrain from your projects no problem but making it a completely separate plugin isn't a good idea imo. That said, I do think that the terrain implementation should be much more modular and reusable for other terrain purposes than the currently implemented one.

To me (as I’ve said a few dozen times now) there is a difference between a plugin that comes with the SDK by default but is still versioned and released separately versus what we have now which is a terrain ‘plugin’ forever directly linked to the version cycle of the SDK. This is a problem because we end up in the “Oh, to get this feature you must run nightly… but oh, you shouldn’t run nightly because it breaks stuff.”

Maybe it’s impossible to do the former. That would be too bad and shows a limitation in how things are packaged… but on the other hand, I don’t think the convenience is worth the downside, really. If step one of the terrain docs has to be “tick this plugin box and update” then so be it. It has the nice side effect that we point directly to the plugin list.

I just somehow believe it must be possible to package some plugins with the SDK by default without having them “core” with the version cycle.

SQLek · May 7, 2014, 12:25pm

@jmaasing said: I'll chime in with a +1 for ripping terrain and especially the noise stuff out of core (I agree with pspeed, even more things should be plugins).
I remember how many hours I wasted trying to understand the noise lib before giving up and just getting a Perlin implementation from the internet.
So I have my self to blame for those wasted evenings? No I don’t think so, including something in core signalled to me as a newbie that this is a special library that fit jME well for some reason and I should try hard to understand why that is. Turns out that wasn’t the case at all.

I know that feel. InfiniteTerrain isn’t usable at all. It gives NANs as a collision, support only 4 types of terrain, only diffuse, no lighting, and generation is just in crap state.

@normen said: I don't think terrain should be taken out of core, the point of jme is that you get a toolkit to make games with. You can remove the terrain from your projects no problem but making it a completely separate plugin isn't a good idea imo. That said, I do think that the terrain implementation should be much more modular and reusable for other terrain purposes than the currently implemented one.

Actually there are two terrains.

One that is finite (probably can be made to infinite), stable, supported by ide, documented, supports lighting, shadows, up to 12 terrain types, normal maps, specular maps, glow maps, physics, vehicles. Main class for it is com.jme3.terrain.geomipmap.TerrainQuad.

There is also second. It only have infinite bounds but: is lacking documentation, is confusing, code have diferent meaning and calling convention, don’t work with anything like bullet or lighting. It’s quality is like junk. Main class of this system is com.jme3.terrain.geomipmap.TerrainGrid.

Currently i working on system that is infinite, is not intrusive, support up to 1024 of texture types (256x4 can be combined to have more diffuse, or to have difuse/specular/glow). It will work with ShaderNodes and as a non intrusive will not break any feature in core.

In my opinion, TerrainGrid soon will be deprecated, not only because of quality of code, but also because there will be better alternative.

–Edit–

GL requirements will be GL ES 2.0 + TEXTURE3D or GL1.5 + GLSL100. Current implementation uses TextureArray because I’m lazy.

normen · May 7, 2014, 12:55pm

@SQLek said: Actually there are two terrains.
One that is finite (probably can be made to infinite), stable, supported by ide, documented, supports lighting, shadows, up to 12 terrain types, normal maps, specular maps, glow maps, physics, vehicles. Main class for it is com.jme3.terrain.geomipmap.TerrainQuad.

There is also second. It only have infinite bounds but: is lacking documentation, is confusing, code have diferent meaning and calling convention, don’t work with anything like bullet or lighting. It’s quality is like junk. Main class of this system is com.jme3.terrain.geomipmap.TerrainGrid.

Currently i working on system that is infinite, is not intrusive, support up to 1024 of texture types (256x4 can be combined to have more diffuse, or to have difuse/specular/glow). It will work with ShaderNodes and as a non intrusive will not break any feature in core.

In my opinion, TerrainGrid soon will be deprecated, not only because of quality of code, but also because there will be better alternative.

–Edit–

GL requirements will be GL ES 2.0 + TEXTURE3D or GL1.5 + GLSL100. Current implementation uses TextureArray because I’m lazy.

TerrainGrid is already deprecated, we suggest not to use it. The code that makes up the base Terrain system (TerrainQuad etc.) is the code I’m talking about which should be more modular.

normen · May 7, 2014, 12:57pm

@pspeed said: I just somehow believe it must be possible to package some plugins with the SDK by default without having them "core" with the version cycle.

Sure but we decided not to do that so the code keeps getting updated. We basically force ourselves to keep things that are in core compatible. If Brent is gone for some month and we make a release in between the terrain will break if its an external plugin.

SQLek · May 7, 2014, 1:59pm

All content of com.jme3.terrain.noise is a part of TerrainGrid. What do with that? Breake all TerrainGrid and rework noise in place, or new version of noise should go into com.jme3.noise or i’ll do R&D, and only interfaces will propagate into core and implementation will be in plugin?

Java gives strong guarantee that changing order of visible methods or adding new methods will not break abi. First class interfaces can be propagated into services. Very nice, simple and lightweight plugins can be made using this approach Currently i have only http://docs.oracle.com/javase/7/docs/api/java/util/ServiceLoader.html on hand because other results are drowning in webserives

normen · May 7, 2014, 2:00pm

@SQLek said: All content of com.jme3.terrain.noise is a part of TerrainGrid. What do with that? Breake all TerrainGrid and rework noise in place, or new version of noise should go into com.jme3.noise or i'll do R&D, and only interfaces will propagate into core and implementation will be in plugin?
Java gives strong guarantee that changing order of visible methods or adding new methods will not break abi. First class interfaces can be propagated into services. Very nice, simple and lightweight plugins can be made using this approach Currently i have only http://docs.oracle.com/javase/7/docs/api/java/util/ServiceLoader.html on hand because other results are drowning in webserives

TerrainGrid is deprecated and can be removed (apart from the noise stuff you want to keep). In what way is it “a part of” TerrainGrid? Afaics the noise classes don’t depend on things from the TerrainGrid but vice versa?

SQLek · May 7, 2014, 3:04pm

TerrainGrid and Noises&Filters are highly coupled by shared buffers Most class implementing Basis have float value(float, float, float) but Filers don’t have it.

That how look implementation of erode perturb filter.
[java]

@Override
public FloatBuffer filter(float sx, float sy, float base, FloatBuffer data, int workSize) {
	float[] arr = data.array();
	int origSize = (int) Math.ceil(workSize / (2 * this.magnitude + 1));
	int offset = (workSize - origSize) / 2;
	Logger.getLogger(PerturbFilter.class.getCanonicalName()).info(
			"Found origSize : " + origSize + " and offset: " + offset + " for workSize : " + workSize + " and magnitude : "
					+ this.magnitude);
	float[] retval = new float[workSize * workSize];
	float[] perturbx = new FractalSum().setOctaves(8).setScale(5f).getBuffer(sx, sy, base, workSize).array();
	float[] perturby = new FractalSum().setOctaves(8).setScale(5f).getBuffer(sx, sy, base + 1, workSize).array();
	for (int y = 0; y &lt; workSize; y++) {
		for (int x = 0; x &lt; workSize; x++) {
			// Perturb our coordinates
			float noisex = perturbx[y * workSize + x];
			float noisey = perturby[y * workSize + x];

			int px = (int) (origSize * noisex * this.magnitude);
			int py = (int) (origSize * noisey * this.magnitude);

			float c00 = arr[this.wrap(y - py, workSize) * workSize + this.wrap(x - px, workSize)];
			float c01 = arr[this.wrap(y - py, workSize) * workSize + this.wrap(x + px, workSize)];
			float c10 = arr[this.wrap(y + py, workSize) * workSize + this.wrap(x - px, workSize)];
			float c11 = arr[this.wrap(y + py, workSize) * workSize + this.wrap(x + px, workSize)];

			float c0 = ShaderUtils.mix(c00, c01, noisex);
			float c1 = ShaderUtils.mix(c10, c11, noisex);
			retval[y * workSize + x] = ShaderUtils.mix(c0, c1, noisey);
		}
	}
	return FloatBuffer.wrap(retval);
}

[/java]

Author cast buffers to array, then back to buffer, then wrap to direct buffer. In some places author uses mix and in others larp but this is the same function. All code looks like quick copy-paste from c++/c with quick hacks for java compatibility. This wasn’t be so big problem, but author wasn’t consistent in methods that he used.

Sploreg · May 7, 2014, 3:39pm

The code is most likely converted from c++ and inefficient and hard to read. But it was in the early stages of development when the author had to leave to deal with family issues so it kind of got dropped.

I am all for getting new noise libs in there. They don’t need to be in core, and a terrian grid should not be in core (hence its deprecation that I will rip out for 3.1). Anything like terrain grid has so many strings attached just to make an endless world exist that including it in core just doesn’t make sense.

Part of the noise libs are used in the terrain editor tool, and could be used elsewhere, but I think having them live in a plugin is ideal. As for terrain itself, I think a default implementation fits in core since a lot of people use it.

We can complain about it all we want, but unless someone has a decent amount of time to sink into rewriting the noise libs they are going to stay there. People do use them, although with some level of difficulty.

SQLek · May 7, 2014, 4:38pm

Can i use package com.jme3.noise for R&D/Alpha stage? Code will be developed outside of trunk.

Also what domain of noise use? [-1,+1] or [0,+1) ? This decision is like big-little-endian, [-1,+1] gives more option in filtering, [0,+1) allow more noise source.

Sploreg · May 7, 2014, 4:42pm

Sure it would be great to have a good/documented noise library, and that package will be fine for that.
Regarding the range, I guess it really depends what the functions actually produce. In some cases I can see [-1,1] working better, but it kind of depends what it is doing.

Thanks @SQLek

sgold · May 7, 2014, 4:59pm

You might want to look also at the 2D Perlin noise generator I wrote for SkyControl: