Decal Generator

Since Projective Texture Mapping has it’s limits and drawbacks, I started working on a Decal Generator. Here’s a demo:

[video]http://www.youtube.com/watch?v=NVfulkXCcO4[/video]

Please note the changing triangle / vertex count and the FPS vs. “DPS” rate at complex target geometry.

Features so far:

  • perspective and parallel projection
  • backface culling with culling angle
  • tangent generation -> supports bump and parallax mapping
  • generator can run in a separate pool thread
  • runs at interactive frame rates with target geometry like a 256x256 sphere
  • simple texture atlas and batching functionality (one material, one mesh, many bullet holes)

It was inspired by Wolfire. I also find the Unity Decal System from Edelweiss Interactive extremely nice. That’s why I’ll try to create a JMP / GDE plugin for static decal placement (and maybe baking). I’m also planning to combine the Projective Texture Mappig plugin and the Decal Generator in a new “Decorations” plugin / library. No idea if I actually get it done but at least it’s a goal. :slight_smile:

One question: I need to be aware of target spatial changes to update the decal mesh. I don’t want to update the decal mesh in every update loop cycle even if nothing has changed. As far as I can see, the only way to reliably get notified of changes (refreshFlags) is to be the parent node of the respective spatial. Or did I miss something?

19 Likes

+1 for the decal generator… and +3 for the awesome music in the vid >.<

@survivor:
It’s awesome like always!!!

I did one in the past which can generate static decal for the good of level design. I was also inspried by series of Wolfire articles.

One thing I still concerned in the scenerio was Wolfire use Deferered rendering and we use Forward one. so the blending can face some obvious issues.
That lead to another thing: caculate the offset-depth to add to the mesh in order to prevent bad blending. I implemented a method which described in a Game Programing Gem book. From what I saw in your method may be you are not implemented blending yet, or did not show it yet… Is it too dumb to ask for another video with blending on
XD

But I think this kind of plugin will save a lot of artist life in level making. Thank a ton!! XD

@survivor
Thumbs up! Awesome work!!! Music it the icing on the cake!

Sweet work dude =) This is gonna be a great plugin!

This is awesome.
I’m looking forward to this plugin!!!

@survivor said: One question: I need to be aware of target spatial changes to update the decal mesh. I don't want to update the decal mesh in every update loop cycle even if nothing has changed. As far as I can see, the only way to reliably get notified of changes (refreshFlags) is to be the parent node of the respective spatial. Or did I miss something?
You should use a custom control. Just check if the "original" spatial has been transformed if you don't want to update on each frame.

@atomix: What exactly do you mean by blending? The decal geometry can use any material JME3 supports, including alpha maps. Does that solve your blending requirement? Or do you mean the decal should not be there immediately but blended in an out of the scene. Like if there are too many bullet holes, blend out the oldest one when adding a new one. That would require to keep track of the start indices and size of the components of the batched decal geometry mesh. One could then remove the oldest mesh and a put it in a temporal geometry to blend it out. I guess I need to keep track of the indices anyway if I don’t want to rebatch everything if only one target has changed.

Edit: Or do you mean normal map blending as described here? Hmmm, taking into account the normal map of the target geometry requires knowledge of the target geometry’s material (i.e. the name of the normal map). It is absolutely possible to use the normals from the normal map of the target geometry and blend it with those of the decal geometry normal map. Notated on the todo list. :smiley:

@nehon: Ah, ok, then I have to save the transforms of all target geometry from the last frame and compare it to those of the current frame. If the decal projector and the target geometry are children of the same node, it’s the local transform. And so on. Thanks. I’ll try that.

1 Like

That vid is full of win. Congrats!

Wow, nice this looks really great.

So easy win with this song. Good sense of humor hahahaha

@survivor said: @atomix: What exactly do you mean by blending? The decal geometry can use any material JME3 supports, including alpha maps. Does that solve your blending requirement? Or do you mean the decal should not be there immediately but blended in an out of the scene. Like if there are too many bullet holes, blend out the oldest one when adding a new one. That would require to keep track of the start indices and size of the components of the batched decal geometry mesh. One could then remove the oldest mesh and a put it in a temporal geometry to blend it out. I guess I need to keep track of the indices anyway if I don't want to rebatch everything if only one target has changed.

Edit: Or do you mean normal map blending as described here? Hmmm, taking into account the normal map of the target geometry requires knowledge of the target geometry’s material (i.e. the name of the normal map). It is absolutely possible to use the normals from the normal map of the target geometry and blend it with those of the decal geometry normal map. Notated on the todo list. :smiley:

I meant 2 , but why not both! :amused: Yeah, in fact, I also read the article from Wolfire blogs for hundreds of times to try to understand the math behind it, then the blending thingy. Poor me back that time, I just had a basic knowledge of OpenGL. I struggled my self to understand the normal and diffuse map blending.

For example, your moss blend into the rock with a threshold … like this:
http://www.m4x0r.com/blog/2010/05/blending-terrain-textures/
Also found the Depth offset article i said in the previous post:
Applying Decals to Arbitrary Surfaces in Game Programming Gems 2
Eric Lengyel <= You can find the brief here

That was like 2 year ago, now I almost forgot everything I done with this problem :stuck_out_tongue:

Anyway, glad to see you if you added it into your todo list! :amused:

Ah, one more thing I remembered that I always want with Decal cause I did a lot of Moss and Ivy in my scenes is: the ability to optimize the decal mesh into simplier one if I need to in real-time. Because after all, ivy (some other kind of Decals) doesn’t have to be perfectly lying on the surface. And because it can have separated UV, it also can have different topology than the orginal mesh.That’s what artists do a lot in 3D Editor to reduce the size of the procedured decal, in fact, it WAS a valuable trick for 3d artists at game company at that time. In fact, I did implemented that feature with QuickHull at that time but i wasn’t good enough, a lot of things went wrong especially the UV. Don’t know if this give you any idea for improvement.

Great job man.

Short update:
I’m likely to sink my teeth into the meaty details before getting it actually done (the opposite of “first make it work, then make it fast”). :smiley: So I tried to get moar dps (decals per second) out of it by optimizing multi-threading. The cropping is per triangle which are independent from each other. So I can easily spread the triangles from the input mesh across all available CPU threads. But getting a benefit wasn’t as easy as I thought.

What I’ve learned so far:

  1. avoid creating new temporary objects
  2. synchronized(buffer) { buffer.put(value) } is a bad idea
  3. avoid copying data

Point 1 is solved by something like jme3’s TempVars. Point 2 is solved by not synchronizing but combining the partial results afterwards. Point 3 is the biggest problem. Since I don’t know the final size of the buffer, it is inefficient to use buffers since growing a buffer means reallocating and copying data. Using ArrayList has the same problem since it copies on grow and there’s also the object overhead (unfortunately, Java doesn’t support primitives as generics).

I came up with the solution of writing my own “DynamicFloatArray” which doesn’t copy on grow but uses a pyramid of float[]. My first attempt was to implement it as a custom java.nio.FloatBuffer, but then I realized the package private constructor and that adding classes to java.nio package is not allowed. For that reason, I have to copy from the DynamicFloatArray to the FloatBuffer of the decal mesh (reusing the buffer if possible). I guess I can’t convince the devs to switch to Apache Mina Buffers? :smiley:

Anyway, I might have missed something that could give me an additional boost. Therefore, I’ll append my DynamicFloatBuffer for review (see the add() method). I’m able to utilize my quad core up to 80% and increase the decals update rate by factor 2.2 compared to single threaded.

I’ve also tested the MemoryUtils of @pspeed. All three values are steadily growing while I run my decals test which updates a decal mesh as fast as possible. Does that mean I have some kind of memory leak? The memory usage in the task manager is stable (thanks to TempVars and buffer reusing) and I’m not using direct buffers as far as I know.

Edit: The java bbcode eats “lower than” and “greater than”. The ArrayList is generic of type float[].

Edit: The java bbcode also eats inner classes. I’ll try to post it different.

[java]
package decals;

import com.jme3.util.BufferUtils;
import java.nio.FloatBuffer;
import java.util.ArrayList;

/**
*

  • @author survivor
    */
    public class DynamicFloatArray
    {
    private int length;
    private int currentIndex;
    private int currentArrayIndex;
    private float[] currentArray;
    private ArrayList arrays;

public DynamicFloatArray()
{
this(16);
}

public DynamicFloatArray(int baseCapacity)
{
this.arrays = new ArrayList();
this.arrays.add(new float[baseCapacity]);
this.reset();
}

public DynamicFloatArray add(float value)
{
this.length++;
this.currentIndex++;
if (this.currentIndex >= this.currentArray.length)
{
this.currentIndex = 0;
this.currentArrayIndex++;
if (this.currentArrayIndex > 1);
this.currentArray = new float[newCapacity];
this.arrays.add(this.currentArray);
}
}

this.currentArray[this.currentIndex] = value;
return this;

}

public float get(int index)
{
int currIdx = index;
int currArrIdx = 0;
float[] currArr = this.arrays.get(currArrIdx);
while (currIdx >= currArr.length)
{
currIdx -= currArr.length;
currArrIdx++;
currArr = this.arrays.get(currArrIdx);
}

return currArr[currIdx];

}

public final DynamicFloatArray reset()
{
this.length = 0;
this.currentIndex = -1;
this.currentArrayIndex = 0;
this.currentArray = this.arrays.get(0);
return this;
}

public DynamicFloatArray clear()
{
this.reset();
this.arrays.clear();
this.arrays.add(this.currentArray);
return this;
}

public FloatBuffer toFloatBuffer()
{
return this.toFloatBuffer(null);
}

public FloatBuffer toFloatBuffer(FloatBuffer result)
{
if (result == null || result.remaining() < this.length)
{
result = BufferUtils.createFloatBuffer(this.length);
}

int currIdx = 0;
int currArrIdx = 0;
float[] currArr;

while (currArrIdx = this.currArr.length)
  {
    this.currIdx -= this.currArr.length;
    this.currArrIdx++;
    this.currArr = arrays.get(this.currArrIdx);
  }

  this.pos++;
  return this.currArr[this.currIdx++];
}

public boolean hasNext()
{
  return this.pos = this.currArr.length)
  {
    this.currIdx -= this.currArr.length;
    this.currArrIdx++;
    this.currArr = arrays.get(this.currArrIdx);
  }

  this.pos++;
  return this.currArr[this.currIdx++];
}

public boolean hasNext()
{
  return this.pos = this.currArr.length)
  {
    this.currIdx -= this.currArr.length;
    this.currArrIdx++;
    this.currArr = arrays.get(this.currArrIdx);
  }

  this.pos++;
  return this.currArr[this.currIdx++];
}

public boolean hasNext()
{
  return this.pos &lt; length;
}

}
}

[/java]

public int getLength()
{
return this.length;
}

public Enumerator getEnumerator()
{
return new Enumerator();
}

public class Enumerator
{
private int pos;
private int currIdx;
private int currArrIdx;
private float[] currArr;

private Enumerator()
{
  this.reset();
}

public final void reset()
{
  this.pos = 0;
  this.currIdx = 0;
  this.currArrIdx = 0;
  this.currArr = arrays.get(this.currArrIdx);      
}

public float next()
{
  if (this.currIdx &gt;= this.currArr.length)
  {
    this.currIdx -= this.currArr.length;
    this.currArrIdx++;
    this.currArr = arrays.get(this.currArrIdx);
  }

  this.pos++;
  return this.currArr[this.currIdx++];
}

public boolean hasNext()
{
  return this.pos &lt; length;
}

}
}

Has this been released as a plugin? I could really use this for bullet decals in my game.

@survivor, growing memory is not necessarily a memory leak, does it gets you and OOM when you reach the max?

Also about mina buffers, did you see the warnings on the page you linked?

@survivor said: I'm not using direct buffers as far as I know.
You do actually, BufferUtils.createFloatBuffer(this.length) creates a direct buffer.

About growing buffers, did you have a look at Guava? I don’t know if they have such thing, but this lib has very handy tools, i wouldn’t be surprised that something like this already exists.

@survivor said: I've also tested the MemoryUtils of @pspeed. All three values are steadily growing while I run my decals test which updates a decal mesh as fast as possible. Does that mean I have some kind of memory leak? The memory usage in the task manager is stable (thanks to TempVars and buffer reusing) and I'm not using direct buffers as far as I know.

As nehon points out, BufferUtils creates direct buffers… but I can’t see how your toFloatBuffer() actually works. Maybe the code pasting got screwed up because I see the result allocated and never used.

When you set the data to the mesh how are you doing it?

The sad fact is that no matter what you do to avoid copying data, if you aren’t working in direct memory buffers then the data will end up being copied anyway. Everything that goes to the GPU has to be direct memory. It doesn’t matter whether we use mina buffers or whatever.

You mention ArrayList’s reallocation but I wonder if you know that it doubles in size (by default) each time it has to allocate more space? You could potentially do something similar. Allocate a direct buffer of a reasonable size and then expand it by some factor (reallocated, copy, delete the old one) when you run out of some space. Ultimately, I think the bit of extra memory it uses might be better than all of the wholesale copying you are doing now. I’m not deep enough in your code, though.

Heheh… actually, reading about mina, it seems like their self-expanding buffers do exactly what ArrayList does, I guess. And they plan to remove self-expansion in that way, too.

The underlying ByteBuffer is reallocated by IoBuffer behind the scene if the encoded data is larger than 8 bytes in the example above. Its capacity will double, and its limit will increase to the last position the string is written. This behavior is very similar to the way StringBuffer class works.

This mechanism is very likely to be removed from MINA 3.0, as it’s not really the best way to handle increased buffer size. It should be replaced by something like a InputStream hiding a list or an array of fixed sized ByteBuffers.

But regardless of how they end up, you’d have to copy it all to one big buffer to send to the GPU.

Alternately, you could split your mesh up into chunks.

@vinexgames: It hasn’t yet been released as a plugin. Still work in progress. But I think I’ll release the core stuff as soon as the dust of continuing changes settles (I’m still experimenting a lot). After that, I’ll start the GDE plugin.

@nehon: I either didn’t read or understand the warning box. :smiley: I’ll take a look Guava. But I think an universal utility class will always have more sanity checks than my little specialized class. The only way I could save the last copy to the VertexBuffer’s FloatBuffer would be if VertexBuffer used a different, efficiently growable (or customizable) buffer class. But I understand that won’t happen so my optimization attempts ends at this point. Maybe someone will find a bottleneck brainbug when I release the whole thing.

I can run my decals test for hours without getting out of mana. :smiley: Task manager memory usage stays stable. What is “the max”?

@pspeed: Ok, another attempt to post stuff via java bbcode. ArrayList doesn’t actually double the size on grow. It does “size + (size >> 1)” which is factor 1.5. A different tradeof between wasted space and logarithmic behavoir. I do it exactly the same way. But I don’t copy existing arrays but just add new ones with 1.5x size (like a pyramid). And I’ll try to reuse buffers whenever possible. This happens in applyOutout().

Edit: As you propose, I’m splitting the mesh up. But I don’t use fix sized chunks. I split it up into #“number of cpu threads” chunks to have as little thread calls and output buffers to join as possible.

Edit: Ok, editing posts makes java bbcode eat something. I have to delete the java block, submit, create a new one, submit. Sorry.

[java]
public static void applyOutput(final CachedDecalMeshGeneratorMT.Output output, final Mesh decalMesh)
{
applyOutput(output, decalMesh, false);
}

private static void applyDynamicFloatArray(final Mesh decalMesh, final VertexBuffer.Type type, final int components, final DynamicFloatArray src)
{
VertexBuffer destVB = decalMesh.getBuffer(type);
if (destVB != null)
{
FloatBuffer dest = (FloatBuffer)destVB.getData();
if (dest != null && dest.capacity() >= src.getLength())
{
dest.clear();
src.toFloatBuffer(dest);
dest.position(0).limit(src.getLength());
destVB.updateData(dest);
return;
}
}

decalMesh.setBuffer(type, components, src.toFloatBuffer());

}

public static void applyOutput(final CachedDecalMeshGeneratorMT.Output output, final Mesh decalMesh, boolean generateIndexData)
{
applyDynamicFloatArray(decalMesh, VertexBuffer.Type.Position, 3, output.Positions);
applyDynamicFloatArray(decalMesh, VertexBuffer.Type.Normal, 3, output.Normals);
applyDynamicFloatArray(decalMesh, VertexBuffer.Type.TexCoord, 2, output.TexCoords);

if (generateIndexData)
{
  int indexDataSize = output.Positions.getLength() / 3;
  IndexBuffer ib = decalMesh.getIndexBuffer();
  Buffer ibb;
  if (ib != null)
  {
    ibb = ib.getBuffer();
    if (ibb != null &amp;&amp; ibb.capacity() &gt;= indexDataSize)
    {
      ibb.limit(indexDataSize).rewind();
    }
    else
    {
      ib = IndexBuffer.createIndexBuffer(indexDataSize, indexDataSize);
      ibb = ib.getBuffer();
    }
  }
  else
  {
    ib = IndexBuffer.createIndexBuffer(indexDataSize, indexDataSize);
    ibb = ib.getBuffer();        
  }
  
  for (int i = 0; i &lt; indexDataSize; i++)
  {
    ib.put(i, i);
  }
  
  if (ibb instanceof IntBuffer)
  {
    decalMesh.setBuffer(VertexBuffer.Type.Index, 3, (IntBuffer)ibb);        
  }
  else
  {
    decalMesh.setBuffer(VertexBuffer.Type.Index, 3, (ShortBuffer)ibb);                
  }        
}
else
{
  decalMesh.clearBuffer(VertexBuffer.Type.Index);
}

if (output.Tangents != null)
{
  applyDynamicFloatArray(decalMesh, VertexBuffer.Type.Tangent, 4, output.Tangents);
  output.Tangents.reset();
}
else
{
  decalMesh.clearBuffer(VertexBuffer.Type.Tangent);
}

output.Positions.reset();
output.Normals.reset();
output.TexCoords.reset();

decalMesh.updateCounts();
decalMesh.updateBound();

}
[/java]

As always!!! YOU ROCK!!! it’s a really cool thing!!!