Error in Particle System with Particle Count > 2^14 ( test case included )

Hi Guys,



I have been trying to visualize data using a particle mesh. My data set is rather large and varies from 10.000 - 40.000. I wanted to use a triangle mesh, rather than a point mesh, since the user will be flying around the 3D data scape. A point mesh looks very bad up close, where points have a constant “pixel size”. So I used a slightly modified particle tri mesh. Now I have found an issue with the particle mesh. When the number of particles exceed 16384 ( = 2 ^ 14 ), the particle positions seem to loop. Meaning that only the first 16384 particles are vissible, the remaining particles will be placed on top of the first 16384, instead of using their own particle.position…



Here is an image that illustrates the issue:

  • the white is the particle mesh
  • colored is the point mesh visualizing the same data
  • As you can see the particle mesh is missing half the data ( of a total of 32768 it is missing 16384 )





    Here is a test case: ( please swap the two lines: this.setupParticleMesh() and this.setupPointMesh() - you will see that the point mesh renders the points just fine, while the particle mesh only renders the first 16384 points. If you set the point generation to 128 x 128 it just fits within the limits of the particle mesh )

    [java]



    import com.jme3.app.SimpleApplication;

    import com.jme3.effect.Particle;

    import com.jme3.effect.ParticleEmitter;

    import com.jme3.effect.ParticleMesh;

    import com.jme3.effect.influencers.ParticleInfluencer;

    import com.jme3.effect.shapes.EmitterShape;

    import com.jme3.export.JmeExporter;

    import com.jme3.export.JmeImporter;

    import com.jme3.light.DirectionalLight;

    import com.jme3.material.Material;

    import com.jme3.math.ColorRGBA;

    import com.jme3.math.Vector3f;

    import com.jme3.scene.Geometry;

    import com.jme3.scene.Mesh;

    import com.jme3.scene.VertexBuffer.Type;

    import com.jme3.scene.debug.Grid;

    import com.jme3.util.BufferUtils;

    import java.io.IOException;



    /**

    *
  • @author nhald

    */

    public class TestLargeParticleCount extends SimpleApplication {



    Geometry g;



    ParticleEmitter particleEmitter;



    @Override

    public void simpleInitApp() {



    Vector3f data[] = this.sampleData( 128, 256 );



    this.setupParticleMesh( data, 0.01f );

    // this.setupPointMesh( data );



    this.getFlyByCamera().setMoveSpeed( 10f );

    this.getCamera().setLocation( new Vector3f( 0, 30f, 0 ) );

    this.getCamera().lookAtDirection( new Vector3f( 0, -1, 0 ), Vector3f.UNIT_X );



    this.attachGrid( new Vector3f( 5, 0 ,5 ), 11, ColorRGBA.White );



    }



    private void attachGrid(Vector3f pos, int size, ColorRGBA color) {

    Geometry g = new Geometry( "wireframe grid", new Grid( size, size, 1f ) );

    Material mat = new Material(assetManager, "Common/MatDefs/Misc/Unshaded.j3md");

    mat.getAdditionalRenderState().setWireframe(true);

    mat.setColor("Color", color);

    g.setMaterial(mat);

    g.center().move(pos);

    rootNode.attachChild(g);

    }



    private void setupParticleMesh( final Vector3f data[], final float scale ){



    particleEmitter =

    new ParticleEmitter( "Emitter", ParticleMesh.Type.Triangle, data.length );

    Material mat_red = new Material(assetManager,

    "Common/MatDefs/Misc/Particle.j3md");

    mat_red.setTexture("Texture", assetManager.loadTexture(

    "Texture/testImage.png"));

    particleEmitter.setMaterial( mat_red );

    particleEmitter.setImagesX( 1 );

    particleEmitter.setImagesY( 1 ); // 2x2 texture animation

    particleEmitter.setEndColor( new ColorRGBA( 1f, 1f, 1f, 1f ) ); // red

    particleEmitter.setStartColor( new ColorRGBA( 1f, 1f, 1f, 1f ) ); // yellow

    particleEmitter.setGravity( 0, 0, 0 );



    particleEmitter.setShape( null );



    particleEmitter.setStartSize( scale );

    particleEmitter.setEndSize( scale );



    particleEmitter.setLowLife( 1000000 );

    particleEmitter.setHighLife( 1000000 );



    particleEmitter.setParticleInfluencer( new ParticleInfluencer(){



    int start = 0;



    public void influenceParticle( Particle particle, EmitterShape emitterShape ) {

    particle.position.set( data[ start++ ] );

    }



    public void setInitialVelocity( Vector3f initialVelocity ) { }



    public Vector3f getInitialVelocity() {

    return Vector3f.ZERO;

    }



    public void setVelocityVariation( float variation ) { }



    public float getVelocityVariation() {

    return 0;

    }



    public void write(JmeExporter ex) throws IOException {

    throw new UnsupportedOperationException("Not supported yet.");

    }



    public void read(JmeImporter im) throws IOException {

    throw new UnsupportedOperationException("Not supported yet.");

    }



    public ParticleInfluencer clone() { return null; }



    }



    );



    this.particleEmitter.emitAllParticles();



    rootNode.attachChild(particleEmitter);



    }



    private void setupPointMesh( Vector3f data[] ){

    Mesh mesh = new Mesh();



    Vector3f[] vertices = new Vector3f[ data.length ];

    float colors[] = new float[ 4 * vertices.length ];

    int colorIndex = 0;

    for( int i = 0; i < data.length; i++ ){



    vertices[ i ] = data[ i ];

    ColorRGBA color = new ColorRGBA( 1, 0, 0, 1 );

    color.interpolate( new ColorRGBA( 0, 1, 0, 1 ), i / ( float ) data.length );

    colors[ colorIndex++ ] = color.r;

    colors[ colorIndex++ ] = color.g;

    colors[ colorIndex++ ] = color.b;

    colors[ colorIndex++ ] = color.a;



    }



    mesh.setMode( Mesh.Mode.Points );

    mesh.setPointSize( 1f );

    mesh.setBuffer( Type.Color, 4, colors );

    mesh.setBuffer( Type.Position, 3, BufferUtils.createFloatBuffer( vertices ) );

    mesh.updateBound();



    Geometry g = new Geometry( "AsteroidMesh", mesh );

    Material mat = new Material(assetManager, "Common/MatDefs/Misc/Unshaded.j3md");

    mat.setColor("Color", ColorRGBA.White );

    mat.setBoolean( "VertexColor", true ); // Vertex colors

    g.setMaterial( mat );



    this.getRootNode().attachChild( g );



    }





    private Vector3f[] sampleData( int xSamples, int zSamples ){



    Vector3f p[] = new Vector3f[ xSamples * zSamples ];



    int i = 0;

    for( int x = 0; x < xSamples; x++ ){

    for( int z = 0; z < zSamples; z++ ){

    p[ i++ ] = new Vector3f( 10f * x / xSamples, 0, 10f * z / zSamples );

    }

    }



    return p;



    }



    public static void main( String args[] ){

    TestLargeParticleCount app = new TestLargeParticleCount();

    app.start();

    }



    }



    [/java]



    I have decided to break the data into smaller meshes, which should solve this issue, and allow culling of parts of the data. But since I spend a day trying to find the cause of this, I´d like someone else to look at it.



    I have tried this on 2 different PCs and both on Beta and RC2. The problem persists.

First, since your particles don’t move then there is really not a strong reason to use a particle emitter over a custom mesh.



And once you use a custom mesh, then you can use an int-based IndexBuffer instead of a short-based one.

2 Likes

…or just cluster your space…

1 Like

Thank you pspeed. It took you 6 minutes to solve and reply to an issue I have wasted a whole days productivity on… Damn.



I short based index buffer would be a reasonable explanation. I tracked the position data all the way to the writing the position float buffer and it was still fine. Makes sense that it a short index buffer would cause this.



I realize that using a particle emitter may seem a little over hand. All I wanted was the functionality for the points to always face the camera. I have altered the standard emitter and particle mesh, to have a flag that removes all interpolations and life counting, so effectively only the rotation stuff remains. But I might rewrite this to a custom mesh, once I have a proof of concept.



Cheers!

Thanks normen, I am planning to break the data into chunks, to be able to cull parts of it.



Would it be worth adding a thrown exception to the ParticleEmitter constructor for people trying to assign 2^14+ particles?



Thank you guys, this was driving me crazy, now I can rest again.

Also, you can use a point mesh and point sprites if you can deal with the limitations. (On some cards, a point sprite cannot get bigger than, say, 64 pixels on screen no matter how close you get.)



You get the added benefit of not having to resend the mesh data every frame just to orient it to camera. A custom shader could also take care of this for triangles. I’m not knowledgeable on how the particle stuff keeps triangles oriented… I didn’t even know that it did, actually.

2 Likes
@pspeed said:
I'm not knowledgeable on how the particle stuff keeps triangles oriented... I didn't even know that it did, actually.

This is why it would be interesting to look into how this could be properly extended as its something thats there already and could at the same time cover some common use cases like huge numbers of units on the screen etc :)

@pspeed Thanks for the Point Sprite hint! They are amazing, my cheap laptop is rendering 200.000 textured point sprites at 25 fps - makes good sense. It saves the recalculation of orientation every frame, and saves pushing the entire mesh onto the GPU every frame. I am very pleased! With point sprites I don’t need anything from the ParticleEmitter / ParticleMesh either. Great!