BufferUtils tracking reimplementation/fix

Well I rewrote the tracking part of the BufferUtils today, as the current one is severally flawed (as stated in its comments)



(file based on jme3 revision 9581) patch file not possible, sorry



Way it works:

On each buffer creation a helper object is created, that contains a phantomreference to the buffer,

the size

and the class of the buffer.

→ This ensures that no direct access to the buffer itself is needed anymore!

This is extremly important if the destroyBuffer method is used, as methods such as equal (prior used in the old trackinghashmap) creash the jvm with a seg fault.



All phantomreferences are added to a queue, where the gc will add them once their attached object (the buffer) is collected.

A helper thread waits for this and then removes the corresponding info object from the trackingarray, by comparing the memory address of the stored and the cleared reference to the phantomreference(sounds funny or? in other words just both pointer addresses are compared) (thats why a == is used instead of an equal)



Pro:

works as expected now, not dependent on buffer content as method prior was



Negative: slightly worse performance



I would be very happy to see this added to the core. If there are any questiosn feel free to ask.



[java]

/**

** Copyright © 2009-2012 jMonkeyEngine

  • All rights reserved.

    *
  • Redistribution and use in source and binary forms, with or without
  • modification, are permitted provided that the following conditions are
  • met:

    *
    • Redistributions of source code must retain the above copyright
  • notice, this list of conditions and the following disclaimer.

    *
    • Redistributions in binary form must reproduce the above copyright
  • notice, this list of conditions and the following disclaimer in the
  • documentation and/or other materials provided with the distribution.

    *
    • Neither the name of ‘jMonkeyEngine’ nor the names of its contributors
  • may be used to endorse or promote products derived from this software
  • without specific prior written permission.

    *
  • THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
  • "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
  • TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
  • PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
  • CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
  • EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
  • PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
  • PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
  • LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
  • NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
  • SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

    */

    package com.jme3.util;



    import java.lang.ref.PhantomReference;

    import java.lang.ref.Reference;

    import java.lang.ref.ReferenceQueue;

    import java.lang.reflect.InvocationTargetException;

    import java.lang.reflect.Method;

    import java.nio.Buffer;

    import java.nio.ByteBuffer;

    import java.nio.ByteOrder;

    import java.nio.DoubleBuffer;

    import java.nio.FloatBuffer;

    import java.nio.IntBuffer;

    import java.nio.ShortBuffer;

    import java.util.concurrent.ConcurrentHashMap;

    import java.util.concurrent.atomic.AtomicBoolean;

    import java.util.logging.Level;

    import java.util.logging.Logger;



    import com.jme3.math.ColorRGBA;

    import com.jme3.math.Quaternion;

    import com.jme3.math.Vector2f;

    import com.jme3.math.Vector3f;



    /**
  • <code>BufferUtils</code> is a helper class for generating nio buffers from
  • jME data classes such as Vectors and ColorRGBA.

    *
  • @author Joshua Slack
  • @version $Id: BufferUtils.java,v 1.16 2007/10/29 16:56:18 nca Exp $

    */

    public final class BufferUtils {

    private static final boolean trackDirectMemory = true;

    private static ReferenceQueue<Buffer> removeCollected = new ReferenceQueue<Buffer>();

    private static ConcurrentHashMap<BufferInfo, BufferInfo> trackedBuffers = new ConcurrentHashMap<BufferInfo, BufferInfo>();

    static ClearReferences cleanupthread;



    /**
  • Creates a clone of the given buffer. The clone’s capacity is equal to the
  • given buffer’s limit.

    *
  • @param buf
  •        The buffer to clone<br />
    
  • @return The cloned buffer

    */

    public static Buffer clone(final Buffer buf) {

    if (buf instanceof FloatBuffer) {

    return BufferUtils.clone((FloatBuffer) buf);

    } else if (buf instanceof ShortBuffer) {

    return BufferUtils.clone((ShortBuffer) buf);

    } else if (buf instanceof ByteBuffer) {

    return BufferUtils.clone((ByteBuffer) buf);

    } else if (buf instanceof IntBuffer) {

    return BufferUtils.clone((IntBuffer) buf);

    } else if (buf instanceof DoubleBuffer) {

    return BufferUtils.clone((DoubleBuffer) buf);

    } else {

    throw new UnsupportedOperationException();

    }

    }



    private static void onBufferAllocated(final Buffer buffer) {

    /**
  • StackTraceElement[] stackTrace = new Throwable().getStackTrace(); int
  • initialIndex = 0;

    *
  • for (int i = 0; i < stackTrace.length; i++){ if
  • (!stackTrace.getClassName().equals(BufferUtils.class.getName())){
  • initialIndex = i; break; } }

    *
  • int allocated = buffer.capacity(); int size = 0;

    *
  • if (buffer instanceof FloatBuffer){ size = 4; }else if (buffer
  • instanceof ShortBuffer){ size = 2; }else if (buffer instanceof
  • ByteBuffer){ size = 1; }else if (buffer instanceof IntBuffer){ size =
  • 4; }else if (buffer instanceof DoubleBuffer){ size = 8; }

    *
  • allocated *= size;

    *
  • for (int i = initialIndex; i < stackTrace.length; i++){
  • StackTraceElement element = stackTrace; if
  • (element.getClassName().startsWith("java")){ break; }

    *
  • try { Class clazz = Class.forName(element.getClassName()); if (i ==
  • initialIndex){
  • System.out.println(clazz.getSimpleName()+"."+element.getMethodName
  • ()+"():" + element.getLineNumber() + " allocated " + allocated);
  • }else{ System.out.println(" at " +
  • clazz.getSimpleName()+"."+element.getMethodName()+"()"); } } catch
  • (ClassNotFoundException ex) { } }

    */



    if (BufferUtils.trackDirectMemory) {



    if (BufferUtils.cleanupthread == null) {

    BufferUtils.cleanupthread = new ClearReferences();

    BufferUtils.cleanupthread.start();

    }

    if (buffer instanceof ByteBuffer) {

    final BufferInfo info = new BufferInfo(ByteBuffer.class, buffer.capacity(), buffer, BufferUtils.removeCollected);

    BufferUtils.trackedBuffers.put(info, info);

    } else if (buffer instanceof FloatBuffer) {

    final BufferInfo info = new BufferInfo(FloatBuffer.class, buffer.capacity() * 4, buffer, BufferUtils.removeCollected);

    BufferUtils.trackedBuffers.put(info, info);

    } else if (buffer instanceof IntBuffer) {

    final BufferInfo info = new BufferInfo(IntBuffer.class, buffer.capacity() * 4, buffer, BufferUtils.removeCollected);

    BufferUtils.trackedBuffers.put(info, info);

    } else if (buffer instanceof ShortBuffer) {

    final BufferInfo info = new BufferInfo(ShortBuffer.class, buffer.capacity() * 2, buffer, BufferUtils.removeCollected);

    BufferUtils.trackedBuffers.put(info, info);

    } else if (buffer instanceof DoubleBuffer) {

    final BufferInfo info = new BufferInfo(DoubleBuffer.class, buffer.capacity() * 8, buffer, BufferUtils.removeCollected);

    BufferUtils.trackedBuffers.put(info, info);

    }



    }

    }



    /**
  • Generate a new FloatBuffer using the given array of Vector3f objects. The
  • FloatBuffer will be 3 * data.length long and contain the vector data as
  • data[0].x, data[0].y, data[0].z, data[1].x… etc.

    *
  • @param data
  •        array of Vector3f objects to place into a new FloatBuffer<br />
    

*/

public static FloatBuffer createFloatBuffer(final Vector3f… data) {

if (data == null) {

return null;

}

final FloatBuffer buff = BufferUtils.createFloatBuffer(3 * data.length);

for (final Vector3f element : data) {

if (element != null) {

buff.put(element.x).put(element.y).put(element.z);

} else {

buff.put(0).put(0).put(0);

}

}

buff.flip();

return buff;

}



/**

  • Generate a new FloatBuffer using the given array of Quaternion objects.
  • The FloatBuffer will be 4 * data.length long and contain the vector data.

    *
  • @param data
  •        array of Quaternion objects to place into a new FloatBuffer<br />
    

*/

public static FloatBuffer createFloatBuffer(final Quaternion… data) {

if (data == null) {

return null;

}

final FloatBuffer buff = BufferUtils.createFloatBuffer(4 * data.length);

for (final Quaternion element : data) {

if (element != null) {

buff.put(element.getX()).put(element.getY()).put(element.getZ()).put(element.getW());

} else {

buff.put(0).put(0).put(0);

}

}

buff.flip();

return buff;

}



/**

  • Generate a new FloatBuffer using the given array of float primitives.

    *
  • @param data
  •        array of float primitives to place into a new FloatBuffer<br />
    

*/

public static FloatBuffer createFloatBuffer(final float… data) {

if (data == null) {

return null;

}

final FloatBuffer buff = BufferUtils.createFloatBuffer(data.length);

buff.clear();

buff.put(data);

buff.flip();

return buff;

}



/**

  • Create a new FloatBuffer of an appropriate size to hold the specified
  • number of Vector3f object data.

    *
  • @param vertices
  •        number of vertices that need to be held by the newly created<br />
    
  •        buffer<br />
    
  • @return the requested new FloatBuffer

    */

    public static FloatBuffer createVector3Buffer(final int vertices) {

    final FloatBuffer vBuff = BufferUtils.createFloatBuffer(3 * vertices);

    return vBuff;

    }



    /**
  • Create a new FloatBuffer of an appropriate size to hold the specified
  • number of Vector3f object data only if the given buffer if not already
  • the right size.

    *
  • @param buf
  •        the buffer to first check and rewind<br />
    
  • @param vertices
  •        number of vertices that need to be held by the newly created<br />
    
  •        buffer<br />
    
  • @return the requested new FloatBuffer

    */

    public static FloatBuffer createVector3Buffer(final FloatBuffer buf, final int vertices) {

    if (buf != null && buf.limit() == 3 * vertices) {

    buf.rewind();

    return buf;

    }



    return BufferUtils.createFloatBuffer(3 * vertices);

    }



    /**
  • Sets the data contained in the given color into the FloatBuffer at the
  • specified index.

    *
  • @param color
  •        the data to insert<br />
    
  • @param buf
  •        the buffer to insert into<br />
    
  • @param index
  •        the postion to place the data; in terms of colors not floats<br />
    

*/

public static void setInBuffer(final ColorRGBA color, final FloatBuffer buf, final int index) {

buf.position(index * 4);

buf.put(color.r);

buf.put(color.g);

buf.put(color.b);

buf.put(color.a);

}



/**

  • Sets the data contained in the given quaternion into the FloatBuffer at
  • the specified index.

    *
  • @param quat
  •        the {@link Quaternion} to insert<br />
    
  • @param buf
  •        the buffer to insert into<br />
    
  • @param index
  •        the postion to place the data; in terms of quaternions not<br />
    
  •        floats<br />
    

*/

public static void setInBuffer(final Quaternion quat, final FloatBuffer buf, final int index) {

buf.position(index * 4);

buf.put(quat.getX());

buf.put(quat.getY());

buf.put(quat.getZ());

buf.put(quat.getW());

}



/**

  • Sets the data contained in the given Vector3F into the FloatBuffer at the
  • specified index.

    *
  • @param vector
  •        the data to insert<br />
    
  • @param buf
  •        the buffer to insert into<br />
    
  • @param index
  •        the postion to place the data; in terms of vectors not floats<br />
    

*/

public static void setInBuffer(final Vector3f vector, final FloatBuffer buf, final int index) {

if (buf == null) {

return;

}

if (vector == null) {

buf.put(index * 3, 0);

buf.put(index * 3 + 1, 0);

buf.put(index * 3 + 2, 0);

} else {

buf.put(index * 3, vector.x);

buf.put(index * 3 + 1, vector.y);

buf.put(index * 3 + 2, vector.z);

}

}



/**

  • Updates the values of the given vector from the specified buffer at the
  • index provided.

    *
  • @param vector
  •        the vector to set data on<br />
    
  • @param buf
  •        the buffer to read from<br />
    
  • @param index
  •        the position (in terms of vectors, not floats) to read from<br />
    
  •        the buf<br />
    

*/

public static void populateFromBuffer(final Vector3f vector, final FloatBuffer buf, final int index) {

vector.x = buf.get(index * 3);

vector.y = buf.get(index * 3 + 1);

vector.z = buf.get(index * 3 + 2);

}



/**

  • Generates a Vector3f array from the given FloatBuffer.

    *
  • @param buff
  •        the FloatBuffer to read from<br />
    
  • @return a newly generated array of Vector3f objects

    */

    public static Vector3f[] getVector3Array(final FloatBuffer buff) {

    buff.clear();

    final Vector3f[] verts = new Vector3f[buff.limit() / 3];

    for (int x = 0; x < verts.length; x++) {

    final Vector3f v = new Vector3f(buff.get(), buff.get(), buff.get());

    verts[x] = v;

    }

    return verts;

    }



    /**
  • Copies a Vector3f from one position in the buffer to another. The index
  • values are in terms of vector number (eg, vector number 0 is postions 0-2
  • in the FloatBuffer.)

    *
  • @param buf
  •        the buffer to copy from/to<br />
    
  • @param fromPos
  •        the index of the vector to copy<br />
    
  • @param toPos
  •        the index to copy the vector to<br />
    

*/

public static void copyInternalVector3(final FloatBuffer buf, final int fromPos, final int toPos) {

BufferUtils.copyInternal(buf, fromPos * 3, toPos * 3, 3);

}



/**

  • Normalize a Vector3f in-buffer.

    *
  • @param buf
  •        the buffer to find the Vector3f within<br />
    
  • @param index
  •        the position (in terms of vectors, not floats) of the vector<br />
    
  •        to normalize<br />
    

*/

public static void normalizeVector3(final FloatBuffer buf, final int index) {

final TempVars vars = TempVars.get();

final Vector3f tempVec3 = vars.vect1;

BufferUtils.populateFromBuffer(tempVec3, buf, index);

tempVec3.normalizeLocal();

BufferUtils.setInBuffer(tempVec3, buf, index);

vars.release();

}



/**

  • Add to a Vector3f in-buffer.

    *
  • @param toAdd
  •        the vector to add from<br />
    
  • @param buf
  •        the buffer to find the Vector3f within<br />
    
  • @param index
  •        the position (in terms of vectors, not floats) of the vector<br />
    
  •        to add to<br />
    

*/

public static void addInBuffer(final Vector3f toAdd, final FloatBuffer buf, final int index) {

final TempVars vars = TempVars.get();

final Vector3f tempVec3 = vars.vect1;

BufferUtils.populateFromBuffer(tempVec3, buf, index);

tempVec3.addLocal(toAdd);

BufferUtils.setInBuffer(tempVec3, buf, index);

vars.release();

}



/**

  • Multiply and store a Vector3f in-buffer.

    *
  • @param toMult
  •        the vector to multiply against<br />
    
  • @param buf
  •        the buffer to find the Vector3f within<br />
    
  • @param index
  •        the position (in terms of vectors, not floats) of the vector<br />
    
  •        to multiply<br />
    

*/

public static void multInBuffer(final Vector3f toMult, final FloatBuffer buf, final int index) {

final TempVars vars = TempVars.get();

final Vector3f tempVec3 = vars.vect1;

BufferUtils.populateFromBuffer(tempVec3, buf, index);

tempVec3.multLocal(toMult);

BufferUtils.setInBuffer(tempVec3, buf, index);

vars.release();

}



/**

  • Checks to see if the given Vector3f is equals to the data stored in the
  • buffer at the given data index.

    *
  • @param check
  •        the vector to check against - null will return false.<br />
    
  • @param buf
  •        the buffer to compare data with<br />
    
  • @param index
  •        the position (in terms of vectors, not floats) of the vector<br />
    
  •        in the buffer to check against<br />
    
  • @return true if the data is equivalent, otherwise false.

    */

    public static boolean equals(final Vector3f check, final FloatBuffer buf, final int index) {

    final TempVars vars = TempVars.get();

    final Vector3f tempVec3 = vars.vect1;

    BufferUtils.populateFromBuffer(tempVec3, buf, index);

    final boolean eq = tempVec3.equals(check);

    vars.release();

    return eq;

    }



    // // – VECTOR2F METHODS – ////

    /**
  • Generate a new FloatBuffer using the given array of Vector2f objects. The
  • FloatBuffer will be 2 * data.length long and contain the vector data as
  • data[0].x, data[0].y, data[1].x… etc.

    *
  • @param data
  •        array of Vector2f objects to place into a new FloatBuffer<br />
    

*/

public static FloatBuffer createFloatBuffer(final Vector2f… data) {

if (data == null) {

return null;

}

final FloatBuffer buff = BufferUtils.createFloatBuffer(2 * data.length);

for (final Vector2f element : data) {

if (element != null) {

buff.put(element.x).put(element.y);

} else {

buff.put(0).put(0);

}

}

buff.flip();

return buff;

}



/**

  • Create a new FloatBuffer of an appropriate size to hold the specified
  • number of Vector2f object data.

    *
  • @param vertices
  •        number of vertices that need to be held by the newly created<br />
    
  •        buffer<br />
    
  • @return the requested new FloatBuffer

    */

    public static FloatBuffer createVector2Buffer(final int vertices) {

    final FloatBuffer vBuff = BufferUtils.createFloatBuffer(2 * vertices);

    return vBuff;

    }



    /**
  • Create a new FloatBuffer of an appropriate size to hold the specified
  • number of Vector2f object data only if the given buffer if not already
  • the right size.

    *
  • @param buf
  •        the buffer to first check and rewind<br />
    
  • @param vertices
  •        number of vertices that need to be held by the newly created<br />
    
  •        buffer<br />
    
  • @return the requested new FloatBuffer

    */

    public static FloatBuffer createVector2Buffer(final FloatBuffer buf, final int vertices) {

    if (buf != null && buf.limit() == 2 * vertices) {

    buf.rewind();

    return buf;

    }



    return BufferUtils.createFloatBuffer(2 * vertices);

    }



    /**
  • Sets the data contained in the given Vector2F into the FloatBuffer at the
  • specified index.

    *
  • @param vector
  •        the data to insert<br />
    
  • @param buf
  •        the buffer to insert into<br />
    
  • @param index
  •        the postion to place the data; in terms of vectors not floats<br />
    

*/

public static void setInBuffer(final Vector2f vector, final FloatBuffer buf, final int index) {

buf.put(index * 2, vector.x);

buf.put(index * 2 + 1, vector.y);

}



/**

  • Updates the values of the given vector from the specified buffer at the
  • index provided.

    *
  • @param vector
  •        the vector to set data on<br />
    
  • @param buf
  •        the buffer to read from<br />
    
  • @param index
  •        the position (in terms of vectors, not floats) to read from<br />
    
  •        the buf<br />
    

*/

public static void populateFromBuffer(final Vector2f vector, final FloatBuffer buf, final int index) {

vector.x = buf.get(index * 2);

vector.y = buf.get(index * 2 + 1);

}



/**

  • Generates a Vector2f array from the given FloatBuffer.

    *
  • @param buff
  •        the FloatBuffer to read from<br />
    
  • @return a newly generated array of Vector2f objects

    */

    public static Vector2f[] getVector2Array(final FloatBuffer buff) {

    buff.clear();

    final Vector2f[] verts = new Vector2f[buff.limit() / 2];

    for (int x = 0; x < verts.length; x++) {

    final Vector2f v = new Vector2f(buff.get(), buff.get());

    verts[x] = v;

    }

    return verts;

    }



    /**
  • Copies a Vector2f from one position in the buffer to another. The index
  • values are in terms of vector number (eg, vector number 0 is postions 0-1
  • in the FloatBuffer.)

    *
  • @param buf
  •        the buffer to copy from/to<br />
    
  • @param fromPos
  •        the index of the vector to copy<br />
    
  • @param toPos
  •        the index to copy the vector to<br />
    

*/

public static void copyInternalVector2(final FloatBuffer buf, final int fromPos, final int toPos) {

BufferUtils.copyInternal(buf, fromPos * 2, toPos * 2, 2);

}



/**

  • Normalize a Vector2f in-buffer.

    *
  • @param buf
  •        the buffer to find the Vector2f within<br />
    
  • @param index
  •        the position (in terms of vectors, not floats) of the vector<br />
    
  •        to normalize<br />
    

*/

public static void normalizeVector2(final FloatBuffer buf, final int index) {

final TempVars vars = TempVars.get();

final Vector2f tempVec2 = vars.vect2d;

BufferUtils.populateFromBuffer(tempVec2, buf, index);

tempVec2.normalizeLocal();

BufferUtils.setInBuffer(tempVec2, buf, index);

vars.release();

}



/**

  • Add to a Vector2f in-buffer.

    *
  • @param toAdd
  •        the vector to add from<br />
    
  • @param buf
  •        the buffer to find the Vector2f within<br />
    
  • @param index
  •        the position (in terms of vectors, not floats) of the vector<br />
    
  •        to add to<br />
    

*/

public static void addInBuffer(final Vector2f toAdd, final FloatBuffer buf, final int index) {

final TempVars vars = TempVars.get();

final Vector2f tempVec2 = vars.vect2d;

BufferUtils.populateFromBuffer(tempVec2, buf, index);

tempVec2.addLocal(toAdd);

BufferUtils.setInBuffer(tempVec2, buf, index);

vars.release();

}



/**

  • Multiply and store a Vector2f in-buffer.

    *
  • @param toMult
  •        the vector to multiply against<br />
    
  • @param buf
  •        the buffer to find the Vector2f within<br />
    
  • @param index
  •        the position (in terms of vectors, not floats) of the vector<br />
    
  •        to multiply<br />
    

*/

public static void multInBuffer(final Vector2f toMult, final FloatBuffer buf, final int index) {

final TempVars vars = TempVars.get();

final Vector2f tempVec2 = vars.vect2d;

BufferUtils.populateFromBuffer(tempVec2, buf, index);

tempVec2.multLocal(toMult);

BufferUtils.setInBuffer(tempVec2, buf, index);

vars.release();

}



/**

  • Checks to see if the given Vector2f is equals to the data stored in the
  • buffer at the given data index.

    *
  • @param check
  •        the vector to check against - null will return false.<br />
    
  • @param buf
  •        the buffer to compare data with<br />
    
  • @param index
  •        the position (in terms of vectors, not floats) of the vector<br />
    
  •        in the buffer to check against<br />
    
  • @return true if the data is equivalent, otherwise false.

    */

    public static boolean equals(final Vector2f check, final FloatBuffer buf, final int index) {

    final TempVars vars = TempVars.get();

    final Vector2f tempVec2 = vars.vect2d;

    BufferUtils.populateFromBuffer(tempVec2, buf, index);

    final boolean eq = tempVec2.equals(check);

    vars.release();

    return eq;

    }



    // // – INT METHODS – ////

    /**
  • Generate a new IntBuffer using the given array of ints. The IntBuffer
  • will be data.length long and contain the int data as data[0], data[1]…
  • etc.

    *
  • @param data
  •        array of ints to place into a new IntBuffer<br />
    

*/

public static IntBuffer createIntBuffer(final int… data) {

if (data == null) {

return null;

}

final IntBuffer buff = BufferUtils.createIntBuffer(data.length);

buff.clear();

buff.put(data);

buff.flip();

return buff;

}



/**

  • Create a new int[] array and populate it with the given IntBuffer’s
  • contents.

    *
  • @param buff
  •        the IntBuffer to read from<br />
    
  • @return a new int array populated from the IntBuffer

    */

    public static int[] getIntArray(final IntBuffer buff) {

    if (buff == null) {

    return null;

    }

    buff.clear();

    final int[] inds = new int[buff.limit()];

    for (int x = 0; x < inds.length; x++) {

    inds[x] = buff.get();

    }

    return inds;

    }



    /**
  • Create a new float[] array and populate it with the given FloatBuffer’s
  • contents.

    *
  • @param buff
  •        the FloatBuffer to read from<br />
    
  • @return a new float array populated from the FloatBuffer

    */

    public static float[] getFloatArray(final FloatBuffer buff) {

    if (buff == null) {

    return null;

    }

    buff.clear();

    final float[] inds = new float[buff.limit()];

    for (int x = 0; x < inds.length; x++) {

    inds[x] = buff.get();

    }

    return inds;

    }



    // // – GENERAL DOUBLE ROUTINES – ////

    /**
  • Create a new DoubleBuffer of the specified size.

    *
  • @param size
  •        required number of double to store.<br />
    
  • @return the new DoubleBuffer

    */

    public static DoubleBuffer createDoubleBuffer(final int size) {

    final DoubleBuffer buf = ByteBuffer.allocateDirect(8 * size).order(ByteOrder.nativeOrder()).asDoubleBuffer();

    buf.clear();

    BufferUtils.onBufferAllocated(buf);

    return buf;

    }



    /**
  • Create a new DoubleBuffer of an appropriate size to hold the specified
  • number of doubles only if the given buffer if not already the right size.

    *
  • @param buf
  •        the buffer to first check and rewind<br />
    
  • @param size
  •        number of doubles that need to be held by the newly created<br />
    
  •        buffer<br />
    
  • @return the requested new DoubleBuffer

    */

    public static DoubleBuffer createDoubleBuffer(DoubleBuffer buf, final int size) {

    if (buf != null && buf.limit() == size) {

    buf.rewind();

    return buf;

    }



    buf = BufferUtils.createDoubleBuffer(size);

    return buf;

    }



    /**
  • Creates a new DoubleBuffer with the same contents as the given
  • DoubleBuffer. The new DoubleBuffer is seperate from the old one and
  • changes are not reflected across. If you want to reflect changes,
  • consider using Buffer.duplicate().

    *
  • @param buf
  •        the DoubleBuffer to copy<br />
    
  • @return the copy

    */

    public static DoubleBuffer clone(final DoubleBuffer buf) {

    if (buf == null) {

    return null;

    }

    buf.rewind();



    DoubleBuffer copy;

    if (buf.isDirect()) {

    copy = BufferUtils.createDoubleBuffer(buf.limit());

    } else {

    copy = DoubleBuffer.allocate(buf.limit());

    }

    copy.put(buf);



    return copy;

    }



    // // – GENERAL FLOAT ROUTINES – ////

    /**
  • Create a new FloatBuffer of the specified size.

    *
  • @param size
  •        required number of floats to store.<br />
    
  • @return the new FloatBuffer

    */

    public static FloatBuffer createFloatBuffer(final int size) {

    final FloatBuffer buf = ByteBuffer.allocateDirect(4 * size).order(ByteOrder.nativeOrder()).asFloatBuffer();

    buf.clear();

    BufferUtils.onBufferAllocated(buf);

    return buf;

    }



    /**
  • Copies floats from one position in the buffer to another.

    *
  • @param buf
  •        the buffer to copy from/to<br />
    
  • @param fromPos
  •        the starting point to copy from<br />
    
  • @param toPos
  •        the starting point to copy to<br />
    
  • @param length
  •        the number of floats to copy<br />
    

*/

public static void copyInternal(final FloatBuffer buf, final int fromPos, final int toPos, final int length) {

final float[] data = new float[length];

buf.position(fromPos);

buf.get(data);

buf.position(toPos);

buf.put(data);

}



/**

  • Creates a new FloatBuffer with the same contents as the given
  • FloatBuffer. The new FloatBuffer is seperate from the old one and changes
  • are not reflected across. If you want to reflect changes, consider using
  • Buffer.duplicate().

    *
  • @param buf
  •        the FloatBuffer to copy<br />
    
  • @return the copy

    */

    public static FloatBuffer clone(final FloatBuffer buf) {

    if (buf == null) {

    return null;

    }

    buf.rewind();



    FloatBuffer copy;

    if (buf.isDirect()) {

    copy = BufferUtils.createFloatBuffer(buf.limit());

    } else {

    copy = FloatBuffer.allocate(buf.limit());

    }

    copy.put(buf);



    return copy;

    }



    // // – GENERAL INT ROUTINES – ////

    /**
  • Create a new IntBuffer of the specified size.

    *
  • @param size
  •        required number of ints to store.<br />
    
  • @return the new IntBuffer

    */

    public static IntBuffer createIntBuffer(final int size) {

    final IntBuffer buf = ByteBuffer.allocateDirect(4 * size).order(ByteOrder.nativeOrder()).asIntBuffer();

    buf.clear();

    BufferUtils.onBufferAllocated(buf);

    return buf;

    }



    /**
  • Create a new IntBuffer of an appropriate size to hold the specified
  • number of ints only if the given buffer if not already the right size.

    *
  • @param buf
  •        the buffer to first check and rewind<br />
    
  • @param size
  •        number of ints that need to be held by the newly created<br />
    
  •        buffer<br />
    
  • @return the requested new IntBuffer

    */

    public static IntBuffer createIntBuffer(IntBuffer buf, final int size) {

    if (buf != null && buf.limit() == size) {

    buf.rewind();

    return buf;

    }



    buf = BufferUtils.createIntBuffer(size);

    return buf;

    }



    /**
  • Creates a new IntBuffer with the same contents as the given IntBuffer.
  • The new IntBuffer is seperate from the old one and changes are not
  • reflected across. If you want to reflect changes, consider using
  • Buffer.duplicate().

    *
  • @param buf
  •        the IntBuffer to copy<br />
    
  • @return the copy

    */

    public static IntBuffer clone(final IntBuffer buf) {

    if (buf == null) {

    return null;

    }

    buf.rewind();



    IntBuffer copy;

    if (buf.isDirect()) {

    copy = BufferUtils.createIntBuffer(buf.limit());

    } else {

    copy = IntBuffer.allocate(buf.limit());

    }

    copy.put(buf);



    return copy;

    }



    // // – GENERAL BYTE ROUTINES – ////

    /**
  • Create a new ByteBuffer of the specified size.

    *
  • @param size
  •        required number of ints to store.<br />
    
  • @return the new IntBuffer

    */

    public static ByteBuffer createByteBuffer(final int size) {

    final ByteBuffer buf = ByteBuffer.allocateDirect(size).order(ByteOrder.nativeOrder());

    buf.clear();

    BufferUtils.onBufferAllocated(buf);

    return buf;

    }



    /**
  • Create a new ByteBuffer of an appropriate size to hold the specified
  • number of ints only if the given buffer if not already the right size.

    *
  • @param buf
  •        the buffer to first check and rewind<br />
    
  • @param size
  •        number of bytes that need to be held by the newly created<br />
    
  •        buffer<br />
    
  • @return the requested new IntBuffer

    */

    public static ByteBuffer createByteBuffer(ByteBuffer buf, final int size) {

    if (buf != null && buf.limit() == size) {

    buf.rewind();

    return buf;

    }



    buf = BufferUtils.createByteBuffer(size);

    return buf;

    }



    public static ByteBuffer createByteBuffer(final byte… data) {

    final ByteBuffer bb = BufferUtils.createByteBuffer(data.length);

    bb.put(data);

    bb.flip();

    return bb;

    }



    public static ByteBuffer createByteBuffer(final String data) {

    final byte[] bytes = data.getBytes();

    final ByteBuffer bb = BufferUtils.createByteBuffer(bytes.length);

    bb.put(bytes);

    bb.flip();

    return bb;

    }



    /**
  • Creates a new ByteBuffer with the same contents as the given ByteBuffer.
  • The new ByteBuffer is seperate from the old one and changes are not
  • reflected across. If you want to reflect changes, consider using
  • Buffer.duplicate().

    *
  • @param buf
  •        the ByteBuffer to copy<br />
    
  • @return the copy

    */

    public static ByteBuffer clone(final ByteBuffer buf) {

    if (buf == null) {

    return null;

    }

    buf.rewind();



    ByteBuffer copy;

    if (buf.isDirect()) {

    copy = BufferUtils.createByteBuffer(buf.limit());

    } else {

    copy = ByteBuffer.allocate(buf.limit());

    }

    copy.put(buf);



    return copy;

    }



    // // – GENERAL SHORT ROUTINES – ////

    /**
  • Create a new ShortBuffer of the specified size.

    *
  • @param size
  •        required number of shorts to store.<br />
    
  • @return the new ShortBuffer

    */

    public static ShortBuffer createShortBuffer(final int size) {

    final ShortBuffer buf = ByteBuffer.allocateDirect(2 * size).order(ByteOrder.nativeOrder()).asShortBuffer();

    buf.clear();

    BufferUtils.onBufferAllocated(buf);

    return buf;

    }



    /**
  • Create a new ShortBuffer of an appropriate size to hold the specified
  • number of shorts only if the given buffer if not already the right size.

    *
  • @param buf
  •        the buffer to first check and rewind<br />
    
  • @param size
  •        number of shorts that need to be held by the newly created<br />
    
  •        buffer<br />
    
  • @return the requested new ShortBuffer

    */

    public static ShortBuffer createShortBuffer(ShortBuffer buf, final int size) {

    if (buf != null && buf.limit() == size) {

    buf.rewind();

    return buf;

    }



    buf = BufferUtils.createShortBuffer(size);

    return buf;

    }



    public static ShortBuffer createShortBuffer(final short… data) {

    if (data == null) {

    return null;

    }

    final ShortBuffer buff = BufferUtils.createShortBuffer(data.length);

    buff.clear();

    buff.put(data);

    buff.flip();

    return buff;

    }



    /**
  • Creates a new ShortBuffer with the same contents as the given
  • ShortBuffer. The new ShortBuffer is seperate from the old one and changes
  • are not reflected across. If you want to reflect changes, consider using
  • Buffer.duplicate().

    *
  • @param buf
  •        the ShortBuffer to copy<br />
    
  • @return the copy

    */

    public static ShortBuffer clone(final ShortBuffer buf) {

    if (buf == null) {

    return null;

    }

    buf.rewind();



    ShortBuffer copy;

    if (buf.isDirect()) {

    copy = BufferUtils.createShortBuffer(buf.limit());

    } else {

    copy = ShortBuffer.allocate(buf.limit());

    }

    copy.put(buf);



    return copy;

    }



    /**
  • Ensures there is at least the <code>required</code> number of entries
  • left after the current position of the buffer. If the buffer is too small
  • a larger one is created and the old one copied to the new buffer.

    *
  • @param buffer
  •        buffer that should be checked/copied (may be null)<br />
    
  • @param required
  •        minimum number of elements that should be remaining in the<br />
    
  •        returned buffer<br />
    
  • @return a buffer large enough to receive at least the
  •     &lt;code&gt;required&lt;/code&gt; number of entries, same position as the<br />
    
  •     input buffer, not null<br />
    

*/

public static FloatBuffer ensureLargeEnough(FloatBuffer buffer, final int required) {

if (buffer == null || buffer.remaining() < required) {

final int position = buffer != null ? buffer.position() : 0;

final FloatBuffer newVerts = BufferUtils.createFloatBuffer(position + required);

if (buffer != null) {

buffer.flip();

newVerts.put(buffer);

newVerts.position(position);

}

buffer = newVerts;

}

return buffer;

}



public static ShortBuffer ensureLargeEnough(ShortBuffer buffer, final int required) {

if (buffer == null || buffer.remaining() < required) {

final int position = buffer != null ? buffer.position() : 0;

final ShortBuffer newVerts = BufferUtils.createShortBuffer(position + required);

if (buffer != null) {

buffer.flip();

newVerts.put(buffer);

newVerts.position(position);

}

buffer = newVerts;

}

return buffer;

}



public static ByteBuffer ensureLargeEnough(ByteBuffer buffer, final int required) {

if (buffer == null || buffer.remaining() < required) {

final int position = buffer != null ? buffer.position() : 0;

final ByteBuffer newVerts = BufferUtils.createByteBuffer(position + required);

if (buffer != null) {

buffer.flip();

newVerts.put(buffer);

newVerts.position(position);

}

buffer = newVerts;

}

return buffer;

}



public static void printCurrentDirectMemory(StringBuilder store) {

long totalHeld = 0;

// make a new set to hold the keys to prevent concurrency issues.

int fBufs = 0, bBufs = 0, iBufs = 0, sBufs = 0, dBufs = 0;

int fBufsM = 0, bBufsM = 0, iBufsM = 0, sBufsM = 0, dBufsM = 0;

for (final BufferInfo b : BufferUtils.trackedBuffers.values()) {

if (b.type == ByteBuffer.class) {

totalHeld += b.size;

bBufsM += b.size;

bBufs++;

} else if (b.type == FloatBuffer.class) {

totalHeld += b.size;

fBufsM += b.size;

fBufs++;

} else if (b.type == IntBuffer.class) {

totalHeld += b.size;

iBufsM += b.size;

iBufs++;

} else if (b.type == ShortBuffer.class) {

totalHeld += b.size;

sBufsM += b.size;

sBufs++;

} else if (b.type == DoubleBuffer.class) {

totalHeld += b.size;

dBufsM += b.size;

dBufs++;

}

}

final long heapMem = Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory();



final boolean printStout = store == null;

if (store == null) {

store = new StringBuilder();

}

store.append("Existing buffers: ").append(BufferUtils.trackedBuffers.size()).append("n");

store.append("(b: ").append(bBufs).append(" f: ").append(fBufs).append(" i: ").append(iBufs).append(" s: ").append(sBufs).append(" d: ").append(dBufs).append(")").append("n");

store.append("Total heap memory held: ").append(heapMem / 1024).append("kbn");

store.append("Total direct memory held: ").append(totalHeld / 1024).append("kbn");

store.append("(b: ").append(bBufsM / 1024).append("kb f: ").append(fBufsM / 1024).append("kb i: ").append(iBufsM / 1024).append("kb s: ").append(sBufsM / 1024).append("kb d: ").append(dBufsM / 1024).append("kb)").append("n");

if (printStout) {

System.out.println(store.toString());

}

}



private static final AtomicBoolean loadedMethods = new AtomicBoolean(false);

private static Method cleanerMethod = null;

private static Method cleanMethod = null;

private static Method viewedBufferMethod = null;

private static Method freeMethod = null;



private static Method loadMethod(final String className, final String methodName) {

try {

final Method method = Class.forName(className).getMethod(methodName);

method.setAccessible(true);

return method;

} catch (final NoSuchMethodException ex) {

return null; // the method was not found

} catch (final SecurityException ex) {

return null; // setAccessible not allowed by security policy

} catch (final ClassNotFoundException ex) {

return null; // the direct buffer implementation was not found

}

}



private static void loadCleanerMethods() {

// If its already true, exit, if not, set it to true.

if (BufferUtils.loadedMethods.getAndSet(true)) {

return;

}

// This could potentially be called many times if used from multiple

// threads

synchronized (BufferUtils.loadedMethods) {

// Oracle JRE / OpenJDK

BufferUtils.cleanerMethod = BufferUtils.loadMethod("sun.nio.ch.DirectBuffer", "cleaner");

BufferUtils.cleanMethod = BufferUtils.loadMethod("sun.misc.Cleaner", "clean");

BufferUtils.viewedBufferMethod = BufferUtils.loadMethod("sun.nio.ch.DirectBuffer", "viewedBuffer");

if (BufferUtils.viewedBufferMethod == null) {

// They changed the name in Java 7 (???)

BufferUtils.viewedBufferMethod = BufferUtils.loadMethod("sun.nio.ch.DirectBuffer", "attachment");

}



// Apache Harmony

final ByteBuffer bb = BufferUtils.createByteBuffer(1);

final Class<?> clazz = bb.getClass();

try {

BufferUtils.freeMethod = clazz.getMethod("free");

} catch (final NoSuchMethodException ex) {

} catch (final SecurityException ex) {

}

}

}



/**

  • Direct buffers are garbage collected by using a phantom reference and a
  • reference queue. Every once a while, the JVM checks the reference queue
  • and cleans the direct buffers. However, as this doesn’t happen
  • immediately after discarding all references to a direct buffer, it’s easy
  • to OutOfMemoryError yourself using direct buffers. This function
  • explicitly calls the Cleaner method of a direct buffer.

    *
  • @param toBeDestroyed
  •        The direct buffer that will be &quot;cleaned&quot;. Utilizes reflection.<br />
    

*

*/

public static void destroyDirectBuffer(final Buffer toBeDestroyed) {

if (!toBeDestroyed.isDirect()) {

return;

}



BufferUtils.loadCleanerMethods();



try {

if (BufferUtils.freeMethod != null) {

BufferUtils.freeMethod.invoke(toBeDestroyed);

} else {

final Object cleaner = BufferUtils.cleanerMethod.invoke(toBeDestroyed);

if (cleaner != null) {

BufferUtils.cleanMethod.invoke(cleaner);

} else {

// Try the alternate approach of getting the viewed buffer

// first

final Object viewedBuffer = BufferUtils.viewedBufferMethod.invoke(toBeDestroyed);

if (viewedBuffer != null) {

BufferUtils.destroyDirectBuffer((Buffer) viewedBuffer);

} else {

Logger.getLogger(BufferUtils.class.getName()).log(Level.SEVERE, "Buffer cannot be destroyed: {0}", toBeDestroyed);

}

}

}

} catch (final IllegalAccessException ex) {

Logger.getLogger(BufferUtils.class.getName()).log(Level.SEVERE, "{0}", ex);

} catch (final IllegalArgumentException ex) {

Logger.getLogger(BufferUtils.class.getName()).log(Level.SEVERE, "{0}", ex);

} catch (final InvocationTargetException ex) {

Logger.getLogger(BufferUtils.class.getName()).log(Level.SEVERE, "{0}", ex);

} catch (final SecurityException ex) {

Logger.getLogger(BufferUtils.class.getName()).log(Level.SEVERE, "{0}", ex);

}

}



private static class BufferInfo extends PhantomReference<Buffer> {

private final Class type;

private final int size;



public BufferInfo(final Class type, final int size, final Buffer referent, final ReferenceQueue<? super Buffer> q) {

super(referent, q);

this.type = type;

this.size = size;

}

}



private static class ClearReferences extends Thread {

ClearReferences() {

this.setDaemon(true);

}



@Override

public void run() {

try {

while (true) {

final Reference<? extends Buffer> toclean = BufferUtils.removeCollected.remove();

BufferUtils.trackedBuffers.remove(toclean);

}



} catch (final InterruptedException e) {

e.printStackTrace();

}

}

}

}

[/java]

ok, done, so anymore changes or is it fine so far?

Unless I missed it, one problem I see is that if user code manually destroys a buffer but still holds a reference to it then it will still count in the memory stats. That may fall into the realm of “just don’t do that”, though.



The error messages should also be changed to logs at the very least.



As long as this defaults to “off” then I don’t have a real problem with it, though I don’t have time to integrate it myself right now. Some better multi-threading might be nice since this effectively serializes all access to buffers.



For example, it should be possible to subclass PhantomReference to hold all of the information you are keeping in BufferInfo and then just keep the total stats all the time as things are added and removed… in something like AtomicLong or AtomicInteger. You wouldn’t need the ArrayList of buffer infos anymore and could totally remove all of the synchronized blocks. I’m not sure it works out to be more efficient but should be better for accidentally creating thread stepping just by turning on memory debugging.



Alternately, if you still want to keep all of the buffers around then either do the phantom reference subclassing and put them in a concurrent hash map<ref, ref> or leave the classes the way you have them and put them in a concurrent hash map <ref, buffer info>… then the memory dump can just iterate over values().

You are right, combined those two classes, to remove the for loop.



However the synchronizing must happen, since I must ensure that while the print method runs no modifications can be made (concurrent modification).



When deactivated there is no overhead at all anyway, since the false is a constant and the stuff will probably never make it into the byte code then.



For the destroyed buffers I must see what I will do, not sure yet how to track those correctly without introducing a for slope again. (also they are not that common usecase)

@EmpirePhoenix said:
However the synchronizing must happen, since I must ensure that while the print method runs no modifications can be made (concurrent modification).


The ConcurrentHashMap will take care of this for you. At worst you add a buffer into the totals right before it is removed but that's still accurate as of the time the method was called.

@EmpirePhoenix said:
When deactivated there is no overhead at all anyway, since the false is a constant and the stuff will probably never make it into the byte code then.


Yeah, if activation is off then I don't care really... but I was trying to avoid the case where a bug appears only when memory tracking is off and then suddenly disappears when it is turned on because now all threads run in lock step instead of concurrently. It's not just about the performance (which is pretty bad for synchronized) but about completely changing the running characteristics of the app by turning on a debug feature... especially when it can be easily minimized.

@EmpirePhoenix said:
For the destroyed buffers I must see what I will do, not sure yet how to track those correctly without introducing a for slope again. (also they are not that common usecase)


I wouldn't worry too much about it. If a caller holds a reference to a buffer they've already destroyed then maybe they just deserve what they get. :)

I guress i can’t follow why hasmap helps here? Could you give a example of the useage? And exxplain why it does not share the prbolem. (cause internally the hasmap must synchronize somewhere as well or?)

@EmpirePhoenix said:
I guress i can't follow why hasmap helps here? Could you give a example of the useage? And exxplain why it does not share the prbolem. (cause internally the hasmap must synchronize somewhere as well or?)


No. The java.util.concurrent classes avoid synchronized at all costs and use more clever locking mechanisms... that's the entire point of the package, actually. ConcurrentHashMap can support many threads without ever using synchronized... in many cases without ever locking at all. Lots of clever code in java.util.concurrent.

When you synchronize where you do, you force all buffer creators to work in lock-step with each other, with the thread showing the stats, and with the thread cleaning up the buffers. This could cause subtle timing problems to disappear when using the memory stats tracking... or worse yet those subtle bugs will show up in released code now that the "in my development environment" stats tracking is turned off for release.

The added performance benefits of a hashmap over a list is just a bonus in this case.
2 Likes

ok i understand, I have to use a custom hashCode tho? Since the default would use the buffers in it again, and rendering making all buffers with identical size with identical hashcode or? (since initial content is all zeros)

References don’t implement equals and hashCode so they pick up Object’s hashcode. Since you are using the references as keys then it will be fine. The problem before was that the buffers themselves were being used.

ok, so the Bufferinfo is my key, what is my value then?

@EmpirePhoenix said:
ok, so the Bufferinfo is my key, what is my value then?


BufferInfo... it doesn't really matter. You are kind of using the map like a set in this case.

Ok, i was wondering about that part ^^

@EmpirePhoenix said:
ok, done, so anymore changes or is it fine so far?


Heheh.... looking good. I think if I had any more ideas on changes that you'd start quoting me contracting rates, though. :)

I may look at integrating this some time if someone doesn't beat me to it.

Aww common won’t need so long to commit it :slight_smile:

@EmpirePhoenix said:
Aww common won't need so long to commit it :)


I don't just commit stuff. I integrate the changes and test it. So it takes me a little longer.

Just bumping this , so it wont be forgotten.

I don’t really know how useful this is … I mean you already have the render stats that show you which data is in GL memory, so if you’re leaking something you should see it there.

I dont understand, where can i see how much mb direct memory I use (without this)?



This is important to optimizse levels and such to not overflow the targeted hardware’s vram since else you get a serious perofrmance drop.

@EmpirePhoenix said:
I dont understand, where can i see how much mb direct memory I use (without this)?

This is important to optimizse levels and such to not overflow the targeted hardware's vram since else you get a serious perofrmance drop.

The level won't reside entirely in VRAM. Textures and meshes are swapped dynamically in and out as you move around the level, thus the buffer stats you get from this method won't help you optimize it.

Well true, but I know if they are far to large for the system to handle. (like more than 3 gb on 32 bit)



Anyway, either integrate this fix, or throw the old code out. Just don’t keep the old code and do nothing.

2 Likes