I am trying to animate a trace file and then animate it using Cinematics. The issue is when I load the trace file (very large one) that has all the motion events needed, it takes literally 5 minutes to load. The loading is done by parsing the file line by line and executing at each line the event on the fly. However, this approach is actually done by reading the whole file and putting it in some buffer that cinematics stores then runs (I assume that’s how it works - not sure). This runs fine however I need to reduce that loading time.
Is reading the file and storing it in an array list will speed up the loading time? Cinematics will then read the events from the array list.
thanks in advance
How big is the file? I rather think its how you process the data which takes long.
The size isn’t actually that big (200KB) it’s the number of lines and events in it (around 100,000 lines).
Yeah, so if you cannot make your processing more efficient then outsource it to another thread:
https://wiki.jmonkeyengine.org/legacy/doku.php/jme3:advanced:multithreading
Can I put it on another thread? I need the lines being read to initiate the animation from the beginning.
Also is putting the events in memory (in an arraylist) any better than creating them on the fly?
It depends.
Normen I know @nehon is the best to answer this but though you might know (since he’s on vacation).
Does Cinemtic works like that?
Code taken from testCinematic.java
[java] cinematic.addCinematicEvent(0, new PositionTrack(teapot, new Vector3f(10, 0, 10), 0));
cinematic.addCinematicEvent(0, new ScaleTrack(teapot, new Vector3f(1, 1, 1), 0));
float[] rotation = {0, 0, 0};
cinematic.addCinematicEvent(0, new RotationTrack(teapot, rotation, 0));
cinematic.addCinematicEvent(0, new PositionTrack(teapot, new Vector3f(10, 0, -10), 20));
cinematic.addCinematicEvent(0, new ScaleTrack(teapot, new Vector3f(4, 4, 4), 10));
cinematic.addCinematicEvent(10, new ScaleTrack(teapot, new Vector3f(1, 1, 1), 10));
float[] rotation2 = {0, FastMath.TWO_PI, 0};
cinematic.addCinematicEvent(0, new RotationTrack(teapot, rotation2, 20));
cinematic.activateCamera(0, “aroundCam”);
cinematic.addCinematicEvent(0, cameraMotionTrack);
cinematic.addCinematicEvent(0, new SoundTrack(“Sound/Environment/Nature.ogg”, LoopMode.Loop));
cinematic.addCinematicEvent(3, new SoundTrack(“Sound/Effects/kick.wav”));
cinematic.addCinematicEvent(3, new SubtitleTrack(“start”, 3, “jMonkey engine really kicks A…”));
cinematic.addCinematicEvent(5.0f, new SoundTrack(“Sound/Effects/Beep.ogg”, 1));
cinematic.addCinematicEvent(6, new AnimationTrack(model, “Walk”, LoopMode.Loop));
cinematic.activateCamera(6, “topView”);
cinematic.activateCamera(10, “aroundCam”);
[/java]
Does it read all those events and stores them in a buffer (that I guess we can’t see) THEN it runs them according to their trigger time??
Yeah, basically. Tho its not much data really.
ok so good so my question is if I want to read that data from a file, should I read it on the fly or using arraylist, or maybe something else?
it’s taking me 5 minutes to load the animation.
PS: early experiments with arraylist seems to be worse, as I am adding now extra time to load the events into the array and then reading them again.
didn’t we agree a LinkedList was better in an earlier thread because your both building and accessing it sequentially?
In that five minutes, how much time is taken just reading the file? There is probably LOTS of room for optimization there. 100,000 lines is nothing, really unless each line is being parsed in an expensive way or something. Rough math says 3 ms per line (unless I shifted a decimal somewhere) which seems pretty bad.
Show us the code… maybe we can offer guidance. I’ve read 10’s of megabytes in seconds before.
If you load using the inputStream directly… wrap it in a BufferedInputStream:
[java]
InputStream in = new BufferedInputStream(originalInputStream);
[/java]
And u probably are right (bad coding in parsing), there you go, Be nice lol
[java]
/** Constructor: starts reading the file /
public Jme3Adapter( ) {
// This function has all the available motion commands to date
attachCommands(hm);
// reads trace file (reads .txt or .atf)
readFile();
}
/* this function opens and reads the file /
public void readFile() // This will eventually be populated using Nifty
{
// File Path for the model (atf dump file)
String url = "C:\Users\garnaout\Desktop\Accurev_workspace\JSLDriveModel\stresstest1\Output\stresstest1.3datf";
try {
/* Sets up a file reader to read the file passed on the command
line one character at a time /
FileReader input = new FileReader(url);
/* Filter FileReader through a Buffered read to read a line at a
time /
BufferedReader bufRead = new BufferedReader(input);
String line; // String that holds current file line
int count = 0; // Line number of count
// Read first line
line = bufRead.readLine();
count++;
parseLine(line, bufRead, count);
bufRead.close();
}catch (ArrayIndexOutOfBoundsException e){
/* If no file was passed on the command line, this exception is
generated. A message indicating how to the class should be
called is displayed /
System.out.println("Usage: java ReadFile filenamen");
}catch (IOException e){
// If another exception is generated, print a stack trace
e.printStackTrace();
}
}
/* this function parses the file read line by line /
private void parseLine(String line, BufferedReader bufRead, int count) throws IOException {
// Read through file one line at time
while (line != null)
{
processLine(line);
line = bufRead.readLine();
count++;
}
}
/* this function takes each lines and process it (this is the main adapter function) */
private void processLine(String line) {
//use a Scanner to parse the content of each line
Scanner scanner = new Scanner(line);
scanner.useDelimiter(" ");
if ( scanner.hasNext() ) // replace with a function/switch case for a better coding practice
{
part1 = scanner.next(); // first word read from file
//System.out.println("READING NEW LINE: First part " + (part1.trim()) + " and second part is " + (part2.trim()) );
if(part1.equals("time")) // separate times from events
{
if(scanner.hasNext()) {part2 = scanner.next();};
time_read = Float.valueOf(part2.trim()).floatValue(); // get trace time read so far
System.out.println("At this sim time " +part2);
}
else
{
/**create a list that stores all the trace arguments of a specific event */
AllEventsList.add(part2); // add the trigger time
AllEventsList.add(part1); // add the type of event
while(scanner.hasNext()) // add the rest of the events parameters
{
//String temp = scanner.next();
AllEventsList.add(scanner.next());
}
// close scanner to avoid memory leak
scanner.close();
// register all commands with mySpatial
for(Command cmd : hm.values()) {cmd.register(mySpatial);}
Command theCmd = hm.get(part1);
//System.out.println("Event Type "+part1);
//System.out.println("TheCommand: "+theCmd);
// execute the command
if(theCmd != null)
{
theCmd.run();
}
// clear the array list
AllEventsList.clear();
}
}
[/java]
A simple change:
BufferedReader bufRead = new BufferedReader(input);
Change to:
BufferedReader bufRead = new BufferedReader(input, 32000);
That might give a little boost.
I’ve never used scanner. It might be kind of expensive to create and setup once per line.
was about to say, you create a Scanner object every line, is there no way to create it once and use the existing object for each line? i also never used Scanner tho xD
what’s the 32000? it didn’t do any significant increase in performance?
what do you use to parse a file other than scanner??
well i do it by serializing the objects, but in your case it wouldn’t work as your given a file and told to parse it right?
yes true. I am going to try inputStream instead and see if that improves it.
Ok so I used simple string Split() with the following code:
[java]private void processLine(String line) {
/* String to split. /
String str = line;
String[] temp;
/ delimiter /
String delimiter = " ";
/ given string will be split by the argument delimiter provided */
temp = str.split(delimiter);
// read first word
part1 = temp[0];
if(part1.equals("time")) // separate times from events
{
part2 = temp[1]; // get time value
time_read = Float.valueOf(part2.trim()).floatValue(); // get trace time read so far
}
else
{
AllEventsList.add(part2); // add the trigger time
for(int i = 0; i < temp.length ; i++) // add the rest of the events parameters
{
AllEventsList.add(temp);
}
//System.out.println("Event Type "+AllEventsList);
[/java]
and that increased it by almost 3X. However the reading time now is still around 30 seconds for a file that was taking me 1.23 min. Is there any more room for optimization? Or more like do you recommend anything better ?
Hello, there is no way it takes so much time. You are doing “other actions” while you are loading the file thats why it is slow.
Use this code it shoudnt take more than 1 second to finish loading the file.
[java]
///
//Converts a file into an array of string
public static ArrayList<String> readFile(File file)
{
return readFile(file, "UTF8");
}
///
//Converts a file into an array of string
public static ArrayList<String> readFile(File file, String encoding)
{
ArrayList<String> lines = new ArrayList<String>(50);
if (!file.exists())
{
throw new IllegalStateException("Unable to find File " + file + " bye");
}//if
String line;
boolean end = false;
try
{
BufferedReader br = new BufferedReader(new InputStreamReader(new FileInputStream(file), encoding));
while (!end)
{
line = br.readLine();
if (line == null)
{
end = true;
}
else
{
lines.add(line);
}
}//while
br.close();
}//try
catch (IOException ioe)
{
System.err.println("Unable to open file");
System.exit(-1);
}
return lines;
}//readFile
[/java]