How to faster load game assets?

Hi. I have currently 120 textures that I need to load. This takes about 2 seconds with a parallel list (the textures are loaded in parallel in different threads. While this sounds fast, it’s really not. If I have 10 times more textures to load I need already 20 seconds. Loading from ZIP file vs. individual files is not really faster.

Is there a faster way to load textures/assets? Maybe if I load the textures once and store a serialized version of Texture2D on disk?

Or is this normal and everybody lives with it? That’s why I see always the progress bars in games on start? I don’t really have good experiences with this. But every game have like a one to five minute loading time for the game assets.

I think it’s better to cache relatively large assets during the booting time, you can run the asset loaders in other threads and use callbacks to get the asset objects as a return to the jMonkeyEngine thread (and dismiss progress screens), for that reason, you could use:

Java’s IO is also slow (compared to doing IO in C etc). And it is easy to create code that make its performance even slower it has to be. There has been some improvements to file readers in jME for the past versions. Are you using your own code to read these textures or jME? And if the latter, better use the latest version. If jME seems excessively slow, it would be nice to profile and fix these bottlenecks if at all possible.

Not very different if the Java code is written properly. …this coming from someone who has written command line Java tools that were faster than their C counterparts even including the JVM startup time (which used to be measurable).

Loading a lot of data will take a lot of time. Loading a lot of data from slow disks will take a lot of time. These are immutable facts and C versus Java doesn’t really come into play beyond 5% or so. In Java, loading a texture and sending it to the GPU is mostly native code anyway.

It’s part of the reason.

In your case, texture atlases, different file formats, etc. can all help. We don’t know enough about your scene or why you need so many textures in the first place. 120 is already kind of a lot and it seems like you are talking about having a lot more.

now you can imagine why in Assasin Creed Odyssey, when you rotate camera, it make disc read. Probably they cache what they need and control it. Tho its very destructive for disc if not SSD.

Also there is reason why a lot of games recommend SSD, so even with “minimal requirements” they say “SSD (Preferred), HDD (Supported)”

Some games also load lower version textures, and slowly replace them with higher.
Or this “texture stream” thing, but im not sure how it work exactly. (it might be gpu optimize only, not disc)

I just use the AssetManager#loadTexture. The textures are in a Zip file and I use the ZipLocator. I do a profile later.

Right, I forgot about this. I need to look into it how I can organize my textures better.

Agree.
BTW
Price of SSD is reaching a historical low, it is time to think about a 2T x 2T SSD RAID0 set :wink:.

Just got two Samsung high performance 8TB SSDs for 1k. Very cheap for high speed solid state at those sizes.

so true about price, also if want to be even safer can make like 4T x 2T SSD due to bigger lifetime of one of them.

Samsung’s SSD may have problem and it’s price is not that “fair”.
Search keywords “glowway” or “zhitai”, these prices are real yummy. :wink:

For data safety, why not set auto backup to an additional cheap HDD.
SSDs are supposed to be consumptions, save the money for a HDD maybe a better plan.

BTW
I don’t think low level IO are much different in languages. There are so many issues need to be considered:

  1. If disk loading IS too slow, why not compress your data in a “better” algorithm other than zip?(some zip just “zip”) and the java zip library is not multi-core friendly. With a multi-core solution, your end-to-end loading time maybe shortened.
  1. Try to save your data in one big file block rather than many small blocks. For example, a morden SSD can do sequential read at 7000 MB/s, yet much slower in random read. Deep down to the Motherboard there is only ONE reading abstraction or channel.
  1. I’m using a 4T x 4T HDD raid0 system, even raid0 may have some bottlenecks. HDD catch is one of them, this is why sequential reading is the key to the loading speed.
  2. Multiple request of reading in any language is a disaster. Each request will init a reading procedure which is “time” and each request consume IOPS which is also “time”.

So,:

  1. save data in one big file with high compress ratio algorithm.
  2. read the file in one thread then process in multi-threads.
  3. try memory-mapped I/O
    hope these helps :slightly_smiling_face:
1 Like