Questions about writing custom storage formats

I am working on some custom file formats for storing asset data, and am not well versed in the different audio or image formats and how they get used by the GPU/audio card.

When an image is in different formats, such as BGR8 and ARGB8 for example, is there any processing before these get sent to the GPU or do they get sent as is and the GPU gets told what image format it is receiving? Is one faster than the other for the GPU to process?

The reason I ask is do I just leave images in the format I receive them, or do I convert them to a different format for faster load times?

The same question applies with audio data.

I have implemented a (very basic) asset streaming pipeline in jmonkeyengine for my Ouside Engine, and the main instigator of building custom formats is faster stream times for high resolution models, but while I am at it, I am going to start working on custom formats for images, audio, and materials as well. Outside does not use j3o as Outside is designed for multiple backend renderers, and I do not want the asset files to be dependent on the jme version. But as you can imagine, when a single model is 400 MB in a gltf format, it takes a LONG time to load, even the j3o version can take a bit just to pull off disk due to the size (which is something that cannot be changed or worked around).

The goal is minimal processing of the assets, being able to put the data right into the buffers and send to the gpu. Oddly enough I thought I was going to find some standard formats for this, but have yet to come across this. Any advice is greatly appreciated, especially on storing images and audio formats.

Thank you,
Trevor

PS: Sorry for the long post :stuck_out_tongue:

1 Like

You can take a look at Webp image format and Gltf Draco mesh compression.

1 Like

That is interesting, I was not aware of that extension. The only disadvantage is the overhead of processing the json, which can be quite large depending on the parsing library.

For images you might find KTX useful, though I’m not sure of its status with OpenGL.

1 Like

That looks useful, thank you @danielp . Too bad there is not one for models.

Yeah - I think the difficulty lies in a format that’s so close to hardware. It can be difficult to manage an almost direct-to-GPU format in a way that’s useable across a broad range of hardware.