[SOLVED] BinaryExporter and materials


#41

Well interestingly /sarc now sdk doesnt show the files that are now part of the build/distribution.

However, if I open up this same project in netbeans (not sdk), they are there.

SDK doesnt show, netbeans does.

Edit: Same thing happens there though in netbeans if you delete the assets. They still show up in the distribution, at least visually.


#42

It looks to me that building custom jar distribution only thing that will work. The assets/lib folder properly updates.


#43

hmm, i dont had issues like this. not sure if this is old netbeans issues.(because SDK use some older version)

what files exactly dont show?


#44

Whats happening is the SDK/netbeans view of the contents of a build/distribution zip file does not update.

I can open the actual zip and everything is there or removed when removed.

No harm no foul.


#45

this sound like very serious issue. i didnt recently check builded zip or tar, because its not required in development.

when before i builded zip using gradle, i were veryfing how it look, and as i remember it had every file.

please note since you use gradle i think it might be more gradle related. might be worth try change build file?

edit: anyway why you need this zip file? you already make production version? if its about assets, load them directly, it will save a lot of time.(in development status)


#46

Its as basic as it gets. See above post. Happens only when opening the file by clicking on the zip inside the sdk/netbeans files view.

Most likely a bug thats fixed in versions other than 1.8.

I am going to change to netbeans 13 as soon as its out.


#47

you might be correct. i never open zip file via SDK.

fact is. i even didnt knew its possible :smiley:

so thats why i could never notice it. anyway looking via system file manager or any other program zip file was fine.

so look like netbeans version issue. still, the SDK update problem exists. i hope there will be someone to make external tool app outside of SDK. (or someone who will just update SDK)

i belive @Darkchaos dont have much time.


#48

Yeah, life got him. heh.

I use this now for sweeping the sdk project files into the gradle project and its pretty sweet.


task syncDependencies(type: Sync) {
    description 'Copies the assets directory from external netbeans project to the target directory.'
    from '../PathToSDKProject/assets'
    exclude ('**/*.j3odata','**/*.mesh','**/*.skeleton','**/*.mesh.xml','**/*.skeleton.xml','**/*.scene','**/*.material','**/*.obj','**/*.mtl','**/*.3ds','**/*.dae','**/*.blend','**/*.blend*[0-9]','**/*.bin','**/*.gltf')
    into 'assets'
}

You can automate it as you know with

compileJava.dependsOn(syncDependencies)

I think I may go the way @Ali_RS went and build individual jars for each asset next.

Looking forward to nb13 since there is no JavaFX-Gradle-Plugin for 1.8 now. Like waiting for a new toy at Christmas as a kid.


#49

Narrowed down the material problem causes but no solution for a non SDK fix.

It seems that if you use test-data model in the SDK by copy/paste they will work fine in SDK no matter where you put them. Even moving them around works fine.

However, the same model will not find its material in a gradle project unless this model is either in the exact same folder structure as in test-data or it has been converted to a j3o by the sdk and that model is in that exact same folder structure as the converted model.

So the jaime model for example in the test-data folder Models.Jaime will only work in the gradle project asset folder Models.Jaime even though it works anywhere in the sdk. You cant convert it so thats it for Jaime in anything except the SDK.

So to use external assets from the SDK, they must be converted in the SDK first and placed in the exact same folder structure in the gradle project.

What a pain in the ass this has been.


#50

For what it’s worth, I downloaded a bunch of different models from sketchfab to play with and plan to poke at what it would take to write a proper command line converter. (And if any core changes are required/desired to make that work.)

I don’t know how much luck I will have… but I didn’t want you to feel totally alone. :slight_smile:


#51

Misery does love company but I hope this one gets easier for everyone so they can avoid it.


#52
task syncDependencies(type: Sync) {
    description 'Copies the assets directory from external netbeans project to the target directory.'
    from '../PathToSDKProject/assets'
    exclude ('**/*.j3odata','**/*.mesh','**/*.skeleton','**/*.mesh.xml','**/*.skeleton.xml','**/*.scene','**/*.material','**/*.obj','**/*.mtl','**/*.3ds','**/*.dae','**/*.blend','**/*.blend*[0-9]','**/*.bin','**/*.gltf')
    into 'assets'
}

Well discovered some more quirkiness using gradle. Being new to me probably not to others but when syncing folders i found out that you can always create a sub directory, and add files to it in the β€œfrom” directory and it will propagate the β€œinto” directory.

However, at least for me, if i were to delete those files and or the sub directory, or move that directory, the β€œfrom” directory now has an empty folder where once there was none. The sync, at least for me, would not update the β€œinto” directory now because it found an empty folder in the β€œfrom” directory and ignores it. This means my β€œinto” directory was now out of sync with the β€œfrom” directory.

I created a chain of task calls to fix it using Pauls default gradle templates createDirs task so now I always have a matching synced folder when I want one.

The first task is still the syncDependencies task but now set to dependOn Pauls default template createDirs task which is set to dependOn a delete task.

task syncDependencies(type: Sync) {
    dependsOn.createDirs
    description 'Copies the assets directory from external netbeans project to the target directory.'
    from '../PolyAssets/assets'
    exclude ('**/*.j3odata','**/*.mesh','**/*.skeleton','**/*.mesh.xml','**/*.skeleton.xml','**/*.scene','**/*.material','**/*.obj','**/*.mtl','**/*.3ds','**/*.dae','**/*.blend','**/*.blend*[0-9]','**/*.bin','**/*.gltf')
    into 'assets'
}

I left the createDirs task intact but it could be cleaned up to remove unneeded things like creating stubs.

task (createDirs).doLast {
    dependsOn makePretty
    def pkg = 'mygame'
    def dirs = [
//        file("./src/main/java/$pkg"),
//        file("./src/main/resources"),
        file("./assets/Interface"),
        file("./assets/MatDefs"),
        file("./assets/Materials"),
        file("./assets/Models"),
        file("./assets/Scenes"),
        file("./assets/Shaders"),
        file("./assets/Sounds"),
        file("./assets/Textures"),
    ]

    dirs.each {
        if( !it.exists() ) {
            println "Creating " + it
            it.mkdirs()
        }
        if( it.listFiles().length == 0 ) {
            def stub = new File(it, 'removeme.txt')
            println "Creating stub file to allow git checkin, file:$stub"
            stub.text = "Remove me when there are files here."
        }
    }
}

The makePretty task.

task makePretty(type: Delete) {
    description 'Deletes all files in the target directory without deleting the target directory.'
    delete file('assets').listFiles()
}

This works real well and I can move things around in the β€œfrom” folder, add to or remove them, leave empty directories anywhere I want and its real fast.

If there are better ways to do this please advise.

Edit: Never mind. Failed again for some reason once all the folders were empty.
That loud smacking sound you hear is me punching the shit out of gradle.


#53

Note, I am not doing this way anymore. I do not know what is the reason you want to do this.
Now I am using just the default asset structure from Paul’s template gradle project. One asset.jar for all of my assets and using runtime files('assets') in build.gradle.

Edit:
I am posting my new approach in case:


I found that getdown already has a mechanism for this by generating patch file, which I think I am going to use this instead of writing my own patcher.


#54

Ok. read it in another post. Sounded good on paper.


#55

Found out my sync problem is a gradle bug. At least part of it is.


#56

I had that getdown bookmarked already. I dont even remember doing it but it looks like the way to go to me also.


#57

For what it’s worth, here is the product of my fooling around for this evening.

With a prototype release:

It’s a command line tool that can load a non-j3o model from one place, convert it to j3o in a new directory structure, including copying over its dependent textures.

Next would be to add some scripting to β€œadjust” things during the process as well as to do things like mark materials for export as j3m, etc…

But it proves that the concept works. I was able to convert some models I downloaded from Sketchfab to j3o into a Models/foo subdirectory, textures were copied into Models/foo/textures automatically… and the j3o loaded fine in my application.

The rest is β€œusability”. (For example, I really want to be able to rescale nodes, center them, etc… because a lot of models seem to treat 1 unit = 1 inch or 1 unit = 1 centimeter.)

Edit: if you download the zip and run the jmec script in there with no arguments it will provide command line help.

Edit 2: Added the temporary command line help to the project readme… repasted here while I’m spamming the thread:

Current command line help:
        _   __  __   ___    ___
     _ | | |  \/  | | __|  / __|
    | || | | |\/| | | _|  | (__
     \__/  |_|  |_| |___|  \___|

    JME3 Asset Converter v1.0.0

Usage: jmec [options] [models]

Where [models] are a list of j3o files.

Where [options] are:
 -sourceRoot <dir> : specifies the asset root for the models.
       Model paths will be evaluated relative to this root.

 -targetRoot <dir> : specifies the asset target root for writing
       converted assets and their dependencies.

 -targetPath <path> : a path string specifying where to place
       the copied/converted assets within the targetRoot.  Any
       internal asset keys will be 'rehomed' to this path.

Examples:

>jmec -sourceRoot C:\Downloads\CoolModel -targetRoot assets -targetPath Models/CoolModel C:\Downloads\CoolModel\AwesomeThing.gltf

 Converts C:\Downloads\CoolModel\AwesomeThing.gltf to a j3o and writes it
 to ./assets/Models/CoolModel/AwesomeThing.gltf.j3o

 Any dependent textures, etc. relative to C:\Downloads\CoolModel will
 be copied until the appropriate ./assets/Models/CoolModel/* subdirectory.

 For example:
    C:\Downloads\CoolModel\textures\awesome-sauce.jpg
 Gets copied to:
    ./assets/Models/CoolModel/textures/awesome-sauce.jpg

The point of all of this is that a) maybe this is the start of something that will work, and b) you can see the AssetKey magic that makes rehoming assets possible during conversion.


#58

I will use this today and read the code.

This is what this thread is about.

I had no idea you set all that stuff with asset key like that. I was just using newKey and passing it in.


#59

If you do that then it will lose all of it’s subclass-specific information if it’s a TextureKey, ModelKey, etc… JME doesn’t provide an easy way to create a repathed key so I used reflection magic.

I have some thoughts on an overhaul of the asset manager. It only does one thing well and the rest kind of… err… less well. (One thing well = caching)


#60

Worked on gltf box and duck from jme-testdata. Duck looks great. Then I tested it on something I made that has 7 pbr textures I created by hand and it had problems.

Shows texture missing when its there and had missing textures. This was exported with the gltf exporter for blender 2.79 so some of this may be from that. This particular model does not show these problems when using SDK to convert it though. However, your conversion blows it away for clarity and depth even without a probe for the textures that made it through.

Failed on ogre model. Think the missing ogg library played a role in that.

12:39:25,763 ←[1;32mINFO ←[m ←[36m[AssetReader]←[m ←[32mUsing source asset root:
assets\Textures
←[m12:39:25,771 ←[1;32mINFO ←[m ←[36m[AssetReader]←[m ←[32mFound assetConfig:jar
:file:/C:/Users/Robert/Documents/JMonkey/x/x/jmec-1.0.0-SNAPSH
OT/lib/jme3-core-3.2.3-stable.jar!/com/jme3/asset/Desktop.cfg
←[m12:39:25,836 ←[1;33mWARN ←[m ←[36m[AssetConfig]←[m ←[1;33mCannot find loader
com.jme3.audio.plugins.OGGLoader
←[m12:39:25,837 ←[1;32mINFO ←[m ←[36m[Convert]←[m ←[32mConvert:assets\Textures\O
to.mesh.xml
←[m12:39:25,837 ←[1;32mINFO ←[m ←[36m[AssetReader]←[m ←[32mloadModel(assets\Text
ures\Oto.mesh.xml)
←[m12:39:25,838 ←[1;32mINFO ←[m ←[36m[AssetReader]←[m ←[32mLoading asset:Oto.mes
h.xml
←[m12:39:26,331 ←[1;32mINFO ←[m ←[36m[Convert]←[m ←[32mLoaded:Oto-ogremesh (Node
)
←[m12:39:26,332 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32mNode(Oto-ogremesh) key:
Oto.mesh.xml
←[m12:39:26,332 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32m  Geometry(Oto-geom-1)
←[m12:39:26,333 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32m      Material[name=Mat
erial.002/SOLID/TEX/bluewarrior1024.j/VertCol, def=Phong Lighting, tech=null]
←[m12:39:26,333 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32m        Vector4 Diffuse
 : 1.0 1.0 1.0 1.0
←[m12:39:26,334 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32m        Boolean UseMate
rialColors : true
←[m12:39:26,334 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32m        Float ParallaxH
eight : 0.05
←[m12:39:26,334 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32m        Vector4 Ambient
 : 0.2 0.2 0.2 1.0
←[m12:39:26,334 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32m        Boolean Backfac
eShadows : false
←[m12:39:26,335 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32m        Texture2D Diffu
seMap : WrapRepeat_S WrapRepeat_T "Oto.jpg"
←[m12:39:26,335 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32m        Boolean UseVert
exColor : true
←[m12:39:26,336 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32m        Vector4 Specula
r : 0.0 0.0 0.0 1.0
←[m12:39:26,336 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32m        Float Shininess
 : 0.25
←[m12:39:26,336 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32mfindDependencies(Oto-og
remesh (Node))
←[m12:39:26,336 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32mfindDependencies(Oto-ge
om-1 (Geometry))
←[m12:39:26,336 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32mfindDependencies(Materi
al[name=Material.002/SOLID/TEX/bluewarrior1024.j/VertCol, def=Phong Lighting, te
ch=null])
←[m12:39:26,337 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32mChecking:Vector4 Diffus
e : 1.0 1.0 1.0 1.0
←[m12:39:26,337 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32mChecking:Boolean UseMat
erialColors : true
←[m12:39:26,337 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32mChecking:Float Parallax
Height : 0.05
←[m12:39:26,337 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32mChecking:Vector4 Ambien
t : 0.2 0.2 0.2 1.0
←[m12:39:26,338 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32mChecking:Boolean Backfa
ceShadows : false
←[m12:39:26,338 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32mChecking:Texture2D Diff
useMap : WrapRepeat_S WrapRepeat_T "Oto.jpg"
←[m12:39:26,338 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32mmaterial asset:Texture2
D[name=Oto.jpg, image=Image[size=1024x1024, format=BGR8]]
←[m12:39:26,339 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32mChecking:Boolean UseVer
texColor : true
←[m12:39:26,339 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32mChecking:Vector4 Specul
ar : 0.0 0.0 0.0 1.0
←[m12:39:26,339 ←[1;32mINFO ←[m ←[36m[ModelInfo]←[m ←[32mChecking:Float Shinines
s : 0.25
←[m12:39:26,340 ←[1;32mINFO ←[m ←[36m[Convert]←[m ←[32mDependency[file=assets\Te
xtures\Oto.jpg (Mipmapped), key=Oto.jpg (Mipmapped)]
←[m12:39:26,341 ←[1;32mINFO ←[m ←[36m[AssetWriter]←[m ←[32mCopying:assets\Textur
es\Oto.jpg (Mipmapped) to:assets\Models\Oto\Oto.jpg (Mipmapped)
←[mException in thread "main" java.io.FileNotFoundException: assets\Textures\Oto
.jpg (Mipmapped) (The system cannot find the file specified)
        at java.io.FileInputStream.open0(Native Method)
        at java.io.FileInputStream.open(Unknown Source)
        at java.io.FileInputStream.<init>(Unknown Source)
        at com.google.common.io.Files$FileByteSource.openStream(Files.java:126)
        at com.google.common.io.Files$FileByteSource.openStream(Files.java:116)
        at com.google.common.io.ByteSource.copyTo(ByteSource.java:267)
        at com.google.common.io.Files.copy(Files.java:315)
        at com.simsilica.jmec.AssetWriter.write(AssetWriter.java:103)
        at com.simsilica.jmec.Convert.convert(Convert.java:162)
        at com.simsilica.jmec.Convert.main(Convert.java:197)

Didn’t do a blend model since that library is missing also.

My gltf is awaiting redo when blender 2.8 gets released but it did work with the SDK convert as I wrote.


Β