[Solved] Migrating standard SDK project to Gradle Template

My game started in the SDK a few years ago so it’s using ant instead of gradle. I’ve been wanting to switch this project to Gradle for some time but have been postponing the switch because the project is somewhat large and I expect it won’t go as smoothly as I’d like.

I am planning on using the SDK’s gradle project template, and want to be prepared as best as possible. I have a few questions about using gradle and am also interested in hearing any useful advice - I have some experience creating small libraries using gradle in the past, but this project is much larger than those so I do have some specific concerns.

  1. First, I recall that a gradle project requires the Assets to be in a folder titled Resources instead and followed this format in my small gradle based projects. Is this always required, and are there any disadvantages to this? For example. I plan to still use the SDK and my own custom scene editor to edit assets in this folder, will this still be possible?

  2. Will the assets take longer to re-compile after I’ve made a small change and attempt to quickly debug the game again? (I recall a thread in the past where a user mentioned long asset re-compile times using gradle with their assets in the resources folder, although I could be remembering incorrectly)

  3. Is there any limit to the size of the assets folder with gradle? Currently a bug with ant has been preventing me from building my game because the assets folder is larger than 4gb. Pretty sure this is just a bug that I’ve acquired as a result of fiddling with this ant based project over the past 6 years, but I can’t find anymore info about fixing it so that’s a primary reason I’m switching over to gradle finally.

  4. Any other advice or tips for using gradle with a large project are greatly appreciated :slightly_smiling_face:

2 Likes

a gradle project requires the Assets to be in a folder titled Resources instead and followed this format in my small gradle based projects. Is this always required

It’s not required. Gradle allows source folders to be configured in the build script.

1 Like

We went with having the assets as resources inside the main project. Around 200 MB of assets. Netbeans/SDK seems to handle this fine. There is absolutely no problem with this setup. But having a separate asset project for these would probably be the best solution. Architecture-wise and to avoid, like you said, asset recompilation need if only code was changed. Gradle is smart but this might be a thing.

1 Like

1)

You can setup resource folder if im not wrong, same as possible to setup SRC dirs for java/groovy/etc.

2)

NO if you for example - will just do like i have:
SDK for Assets project
ANY IDE you like(SDK can be too) for code where:

runtime files("..//..//Assets/assets")

then it will not compile. But its for developer mode only, since for production you would need something like:

runtime project(':Assets')

if i remember correctly this would be for a production.

3)

there is ZIP limit(how much it can package, so i suggest use zip64 here):

tasks.distZip.enabled = false
tasks.distTar.enabled = false
tasks.distZip.zip64 = true

for a production, i remember above zip or tar need to be true if you want packed.(if im not wrong)

4)

Yes! Use Gradle multiproject BIG project (where you can split to smaller projects)

You can specify “allprojects {” there that will apply for all projects, but you also have separate gradle.build for each project.

in settings.gradle you just need specify what subprojects this subproject require like:

rootProject.name = 'NAME'
include 'core(ore other subproject name)'

last one tip, i dont remember why i had some problems earlier with version, but if you will also have, you have force mode:

    compile("org.jmonkeyengine:jme3-core:$jmeVersion") {
        force = true
    }

Glad you move to Gradle, wish you no trouble with it :slight_smile:

Gradle is magic for me, but:

itjustworks

1 Like

Note: I’ve never had to do this in any of my many many many gradle projects.

So another piece of advice for adopting a new build tool is to be careful of picking up unattributed lore that haunts you for the rest of your days. If you come across a problem that it looks like that would solve then so be it… but you probably also want to post here and find out what the real solution is.

And that’s not just picking on oxplay2’s advice, just advice in general to be careful adding odd-looking magic-sauce without understanding why.

1 Like

Thank you all for the very useful information, I feel a lot better about going forward with the migration now :grinning:

Once I’ve tied up some loose ends with other stuff I’m working on with this project, then I’m going to finally focus on switching over to Gradle, and will likely post again when I have anymore questions or issues with the process.

3 Likes

I created a new gradle based project in the SDK using the “Basic Gradle Game” Template and copied my entire asset folder over to the “asset” folder in this project, and am attempting to load in one of my models next to the blue cube first.

I am waiting to copy over my code from the src file and setup all the dependencies until I can load an asset properly and build a runnable version of the game, to minimize the amount of things I’m troubleshooting at once.

But I am getting an AssetNotFoundException when I attempt to load in my model, even though it is indeed the right path (and the SDK autofill confirms this). So my guess was that I need to do a clean and build to compile the assets jar first.

But when I try to do a full clean/build, I am getting a similar “assets too big” error as I got with ant.

> Task :assets:jar FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':assets:jar'.
> archive's size exceeds the limit of 4GByte.
  
  To build this archive, please enable the zip64 extension.
  See: https://docs.gradle.org/6.3/dsl/org.gradle.api.tasks.bundling.Zip.html#org.gradle.api.tasks.bundling.Zip:zip64

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 18m 14s

I tried to include the zip64 = true line as @oxplay2 suggested, and I followed the documentation that the error provided, but I cannot seem to find the correct place in my build file to put the zip64 = true line.

I tried a google search and couldn’t find anything that works, the best looking solution was to put that line in the jar { } section of the build file, but this didn’t work, so I tried other places which also didn’t work. Then I stumbled upon some stack overflow posts suggesting to put it in a shadowJar { } bracket which seems to also rely on some other dependency (I think?) and I tried working with that but it didn’t seem to work or have a lot of information I could find, and IDK if that’s even what I need.

The build time also takes 18 minutes before I find out if my attempt failed (oof) and have tried everything that I thought was the right way to do it. So hopefully someone can help me figure out how to get zip64 working.

Here is my gradle build file, I don’t think I’ve changed anything since the SDK project auto-generated it, other than attempting to place zip64 = true in multiple places with no success - it always produces the same build error as I posted above


plugins {
    id 'java'
    id 'application'
}

group 'com.mygame'
version '1.0'

sourceCompatibility = 1.8
mainClassName = "com.mygame.Main"

repositories {
    jcenter()
}

project.ext {
  jmeVer = '3.3.0-stable'
}

project(":assets") {
    apply plugin: "java"
    
    zip64 = true

    buildDir = rootProject.file("build/assets")

    sourceSets {
        main {
            resources {
                srcDir '.'


            }
        }
    }
}

dependencies {

  implementation "org.jmonkeyengine:jme3-core:$jmeVer"
  implementation "org.jmonkeyengine:jme3-desktop:$jmeVer"
  implementation "org.jmonkeyengine:jme3-lwjgl:$jmeVer"
  
    

}

jar {    
    
    zip64 = true

    manifest {
        attributes 'Main-Class': "$mainClassName"
    }
}

I tried to include the zip64 = true line as @oxplay2 suggested, and I followed the documentation that the error provided, but I cannot seem to find the correct place in my build file to put the zip64 = true line.

in build.gradle (directly without project(":assets"), but since you have assets project i suppose its correct there, anyway it need to be tasks.distZip.zip64 not just zip64 i assume.):

//todo:
//enable epel repository
//yum install zip lrzip lzip p7zip unzip p7zip

tasks.distZip.enabled = false
tasks.distTar.enabled = false
tasks.distZip.zip64 = true

Also maybe you got missing package in system.

Anyway here is example link that use this parameters:
https://docs.gradle.org/current/userguide/distribution_plugin.html

(tho please note i didnt use this plugin)

Were testing it on single-gradle-project and it was fine.

Anyway it’s for “production”, because for development noone wants compile/zip/jar all assets every time.

I created a new gradle based project in the SDK using the “Basic Gradle Game” Template and copied my entire asset folder over to the “asset” folder in this project, and am attempting to load in one of my models next to the blue cube first.

I am waiting to copy over my code from the src file and setup all the dependencies until I can load an asset properly and build a runnable version of the game, to minimize the amount of things I’m troubleshooting at once.

But I am getting an AssetNotFoundException when I attempt to load in my model, even though it is indeed the right path (and the SDK autofill confirms this). So my guess was that I need to do a clean and build to compile the assets jar first.

generally assets(resources) are in

"/src/main/resources"  

(that i left default but can also setup as i seen)

Tho i were very surprised that when using separate project for assets:

runtime files("..//..//Assets/assets")

this somehow work for me without redirecting it into “/src/main/resources” (and i didnt really investigate why it work for now)

Anyway i got single-gradle-project here and if i just put assets into:

"/src/main/resources"  

it will work.

You can also look how public projects like Minie are made, and here is example(where you can see Oto model):

(please note that JME3Examples is sub-project so its similar like gradle-project standalone)

while

i see Sgold used interesting way, via using

assetManager.registerLocator("TerrainGridTestData.zip", ZipLocator.class);

where this file is also not in “/src/main/resources” and it works.

I appear to have got the zip64 working by putting it like this:

project(":assets") {
apply plugin: "java"

  jar{
      zip64 = true
  }
  
  buildDir = rootProject.file("build/assets")

  sourceSets {
      main {
          resources {
              srcDir '.'
              zip64 = true  //pretty sure this does nothing here, but I didn't want to wait another 20 minutes of build time so I left it
          }
      }
}

}

I wasn’t sure if i needed to set the other 2 variables to false or not since the documentation did not mention it, but in this case it seems to have finally successfully built the game without the size limitation error.

But now I still can’t run it (in the SDk nor the distrubatle copy) without getting the asset not found error. So I got the assets to compile finally, but I must still be misunderstanding something that they aren’t being found by the asset manager.

I thought it was okay to leave the assets in the assets folder? This seems to be the default for the SDK gradle template, although I still am getting the AssetNotFound error so I must be doing something wrong still.

I thought it was okay to leave the assets in the assets folder? This seems to be the default for the SDK gradle template, although I still am getting the AssetNotFound error so I must be doing something wrong still.

Generally what i see is that you have 2 projects. Main :project and :assets project

while i do not see that Main project Require assets project so im not sure if its using it at all.

just guessing now but might be worth to check

compile project(':assets')

in dependencies of main project

or like i do in multi-project:
in settings.gradle:

include 'assets'

never did project a way you did in gradle now, so thats just my guess

Maybe you will be interested in gradle multi-project template? (same as Minie and i have)

here is example link:
https://docs.gradle.org/current/samples/sample_building_java_applications_multi_project.html

and you could also look at Minie lib

Also for a Development mode, to skip waiting that 20 minutes every time, i suggest try something like:

runtime files("..//..//Assets/assets")

in your dependencies.

1 Like

Putting this in the dependencies bracket seems to cause a bunch of errors with all of my imports, its like it messes up the dependency for the engine libraries. Is this not what you meant?

dependencies {

  compile project(':assets');
    
  implementation "org.jmonkeyengine:jme3-core:$jmeVer"
  implementation "org.jmonkeyengine:jme3-desktop:$jmeVer"
  implementation "org.jmonkeyengine:jme3-lwjgl:$jmeVer"

}

What is the benefit of doing a multi-project? From my understanding, it seemed like this is used for projects that have multiple parts that can be compiled as a whole, or compiled and used separately, so I’m not certain if it would be suited best for this project since it isn’t made to be split up into smaller projects. But I’m interested in hearing if there are more benefits or if I’m mistaken.

Adding this worked to make my project load the model in the SDK, thanks. That’s also great to know that it won’t take as long to build the assets every time this way

exactly. both as i noticed.

For me, im using it as “multiple parts” since for example main game require :core and other projects.

But you can also see Minie example where some projects are standalone like Jme3Examples in it, that require just implementation project(’:MinieLibrary’), or DacWizzard that is also standalone project there that also require just library.

Adding this worked to make my project load the model in the SDK, thanks. That’s also great to know that it won’t take as long to build the assets every time this way

Great news!

But we will still need solve Production mode issue. Maybe Gradle multi-project will solve it itself.

I suggested multi-project here because i seen you were using :assets as separate project there.

If you will look at Minie example, it also have assets as separate project " MinieAssets"

1 Like

what about this

runtime project(':assets')

Edit:

or

runtimeOnly project(':assets')

if you are using newer gradle versions.

1 Like

also, im not sure if i told.

but if you have separate “libs” folder where you got JARs for example.

you can do:

run {
    systemProperty "java.library.path", 'libs' //where libs is a folder
}

edit:

also forgot to tell. Multi-project have also one big advantage:

you have additional shared build.gradle file for all projects where you can write “code that normally would be dupplicated for each project”. For example repositories setup or some vars / etc, etc

As you noticed Gradle is very elastic, so you can setup own way of building your game here, so shared build.gradle is very helpfull here.

1 Like

For example, in my case, I have separated each of these projects into a separate module (subproject)

game // for game components (ES) and RMI interfaces used by both client and server

game-client // for view related stuff like app states, UI and client services (implementation of RMI interfaces for client)

game-server // for game systems and hosted services (implementation of RMI interfaces for server)

account-server // a rest server for user registeration

master-server // a rest server to keep track of game servers, lobbies, social profile,…

1 Like

It sounds like the multi project could be useful for my custom scene editor when I eventually switch that over to gradle as well. But the way my main game project is setup might not benefit as much from being a multiproject, as I have it all packed together aside from a few things that I’ve already separated into libraries for my scene editor, and import to the main game.

I also must have been mistaken when I said that I fixed the issue with zip64… I thought I did because I successfully built the game and got the jar and batch file in the distribution folder from 30 minutes ago, but I must have accidentally changed something since then that broke things, because my most recent attempt at building it to test your recent suggestion of :

failed for the same 4gb limit error. Right now my build file is littered with zip64 = true in every place that it will let me put it without preventing a clean & build attempt. But since the build then takes 18 minutes, it’s hard to figure things out by trial/error which is my usual way when I have no clue what is going on…

Anymore insight into using zip64 would be extremely helpful, because this is what my current build file looks like, and even if the current build attempt succeeds again, I will have no clue why :sweat_smile:

plugins {
    id 'java'
    id 'application'
}

group 'com.mygame'
version '1.0'

sourceCompatibility = 1.8
mainClassName = "com.mygame.Main"

repositories {
    jcenter()
}

task{
    zip64 = true
    
}

project.ext {
  jmeVer = '3.3.0-stable'
  zip64 = true
}

project(":assets") {
    apply plugin: "java"
    
    
    
    jar{
        zip64 = true
    }
    
    buildDir = rootProject.file("build/assets")

    sourceSets {
        main {
            
            resources {
                
                srcDir '.'
                zip64 = true
            }
        }
    }
}

dependencies {

  runtime files("assets")
  
    
    
  implementation "org.jmonkeyengine:jme3-core:$jmeVer"
  implementation "org.jmonkeyengine:jme3-desktop:$jmeVer"
  implementation "org.jmonkeyengine:jme3-lwjgl:$jmeVer"
 
    
    
     

}



jar {    
    
    zip64 = true
    manifest {
        attributes 'Main-Class': "$mainClassName"
        
        zip64 = true
    }
}

Have never used zip64 but another option (if zip64 did not work) may be to manually zip the assets and add them to your distribution for production, instead of building it with Gradle?

then you can get rid of this block altogether

project(":assets") {
    apply plugin: "java"
    
    
    
    jar{
        zip64 = true
    }
    
    buildDir = rootProject.file("build/assets")

    sourceSets {
        main {
            
            resources {
                
                srcDir '.'
                zip64 = true
            }
        }
    }
}
1 Like

I will do this as a last resort until I can get the zip64 working, but I’m still going to try to figure out zip64 if I can, since I think it would be better to have it in a compiled jar that’s not as easy for a player to mess with.

Plus the only 2 real reasons I’ve decided to switch to gradle in the middle of this project are to be able to compile >4gb of assets, and also to be able to use the newer releases of JME easily.

So although it may take a while for me to figure out zip64 with the long build time, I’m going to continue trying things out with zip64 and am open to suggestions.


I’m also running into some troubles importing some libraries. Specifically Minie and Lemur. This is the relevent part of my build file:

repositories {
    mavenLocal()
    mavenCentral()
    jcenter()
    maven { url "https://oss.sonatype.org/content/repositories/snapshots/" }
    maven { url "https://oss.sonatype.org/content/repositories/releases/" }
    maven { url "https://jitpack.io" }
}
dependencies {

runtime files("assets")
  

implementation "org.jmonkeyengine:jme3-plugins:$jmeVer"  
implementation "org.jmonkeyengine:jme3-core:$jmeVer"
implementation "org.jmonkeyengine:jme3-desktop:$jmeVer"
implementation "org.jmonkeyengine:jme3-lwjgl:$jmeVer"
implementation "org.jmonkeyengine:jme3-effects:$jmeVer"
implementation "org.jmonkeyengine:jme3-networking:$jmeVer"
implementation "org.jmonkeyengine:jme3-lwjgl3:$jmeVer"
implementation "org.jmonkeyengine:jme3-niftygui:$jmeVer" 
 

implementation "com.github.stephengold:Minie:4.6.1"

compile "com.simsilica:lemur:1.14.0"
compile "com.simsilica:lemur-proto:1.11.0"

 
run {
    systemProperty "java.library.path", "libs" //where libs is a folder
}

implementation "com.google.guava:guava:31.0.1-jre"
implementation "org.codehaus.groovy:groovy-all:2.4.12"
implementation "org.slf4j:slf4j-api:1.7.25"

}

The rest of the dependencies are fine, but when I have the link to lemur and minie included, it breaks all my imports to every other dependency.

I also tried putting minie and lemur the jars I already have from my ant project into a “libs” folder as @oxplay2 suggested, but it doesn’t seem to be finding them:

    run {
        systemProperty "java.library.path", "libs" //where libs is a folder
    }

I put the libs folder in the root directory of my gradle project, then tried the build.

I’d obviously rather figure out how to import Minie and Lemur the proper way with the implementation link so I can access new releases easier, but I do have other custom jars that I don’t have hosted anywhere that I will need to include, so I need to figure out how to include a folder of jars regardless.

Can you post the Gradle error log?

By the way what Gradle version are you using?

Edit:

I think compile is removed in newer versions of Gradle, so you might use implementation instead.

2 Likes

I just tried to get the error again, and suddenly the imports work… I have no clue what made it work now, but it seems like Gradle takes a while to update the code files that are open in the IDE after I’ve made a change to the build file, and after I took a break to play another game with the project open, the imports for Minie and Lemur all suddenly worked when I came back to them.

It seems like there’s also no progress bar or anything to let me know its still doing stuff, it does say “scanning background projects” for a few seconds after I make a change to the Gradle build file, but then usually it still takes a while longer to register changes, with no indication when it’s done. And I guess in this case it just took really long? I guess I just have to get used to some of the unique quirks of using Gradle in the SDK still…

I also believe I’m using Gradle 6.3. It looks like that’s the version that the gradle SDK template gets set up with.