Unified build process

Hey all,



So it seems to me that one of the major problems people are having with the current setup is the majority of options in building.  By options I mean: Eclipse/NetBeans and SVN/Maven/(CVSobsolete?). Now I am not very familiar w/ Maven, but it seems that it is just another 'distribution' protocol similar to SVN but has some 'auto-build' stuff too.  The one person I have helped with Maven had stack overflows just trying to build jME initially; and although that may have been issue w/ the user's NetBeans, it also may have been an issue w/ the pom file.  But if it can ease the getting/building of jME without creating more complexity/problems and if its widely available for a number of IDEs then I am all for it.



Regardless, I think it would be a good idea if we could settle on one 'standard' build process; possibly with several other 'options' available for people that don't want it 'our way'.  If possible, it should be an IDE independent solution; maybe we could use ANT scripts and eliminate the whole 'classpath' problem (then again maybe that's to complex)…



call this a probe constructive comments/feedback :slight_smile:



(hehe, im probing…)

basixs said:

Now I am not very familiar w/ Maven, but it seems that it is just another 'distribution' protocol similar to SVN but has some 'auto-build' stuff too


Free Book: http://books.sonatype.com/maven-book/index.html

Wait a sec!! Maven is a Build Tool nothing to do with SVN, much like Ant but much more powerful. You could easily call an Ant script from Maven if you wanted so Maven > Ant (in fact it was born because of Ant's limitations). That said, Maven is not actually just a Build Tool, it's a project management tool. The pom.xml file describes the entire project and build proccess. Maven is supported via plugins in most IDEs, NetBeans and Eclipse for sure. By describing the project only with the pom, it should behave exactly the same in every IDE and every OS.

It beats Ant at building because Ant has no rules, every piece of an Ant script behaves like a "goto", it's not smart. Maven has phases: validate, compile, test, package, integration-test, verify, install and deploy. Then you attach goals to those phases making it much more simple to know when it's going to happen. I comes already with java goals and the default project is java so you don't have to configure nothing to work on a java project.

It has a set generation tools, think like Ruby on Rails, you just say "I want a this type of project", (java, groovy, web service, web page, etc, etc, etc) and it automatically creates a pom.xml and some skeleton folders and files.

Also, Maven handles dependencies, so I can state in the pom file that I need "junit version 4.5" Maven will automatically download that jar file (from the default global maven repository) and all of it's own dependencies too recursively. It will also bundle those when doing the deploy and will automatically find the when running. All dependencies are versioned which help a lot!

It doesn't stop there, you can then configure many ways of running and deploying at the same time, you could make many distributions lke a zip, a tar.gz etc or have many outputs like a src.zip and the runnable.jar or whatever you could think of. Also you can create profiles like for example "development" and "production" and have different configuration for those, or you can have profiles defining an OS and get the right one chosen automatically.

Maven can also connect to SVN or CVS repo and download the source code of any dependency for you if you wanted. It could search through the files your are building and replace certain tags you make with another values, imagine having several xml files and leaving a ${my_name} tags in there, it could search for those ${my_name} and replace it.

pom inheritance is another feature, you can have a really complex hierarchy where you could re use already defined stuff or override it, think OOP!

On top of that every single piece of maven is pluggable, you could make a java plugin to enhance it in almost any way you could imagine.

That said you should now be loving Maven. Now let's talk about jME after a lot of effort I managed to create a pom.xml that describes a custom project that uses jme2. It automatically manages all dependencies, including the natives files!!! when running the app and it also deploys it nicely in a zip files that contains the runnable jar and a lib folder with all necessary libs ready to give to the end user.

Using my way the steps to get jme2 running would be:

Step1:: Build JME2

  • checkout code from svn (run a single command on the console)

  • add libraries to your local maven repository (run a command for every jar file)

  • build jme (single command on the console)



Step2:: Create Project

  • build default maven project (one command)

  • replace pom.xml with mine

  • add another xml, no need to modify it

  • add some sourcecode that actually uses jme



Done. The project compiles, tests, run and deploys.

Step3:: Want to use an IDE? Duh!

  • Open your IDE

  • File->Open Project

  • Navigate to the folder your maven project was created and click OK

  • Write your code

  • Compile

  • Run

  • Test?

  • Wait second! That's exactly the same you do for every normal java project!! Yeah, it's as if maven wasn't there...



My approach is not complete. The things it lacks are:
Step1: Having a Maven Repository hosted here and having the jar files and native files hosted there, by doing that people wouldn't even have to download jme2 source code, eliminating Step 1 altogether
Step2: a project using jME should do pom inheritance, making it much simpler to configure, just a few lines to have you pom ready
Step3: Couldn't be any simpler, it's just a regular project from your IDE point of view

For the beginner, Maven is invisible, for the hardcore is a freaking awesome tool and it's really not that hard to understand, it's much much much simpler than ant yet it's much more than just a build tool. It seamlessly integrates into every aspect of project management and is great for sharing the project among many people.

Setting up a maven repository here would be really simple, using a Archiva wich comes bundeled with a Jetty server it just unzipping and running a single command to get it up. Then just really simple configuration of it (from a web browser), just create a few users with rights, or create non at all just leaving the admin, upload the jar files and done. I could maintain it if you let me.

I would really appreciate if everyone took the time to understand what I'm saying and really consider using maven as the default for managing jme2 project and projects that use it.

PS: Wow, I took so much time writing this that my session timed out and lost the whole post, lucky I already learned that lesson long ago and backed up before posting...

EDIT: PS2: Thanks basix for bringing this up and also for helping me with my guide

Today I worked on making a custom jme2 maven archetype called jme2-quickstart. What does this mean? Archetype is just a fancy word for prototype, I made a project prototype that will build with a single command  this structure:



project-name

|-- pom.xml

-- src<br /> &nbsp; &nbsp; |-- main<br /> &nbsp; &nbsp; |&nbsp; &nbsp; |-- java<br /> &nbsp; &nbsp; |&nbsp; &nbsp; |&nbsp; – App.java

    |    -- resources<br /> &nbsp; &nbsp; |&nbsp; &nbsp; &nbsp; &nbsp; – Monkey.jpg

    |-- META-INF

    |  -- assembly.xml<br /> &nbsp; &nbsp; – test

        -- java<br /> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; – AppTest.java



Actually java files will be inside folders like java/com/mycompany, those will created be automatically with the name you specify. It includes a complex pom.xml that will manage all dependencies and packaging of the project, including the native files and stuff like that. When run you will see a ball in the middle of the screen with the jme logo.



The command would be:

mvn archetype:generate -DarchetypeGroupId=com.jmonkeyengine -DarchetypeArtifactId=jme2-quickstart -DarchetypeVersion=1.0 -DgroupId=com.mycompanyname -DartifactId=coolgame -Dversion=1.0 -DinteractiveMode=false



After that the project can be built and run without any configuration whatsoever.

If this archetype was on a repository as well as the jME2 build and dependencies, the whole jME2 setup would become that command, all the required files would be downloaded automatically. I will say it again, the whole setup could become just one, command, only one.

Could the site host a maven repository?

i think for people familiar with maven it would be a nice way to set up a project.


Core-Dump said:

i think for people familiar with maven it would be a nice way to set up a project.


You are missing the point completely, it's not for people familiar with maven, I learned maven in the past month and it allowed me to accomplish this level of control. For people not familiar with maven it would be just a regular IDE project that can be used from Eclipse or Netbeans. It's all automatic!!! That's the whole point, it's for people not familiar with complicated stuff. The other approaches are "tried and proven" yeah right, tried and proven NOT to work because once a week a new comer pops up and says "hey I can't do the setup right, I need help".

Why is it so hard to try something new? I'm trying to improve jME but all I hear are lame excuses, someone give me a real reason why not to use maven and I will stop bothering with it.

I don't see why not. We should be able to have the old ways coexist with a maven project, no? Tomygun, if you can provide complete step by step instructions (via a setup guide in the wiki) and a complete POM (I think you have already) then I don't know what the hold up is.

Tomygun , i think you misunderstood my comment, sorry :)  (i'm often a bit short on words in my comments as english is not my native language)



I think its a great Idea!

Its just that maven also needs to be downloaded and installed (but ofcourse thats simple), thats why i added 'for people familiar with maven'.



I'm also using maven currently in a j2ee project and its really a very useful tool.

Hello!



I am using the supplied Maven-Project already and it works like a charm to build Jme-2.0 (Ok, running findbugs and PMD takes a while). I is an excellent way to build JME. Given the good integration of Maven via plug-ins into the popular IDEs like Netbeans, Eclipse and IDEA it takes a lot of hassle out of the setup phase. Whoever wrote the pom.xml, I am very grateful for it. Having a jme-archetype would be wonderful! Go for it!



A happy new Year everyone,

    —Florian 

nymon said:

I don't see why not. We should be able to have the old ways coexist with a maven project, no? Tomygun, if you can provide complete step by step instructions (via a setup guide in the wiki) and a complete POM (I think you have already) then I don't know what the hold up is.


Yeah, that's not a problem, as long as the build.xml is there it's still an ant project too. I wrote a maven guide and it's on the wiki but it doesn't use the archetype to do the setup because we need the archetype to be hosted in a maven repository*.

Core-Dump said:

Tomygun , i think you misunderstood my comment, sorry :)  (i'm often a bit short on words in my comments as english is not my native language)

I think its a great Idea!
Its just that maven also needs to be downloaded and installed (but ofcourse thats simple), thats why i added 'for people familiar with maven'.

I'm also using maven currently in a j2ee project and its really a very useful tool.


No problem (english is not my native language either so sometimes parts of my comments don't make much sense). You are right, people will have to install maven so that's a new step, that should be simple but could bring new problems with people not familiar with it. I'm using it at work also for regular java projects, it took me a whole week of annoying the hell out of my boss to do so :D (and now he sees it was a good idea).

wallner said:

Hello!

I am using the supplied Maven-Project already and it works like a charm to build Jme-2.0 (Ok, running findbugs and PMD takes a while). I is an excellent way to build JME. Given the good integration of Maven via plug-ins into the popular IDEs like Netbeans, Eclipse and IDEA it takes a lot of hassle out of the setup phase. Whoever wrote the pom.xml, I am very grateful for it. Having a jme-archetype would be wonderful! Go for it!

A happy new Year everyone,
    ---Florian  


Glad someone likes it (I didn't write that pom but I'm thankful for it too). If we had a repository people wouldn't have to build jme, they would just automatically download the latest jar and all of it's dependencies when trying to build for the first time their newly created project, only once because those will be shared for every other project the user makes.

*Maven Repository
Last night I went on a google rampage trying to find maven repositories that hosted jme dependencies. I discovered that java.dev give creates a repo for each project so jinput, glugen and the others are hosted there. All but lwjgl... it's been discussed on their forums so I don't think they would bother doing it. I also found that project darkstar hosts in it repo all those dependencies including jinput and stuff but prefixes them like this: com.projectdarkstar.ext.org.lwjgl, then we could

a) mark our dependencies like that (not a good idea to depend on darkstar project dependencies)
b) create a jme2 java dev project and do the same they do like com.jmonkeyengine.ext.org.lwjgl and also hosting the jme2-quickstart archetype.

If an admin could create the jme2 in java dev I would configure and upload all the stuff need for the repo and maintain it uploading the latest jme2 build once a week or so. I would revamp for the third or fourth time the maven setup guide in the wiki to show how to do the new setup. As that would be just a single step the guide could focus more on installing maven and using the IDEs with it.
basixs said:

TomyGun, lets not get carried away.   Maven seems like a great technology, but there are already several issue that have much higher precedence than a "Unified Build Process" (such as what to do about 1.0).  This topic was for suggestions and possibly a plan (if one is needed), I understand you are very excited about Maven but something like this should be planned out to the best of our abilities then implemented (when it is time) with care to avoid any issues.


When is the right time? Never? You know what? Let's do nothing at all. Instead of community driven this looks like community stopped, where everyone handles their little help not to do stuff. I'm trying to help with this stuff as well as documentation but everyone is just too busy to say "OK, let's do it". Want to know what to do with jme 1? Burn it, move on, do something new, whatever it is, don't hide behind "we need to think carefully" but think too carefully too much time.

I'm sorry if I'm being rude, I don't want to waste more efforts. I won't suggest anything else, I will just stick to asking help when I need it to use the engine.

Happy new year.

Don't let yourself stop from a few single opinions.

We all have different views of things, and maybe basixs is maybe a little bit over cautious here :slight_smile:

Given that there is now a pom.xml which builds a jar, is it possible to set up a remote repository for JME online where others can get access to the latest build?  The Sonatype Nexus repository is quite good.



The pom.xml refers to a fictitious repository at  http://www.jmonkeyengine.com/maven .  I think it would be brilliant to make this a real repository and do a mvn deploy to this repository when the code base is updated.



Thanks,

  Stephen

I'd like to put in a vote against using maven. A lot of people (myself included) have been burned by using maven on projects and have an almost pathological dislike of it as a result. Before we look at moving to maven what specific problems are we seeing with Ant that can only be solved by moving to a different build tool?

It is not meant to 'move' to maven build process.

The idea was to add the possibility to build / depoy with maven.

I also had my share of issues with maven…



At Radakan you were required to have all needed jars in the classpath to compile the project, the build script was to be generated by your IDE.



At one point we decided to switch to maven for the same reason mentioned here; so newbies would have an easier time setting up libraries and their jars without any headaches. So an experienced programmer set maven up, I put up all dependencies online. This is where the problems started; each IDE has it's own issues that make maven build just as difficult as an ant/IDE build. Source check out follows the same process as normal build, then it's getting maven and installing it, requiring the user to set up an environment variable in the process. Next is the command line call "mvn compile"… Depending on your IDE, you have to make different calls.



Eclipse: "mvn eclipse:eclipse" which makes the eclipse project files. And then some issues, first you have to set some variable so that the paths in the pom.xml file make sense, that was "M2_REPO" for us. Some users had issues with eclipse showing missing artifacts.



NetBeans: Version 6.0-6.1 would fail miserably because of missing dependencies. Version 6.5 works okay once it's maven plugin is installed. However the compile/run times are very long (5-20 secs) because it checks for dependency updates online. This makes it a complete chore to test/debug the project.



If you want to use the compiled maven dependencies in a regular project, here comes another headache, they are located in the remote path "C:Users<username>.m2repository" instead of where the maven project is actually located in.



So, certainly not the one-button-do-it kind of thing that we hoped for. Easier for newbies? Not really. Maven is intended for large scale project and for people experienced with it.