lanmower:
What is your recommendation or intention for hooking MapSpinner into a real app? Critical methods in MapSpinner, Terrain, and GrassSpinner (at least) use final constants for some reason, and therefore do not take input params/setters and also defy class extension. Do you intend that any user of MapSpinner who wants to change grass density or game world scale, for example, should replace your provided classes with customized versions? I could provide numerous advantages to your project, but I won't be doing so if the project setup requires me to continually do manual merges to keep up-to-date with HEAD. Althought the Subversion setup is dramatically better now than it was in the same project about 6 months ago, the persisting lack of proper Subversion configuration is also a real impediment to integrating MapSpinner with a shared, platform-independent development project like mine.
I understand that you do what you have time to do, and also that most jME apps do not require tight configuration management and control like mine does. I am refactoring my terrain system now and I need to decide ASAP whether to branch MapSpinner yet again (in which case you will have no advantage from my ongoing work), or if the MapSpinner project is going to be properly encapsulated as a library to accommodate professional app integration. I'd appreciate any estimation you can make of future improvements in these areas.
To repeat, I'm not criticizing: If your short-term goals are more towards finishing the implementation than accommodating controlled integration, I respect that and will have to deal with it.
lanmower said:
...
It has already undergone some changes where thats concerned, a lot of the settings are getting wrapped into a properties class, wich should simplify things a lot, the persistence system is completely pluggable by the way, as long as you pass it a class implementing the persistence interface, as demonstrated by the rmi system
You are using a very generous definition of "completely pluggable". When one installs any good Java pluggable product (Spring, Tomcat, Hibernate, JBoss, Ant, Maven, Eclipse, JFreeChart, Jira,...), the user does not need to change the Java code of the product itself. Those cases where the user has to change the provided Java code itself are either to support purposes out-of-scope for the product, or admitted weaknesses. For the jME code base we have made continuous progress reducing the admitted weaknesses. Requiring the user to replace the most basic classes of the product (like MapSpinner.java and Terrain.java in this case) just in order to plug in another interface impl. works fine for demos but causes a change-control nightmare for real applications like dOminicB and I have.
Could I get an answer to my original question which is the ultimate purpose of this Topic: What is your recommendation or intention for hooking MapSpinner into a real app?. WRT parameterization, you admit a need for some improvement, so do you have any timeline? WRT your pluggability, are you satisfied with your current design in that respect and you will require users to replace your principle classes if they want to "plug"?
.... All the staticness isnt gone yet, and some of it will eventually stay, static variables dont break anything and in some cases they are beneficial, especially in cases where I have to bypass jme's limitations.
I hope when you say, static variables dont break anything, that you actually mean in some cases... though if so, that's rather obvious.
...
As a note, mapspinner is by no means monolithic if its the size thats being referred to, considering it has a 200k codebase including documentation. I'm still quite happy with its small size.
Agreed. Mapspinner's size is a great attraction to me. I meant monolithic in the holistic sense, not the quantitative sense. A quick look at Terrain.java, for example, shows tight binding of several features (i.e. lack of method encapsulation and pluggability).
As for submitting changes, I will look over any submissions that include 1 complete change, I am not willing to look through comlete branches that have hundreds of changes and try to reverse-engineer them to add them to mapspinner. And if the change is beneficial it will be either added or modified and added.
A very reasonable stance.
As for textures, a mapspinner terrain can have thousands of textures without a problem, I'm simply imposing the limit of 4-textures-per-single-block, with good reason wich to make terrain blocks more compressable and compatible, the actual texture management is typically left up to the artist/editor where he decides wich four textures may go on to a specific block...
But as things now stand, these changes require replacement of the most basic classes in your product. That is not what Java developers call "pluggable". As I have said, hacking up a copy of your code is fine for demos and such, but is not feasible for real applications because of the CM problems that causes.
As for data systems, mapspinner presents a landscape engine with data systems that are already implemented and by all means correct, if somebody went and already created a great data system for their game a year ago, but it is so unmanagable that he cant convert his terrain blocks into managable, streamable sizes with four-textures-per-block and pass the heightmaps and texture names of each block onto a persistence class for it to make a folder and save it, then I really cant help. Mapspinner is intended as a project mainly to be used for new projects
IMO these design goals make MapSpinner unsuitable for adoption into the jME code base. The posts to this Topic show, for example, that there are real jME users who would like to use MapSpinner but have needs beyond the fixed 4-textures-per-block. Is d0minicB incompetent because he has designed a game that has great performance and snazzy effects which require the use of 5 channels and the ability to control the textures' settings individually? We know that there are algorithms which MapSpinner could use to accommodate these use cases in a real pluggable way, because we have done it individually.
...
It should also be considered that rendered terrain doesnt change all that much in granularity, size or polycount, and I never intend to really have more than one mapspinner going at once...
More constraints which conflict with the goals of the jME project. The jME core developers don't use most of the features that they implement in their own games, but the goal is to accommodate other peoples' reasonable use cases. You apparently have the model of one game type in your mind, but there are tons of game designs where the game would have to manage multiple independent maps.
, so I dont see the need to make abslutely everything standard 'beans' as such....
Nobody here has advocated pure JavaBean conformance. dOminicB, in fact, has proposed a non-JavaBeans method for parameterization. What we are asking for is some flexibility.
Since it looks like Java code customizations will be necessary for any significant usage of mapspinner, would it at least be possible to get normal Subversion configurations applied so that we use Subversion to track our deltas, and in multi-platform environments?
Yeah, I agree with you blaine.
This and the fact that MapSpinner is too tied into map streaming/tiling is the only thing halting my plans of integrating MapSpinner as the primary terrain engine in jME3. It would be very nice if these things were taken care of before any new features are added.
Another related (and largely overlapping) thing which I had to do a lot of customization for was to defeat the zealous usage of statics. For good reasons, which I won't go into, I need to use multiple terrain map objects in normal Java OO fashion. This doesn't work when state is shared at the static class level.
Momoko_Fan said:
Yeah, I agree with you blaine.
This and the fact that MapSpinner is too tied into map streaming/tiling is the only thing halting my plans of integrating MapSpinner as the primary terrain engine in jME3. It would be very nice if these things were taken care of before any new features are added.
I have similar serious concerns about lack of modularity with the present and upcoming RPC systems. RPC isn't really a good fit for some client/server models where the server's primary responsibilty is state, session, and data control; and is at a design level which should not get tangled up with client-side scene/game-engine classes. A nice, pluggable interface system would be nice. That way one could unplug network sharing of specific item types, or could swap out RPC entirely and plug in an alternate or custom network sharing implementation. I see in the current baseline the largeley unsuccessful attempt to use the interface pattern for persistence. (The problems being primarily tight impl. bindings, circumvention of the interface system, and statics which are incompatible with the interface pattern).
It seems like many people already did the necessary changes to un-staticifiy MapSpinner. Maybe these changes can be contributed to the main codebase?
d0minicB said:
...What additional features were you considering implementing?
I don't know who you are asking. I'm not proposing any new features, but fixing existing integration problems and adding some modularity to the growing feature set which is making MapSpinner more and more monolithic. An example is the fixed texturing system. The texturing system that I added to MapSpinner is optional and is independent from the height-map data, and I can add as many textures as I want to. If the landscape-object-population-system is added in the same fashion as the texturing system, your game data systems will have to be designed (or re-designed!) to accommodate it if you want to use that feature.
Of course, it is lanmower's product and he has the right to make all design decisions. If it doesn't satisfy my needs, I'll have to do more work myself.
I agree with most of what you say, however I still believe that a design limitation of four textures per block is reasonable since the blocks are small. If you want riversand, grass, road, rock and goo on one single spot on a terrain then its reasonable that an efficient terrain system wont allow this.
I do agree to a certain extent about pluggabilaty, however I was referring to the data systems, wich is completely pluggable, dont like RMI? you can write your own persistence class. Simple.
IIRC jme doesnt really define an interface for its terrain either, and I also had to replace the entire terrain class to implement stitching and streaming.
Mapspinner is pluggable, and completely so. Winamp is pluggable, it doesnt mean you have a 'winamp' interface that replaces the binary, it means that core elements like saving, loading, processing and displaying of music can be modified. The same way in mapspinner you will be able to save, load, edit, and display terrain in a different manner by adding a file or two. (it still needs work on the shader class and the editor admittedly) That doesnt mean that the core class is replacable or interfaced.
My definition of completely pluggable is not that sparse if you consider how pluggable mapspinner is 'supposed' to be, there is a lot of work that needs to be done to make editing systems more abstract, since I can understand an 'editing' plugin, and the persistence system allows for 'data' plugins wich is understandable if someone wanted to implement a different networking system or the like, but I dont see the need for the support for a 'texturing' plugin in unless the plugin simply uses a different shader or the like. Plugins should add functionality, not change the way the core program works. I agree that the option for different texture sizes per block is cool, but there are general questions that one simply cant answer in that case 'do you save the texture sizes in each map block? how do you guarantee continuity then?' 'do you save the texture sizes per layer globally? what if two different textures are used on the terrain in that layer somewhere?' those two questions prevent me from making texture layers resizable per layer or per terrain block, and theres no other way of doing it, so the answer is using your base textures properly and making use of the provided noise examples to break repetition. Why cant mapspinner have 5 or 24 textures on one spot? its simple, a given point in any world is a combination of a single thing, in mapspinner you can have four things on a certain spot, wich is plenty, if you want 5 things on one spot you are out of luck since you have to change about 25 lines of code to get past that limitation, but mapspinner uses liitle data on the hard drive and very little ram, i'm not planning on breaking that functionality for a person that wants the luxury of improper texturing. If you can demonstrate how mapspinner could run 5 textures using under 400mb of ram with 34x34 loaded terrain blocks with someone elses texturing mechanism I suggest you submit me that change so I can test its speed on lower end systems, read through the code, fix any problems with it and submit the changes, but I will only do that if the new code doesnt break any of mapspinners new functionality, especially in the area of ram usage.
If you want to maintain changes across versions of mapspinner, use patch, make a patch between your version of mapspinner and the last head, apply the patch to the current head, going over any errors manually. Simple.
The mapspinner source is updated btw, many structural changes wrt parameters, and lots of refactoring done to start shaping it into something that can eventually be part of jme.
As for the main original question, try the new source, thats a great way to start, and work with mapspinner head instead of some hacked up old copy, apply your changes, maintain updates, submit anything decent that doesnt break current functionality. And make suggestions, I like this thread cause its the first big complaint about mapspinner and discussing these things gives me a clearer picture of mapspinners future.
If you have a design that can allow many textures without breaking mapspinners core functionality namely the memory usage (the new mapspinner uses only four color channels per terrain block that goes into ram and goes to the shader, thats the only data thats needed for one block) if you really want me to implement a way of multiple textures per terrain block, code me up a change that bitpacks splatting data into 4-bit values so that the 3 floats that hold color data can hold more textures, thats understandable and easy enough to do if its possible to bitpack in a shader. Then we can look at the differences in code and make a way for mapspinner to 'plug' both concepts.
Submissions in that manner are healthy to the project and will shed a lot of light on how mapspinner can become more pluggable, its much better for me to see other peoples code and say 'I dont want this in it as a default, but lets change the base that it supports either way'.
lanmower said:
...
Mapspinner is pluggable, and completely so. Winamp is pluggable, it doesnt mean you have a 'winamp' interface that replaces the binary, it means that core elements like saving, loading, processing and displaying of music can be modified. The same way in mapspinner you will be able to save, load, edit, and display terrain in a different manner by adding a file or two. (it still needs work on the shader class and the editor admittedly) That doesnt mean that the core class is replacable or interfaced.
You don't get it. Pluggable means that one does not have to change source code of the product in question in order to plug.
My definition of completely pluggable is not that sparse if you consider how pluggable mapspinner is 'supposed' to be, there is a lot of work that needs to be done to make editing systems more abstract, since I can understand an 'editing' plugin, and the persistence system allows for 'data' plugins wich is understandable if someone wanted to implement a different networking system or the like, but I dont see the need for the support for a 'texturing' plugin in unless the plugin simply uses a different shader or the like. Plugins should add functionality, not change the way the core program works. I agree that the option for different texture sizes per block is cool, but there are general questions that one simply cant answer in that case 'do you save the texture sizes in each map block? how do you guarantee continuity then?' 'do you save the texture sizes per layer globally? what if two different textures are used on the terrain in that layer somewhere?' those two questions prevent me from making texture layers resizable per layer or per terrain block, and theres no other way of doing it, so the answer is using your base textures properly and making use of the provided noise examples to break repetition.
You speak of these constraints as if they are intrinsic to the problem domain. They are just limitations of your design. I'm not going to debate about it because I have overcome all of these obstacles by implementing a more flexible texturing design, and I see that other people have also. I had my texture-splatting system working with mapspinner before you had any texture functionality. Your argument is analogous to the fallacious Intelligent Design argument which says that they can conceive of no mechanism to account, and therefore no such mechanism is possible.
I think your basic traversal updating mechanism is great. I have added some improvements and get great net performance. But it does not follow from this that nobody else (including dOminicB) is capable of designing a texturing system which is more flexible and powerful than what you designed and which eliminates constraints imposed by your design.
...
If you want to maintain changes across versions of mapspinner, use patch, make a patch between your version of mapspinner and the last head, apply the patch to the current head, going over any errors manually. Simple.
I know how to CM. I design shared dev systems professionally for international development teams. If you look at any respected Java product, you will find that users do not have to manage change sets and continually update and do merges as the product changes-- a typical java application that uses 6 third party products certainly does not manage 6 change sets-- these products are expected to be self-consistent and the public interfaces (in the general sense) are "used" as-is, not "modified". As all of the functionality is intermixed in your principle classes, frequent adjustments and maintenance will be necessary to customization code and change sets. This substandard procedure is made even more difficult because your Subversion configuration is not set up to allow Subversion clients to track local updates how the clients are designed to; and whereas jME is a Java-platform product, you have not made the Subversion configs to share your files in a platform-independent manner.
The most glaring problem, though, remains that any real usage of mapspinner requires source code changes to the principle classes of your product.
blaine said:Time to share some code then maybe? From what I've read, it seems this is the kind of thing lanmower would love to see explained in code, more so than just words. I think code would do this debate a lot of good just about now :)
(...)I have overcome all of these obstacles by implementing a more flexible texturing design, and I see that other people have also. I had my texture-splatting system working with mapspinner before you had any texture functionality.
blaine said:I also wholly agree with this principle. As a novice user of technology, only knowing how a lot of things things work in principle but without the knowledge to make it work if it's broken, I feel much more at ease when I know that all my plugin is doing is adding some functionality in a sort of confined space. Just one less reason for me to cry for support when something isn't working, because I should be able to quite easily just take it out again if it didn't work as I expected it to.
Pluggable means that one does not have to change source code of the product in question in order to plug.
erlend_sh said:
blaine said:Time to share some code then maybe? From what I've read, it seems this is the kind of thing lanmower would love to see explained in code, more so than just words. I think code would do this debate a lot of good just about now :)
(...)I have overcome all of these obstacles by implementing a more flexible texturing design, and I see that other people have also. I had my texture-splatting system working with mapspinner before you had any texture functionality.
That's a great idea in general, but (a) lanmower owns the product and doesn't share our appreciation for normal pluggability (really of the benefits of encapsulation in general); and (b) my modifications were purely customizations not intended to be portable/sharable (beyond my proprietary products and my own dev team), and I am hoping that d0minicB or somebody else has a design more compatible with the current mapspinner HEAD.
The reason why my mods are so out of sync with mapspinner trunk and non-sharable is that when I did my work it looked like mapspinner was completely dead. The last forum post at that time said that lanmower would push a code update "later in the day" or similar, but no such code push was made, and the author could not be reached for months. As I had no privileges to contribute to the project and it took considerable effort to understand the (at that time) undocumented code which contained many bugs, there seemed to be no point to maintaining compatibility with a dead-ended product. (I am not complaining about the bugs as the product was clearly early alpha at that point).
But if it's still helpful and wanted, I can share the relevant code.
blaine said:I also wholly agree with this principle. As a novice user of technology, only knowing how a lot of things things work in principle but without the knowledge to make it work if it's broken, I feel much more at ease when I know that all my plugin is doing is adding some functionality in a sort of confined space. Just one less reason for me to cry for support when something isn't working, because I should be able to quite easily just take it out again if it didn't work as I expected it to.
Pluggable means that one does not have to change source code of the product in question in order to plug.
It's probable that all of us jME core folk are in agreement that unless there is some specific reason for an exception, jME products and sub-products should be usable as Java libraries. Java developers should be able to work from a binary distribution and the API spec. Once kinks are worked out, jME users should not have to go looking through .java files unless they are implementing some out-of-scope feature. In this case, terrain texturing is intended to be a supported feature of mapspinner.
Your final point about isolated local modifications is further justification for my calls for proper Subversion configs. One should be able to use their Subversion client features (IDE-embedded or not) to clearly distinguish real local code changes without clutter from env-specific and derived files. It would also be nice if a build file or similar were provided to jump-start larger projects which need to do non-IDE builds from source (which practice is the standard for open source Java projects and is much more important when basic product usage requires recompilation).
my two cents…
i've looked a lot at mapspinner and i like what i see…
the pluggable editor part of it is very impressive…
and the terrain management is excellent, which to be honest goes without saying…
i agree with blaine with regards to customisation of the number of textures to be used etc…
i understand the design requirement to be focused on performance and memory footprint however if it is to be part of the engine then it needs to be flexible in it's design to allow for customisation, the likes of which blaine mentioned…
every developer who gets their hands on mapspinner will want to do things with it that you can't foresee…
even from scene worker's perspective i would imagine a user wanting to specify how many texture layers they wish to use, map size, tile size etc etc when they go to create a map spinner terrain…
however i would have no problem with adding map spinner as is with the cap on the number of textures and hard coded terrain properties…it would just mean users would have to work within the confines of the design of the code…
so basically what i would suggest is that either mapspinner becomes a self contained project and keeps it's design specifics intact or map spinner is refactored with the aim of becoming part of the jme engine itself (and to honest from what i've seen of the code this would not be a big job as the code itself is becoming very modular)…
either way it's a great project and i hope it continues to impress as much as it has done in the past…
It seems I need to quote myself.
if you really want me to implement a way of multiple textures per terrain block, code me up a change that bitpacks splatting data into 4-bit values so that the 3 floats that hold color data can hold more textures, thats understandable and easy enough to do if its possible to bitpack in a shader. Then we can look at the differences in code and make a way for mapspinner to 'plug' both concepts.
Go at it guys, if you want it, make it, I have bigger fish to fry. :)
ncomp said:
however i would have no problem with adding map spinner as is with the cap on the number of textures and hard coded terrain properties....it would just mean users would have to work within the confines of the design of the code...
erm, have you noticed the new 'non hard coded' parameters? ;)
i downloaded the latest map spinner from svn twice this week but it has compile errors…
hence i didn't look at the code as i too have bigger fish to fry 
apologies if i got it wrong…
ncomp said:
i downloaded the latest map spinner from svn twice this week but it has compile errors...
...
Same results here.
blaine said:I think it's essential to this debate that lanmower gets to see this code. Whenever I read over these conversations I can't help but feel like a lot of points are left out, while others are blown out of proportion.
I can relax the proprietay license and send you my working class (which properly encapsulates all texture-related work), but I am very soon going to either refactor or completely replace it (perhaps with d0minicB's design). I very much doubt you would undertake the work to make this class run with mapspinner HEAD (and it wouldn't make much sense to assume that task), but you can see the algorithms; or you can wait for the new version.
"I did not have time to write a short letter, so I wrote a long one."
~ Mark Twain
@Blaine: Nevermind the refactored version. I assume what you got now is what you've been referring to in a large portion of your comments, making it the missing link. I suggest you contextually comment your code and pass it over immediately.
@lanmower: You clearly have a detailed idea of what Mapspinner should and shouldn't be. Let's see it! If there was a fairly detailed doc available, I believe this debate would be more brief and to the point.
erlend_sh said:
@Blaine: Nevermind the refactored version. I assume what you got now is what you've been referring to in a large portion of your comments, making it the missing link.
That conclusion does not follow. I've been referring to that only because it is living code that proves flexible texturing is possible with good performance and with good encapsulation. I have no reason to think my implementation is better than d0minicB's. I'm not a masochist: I would not be preparing to re-write it if I thought it were suitable for merging with mapspinner HEAD-- I very much wish it were! The compatibility issues are not simple ones. As one example, the paintable image system that I made is probably totally incompatible with the subsequent work lanmower did to accommodate dedicated editors. Some of the texture use cases which I satisfied with this implementation are dependent upon my specific game. Many jME people like the world-modeling-simplification tactic of deriving target texture based on terrain height at a specific point. I implemented the more challenging use case of merging images painted for the purpose. If I intended to make a jME product, I would have certainly satisfied the simpler height-based-texturing use case too.
More importantly though, the current sub-topic is just about one feature. Though it has been said here that reliability and performance of the core feature set are the highest priorities, it looks like features such as dedicated editor support and landscape object population will continue to be added while the encapsulation, flexibility and CM problems will remain. Please correct me if I'm wrong, but I detect no desire to address these issues in the short term. Judging by this post , mapspinner is nearly ready to graduate from beta status.
I suggest you contextually comment your code and pass it over immediately.
I'll email you and lanmower the class as it is, untouched since May. It's already well-documented for the Java developer audience.
UPDATE: Source code sent via PM.
… Sorry about the source not working, I have been offline for about a week building a movie studio for a guy. Now I have to work in the studio for him so I'm also very caught up, I hope that my lifestyle will relax a little in the next few days (definately not today) to allow me to review some more things. Later today I will completely delete the code available on google code and replace it with mapspinners new source (its the only thing that seems to work when any major refactoring happens.)
The problem is likely to be connected to the new source having terrain be the manager class, terrainnode being the class that holds a terrain block and grass being the new name for grasspinner.
I think the new code is going in the right direction taking peoples advice (especially blaine and ncomps) into account, however there is yet more to be done so expect some major changes in the way texturing works soon (I suspect a good way to go about it is to take any texture-specific things out of terrainNode itself and seperate it)
Many jME people like the world-modeling-simplification tactic of deriving target texture based on terrain height at a specific point. I implemented the more challenging use case of merging images painted for the purpose. If I intended to make a jME product, I would have certainly satisfied the simpler height-based-texturing use case too.
I was thinking about making the system able to paint that mechanism on where its needed, how does that sound to you guys? also painting normal based splatting is cool since angled terrain would normally be rock and flat ones grass, while the math is already inside mapspinner, I need to go hack up the terrain editor to support these painting mechasims, however my long term plan was to make the editor itself a little more open before I do that since it is an absolute mess in my books :)
The plans to beta mapspinner is reliant on what everyone requires in the short term to be done, once all or most of the basic requests have been fulfilled I will beta mapspinner and let it ferment for a while only fixing errors while I work on maloader and other map building methods, that is to me the correct way to go about it since mapspinner v2 will be something that allows buildings on your terrain that you can walk into and have detail inside. v2 will also likely be jme3 based cause I would like to keep any major developments on a 'cutting edge' evel rather than make something that resembles a tumor. ;)