Terrain Generation with Music

Hi all, I am currently making a side-scrolling-shooting game which involves the player choosing a music and the music will in turn generate the terrain shape. Recently however, I have ran into some problems.



I followed some of the tutorials’ advice to test out terrain building via flagrush tutorial and HelloTerrain. I created a very crude run of the game and regarding the terrain, I used the flagrush tutorial’s MidHeightMap option which randomly generate terrain shapes according to some parameters.



I am not particularly sure how I should approach the generation of Terrain. I have tried both real-time generation and pre-generation using the MidHeightMap and I personally do not think the result is satifactory. Is there a suggested method, in which given that attributes of a piece of music can be obtained (amplitudes etc), that could possibly do what I wanted to do?



And some errors that I ran into. I was building terrain in a GameState called TerrainState but when I run this:

TerrainBlock tb = new TerrainBlock("Terrain", heightMap.getSize(), terrainScale,
            heightMap.getHeightMap(),
            new Vector3f(0, 0, 0));



I got the following error:

Exception in thread "main" java.lang.NoSuchMethodError: com.jmex.terrain.util.MidPointHeightMap.getHeightMap()[F



This error never came up before when I was using the same method on SimpleGame or BaseGame and so I tested this with HelloTerrain again and I got the following errors after the game starts:


java.lang.NoSuchMethodError: com.jmex.terrain.TerrainBlock.<init>(Ljava/lang/String;ILcom/jme/math/Vector3f;[FLcom/jme/math/Vector3f;)V
   at TerrainTest.HelloTerrain.homeGrownHeightMap(HelloTerrain.java:48)
   at TerrainTest.HelloTerrain.simpleInitGame(HelloTerrain.java:27)
   at com.jme.app.BaseSimpleGame.initGame(Unknown Source)
   at com.jme.app.BaseGame.start(Unknown Source)
   at TerrainTest.HelloTerrain.main(HelloTerrain.java:22)




Thanks a lot and please bear with me for I am not particularly skilled in programming.

You have a very cool idea. Darkfrog has made the terrain from file in flagrushnetworking example.



   private void buildTerrain() {
      //for random terrain
      //MidPointHeightMap heightMap = new MidPointHeightMap(64, 1f);
     
      int[] heightMap = null;
      ImageIcon imageIcon = null;
      try {
         ObjectInputStream ois = new ObjectInputStream(getClass().getClassLoader().getResourceAsStream("resource/heightmap.data"));
         heightMap = (int[])ois.readObject();
         imageIcon = (ImageIcon)ois.readObject();
         ois.close();
      } catch(Exception exc) {
         throw new RuntimeException(exc);
      }
     
      float[] heightMapf = new float[heightMap.length];
     
      for(int i=0; i<heightMap.length; i++){
         heightMapf[i] = Integer.valueOf(heightMap[i]).floatValue();
      }
     
      // Scale the data
      Vector3f terrainScale = new Vector3f(4, 0.1f, 4);
      // create a terrainblock
      terrainBlock = new TerrainBlock("Terrain", 64, terrainScale,
              heightMapf, new Vector3f(0, 0, 0));
         
      terrainBlock.setModelBound(new BoundingBox());
      terrainBlock.updateModelBound();



Maybe this might give you some insipiration.
Henri said:

You have a very cool idea. Darkfrog has made the terrain from file in flagrushnetworking example.


Thank you very much. I just thought it would be nice to have a DDR game that instead of the player hitting arrows at the right time, enables the player to "see" what the music's effects to the game would be. So other than the terrain generation, I am also looking to create "enemies", effects etc according to the music.


Maybe this might give you some insipiration.


This looks great! I will give it a shot and see if I can generate anything useful. Thank you very much for the heads up  :)

Seems to me that making the terrain from a pre-created file is exactly the opposite of what you want. If I understand correctly, you want real-time generated terrain based on the rhythm, volume, and pitch of a piece of music. Correct?

Trussell said:

Seems to me that making the terrain from a pre-created file is exactly the opposite of what you want. If I understand correctly, you want real-time generated terrain based on the rhythm, volume, and pitch of a piece of music. Correct?


This is indeed the final stage that I wish to be able to achieve. I have tried doing that by fiddling with the rendering loop, trying to rebuild terrain there but it didn't seem to work.

Pre-generating the terrain would be good enough for testing purposes at the moment, though.

well, i guess what you are looking for is altering the terrain in realtime, according to the music.

So i doesn't matter how you create the terrain at the beginning, it might be even flat.



What is interessting is WHAT you want to change, and based on which logic?!

(which areas are influenced by which frequencies / beats per minute / amplitude or whatever - your choice)



HOW you can alter the terrain during the runtime you can see e.g. in Monkeyworld3D.

Basically its just altering float values in a Float[].



And as you said it's a side scrolling shooter i guess the player is not able to move into depth.



So my first thought was:



  • create the terrain.

  • then use the playerposition as reference point (very foreground, centered horizontally)

  • depending on this alter the terrain in X or Z direction (or both or whatever)

  • OR let the music flow through the terrain from one side to the other

  • or or or



and before i'm typing too long threads.. i hacked a simple test (JME2).

check it out, it should give you a pretty good overview how to handle the terrainalternation and what to do during the update.
The more problematic step is to get the frequencies / amplitude out of the musicfile.. i guess there is a usefull api out there.

anyway - hope this helps.

package jmetest.mytests;

import com.jme.app.AbstractGame;
import com.jme.app.SimplePassGame;
import com.jme.input.KeyBindingManager;
import com.jme.input.KeyInput;
import com.jme.math.FastMath;
import com.jme.math.Vector3f;
import com.jme.renderer.ColorRGBA;
import com.jme.renderer.pass.RenderPass;
import com.jme.scene.shape.Sphere;
import com.jme.scene.state.MaterialState;
import com.jme.scene.state.WireframeState;
import com.jmex.terrain.TerrainBlock;

public class TestHeightmapAltering extends SimplePassGame {

   private TerrainBlock m_terrainBlock;
   private Sphere m_player;
   private float[] m_heights;
   private TerrainBlock m_terrainBlockWireframe;
   private RenderPass m_renderPass;
   private RenderPass m_wireFramePass;
   private static float m_playersZpos = 2.5f;

   @Override
   protected void simpleInitGame() {

      m_renderPass = new RenderPass();
      m_wireFramePass = new RenderPass();

      setupCamera();
      setupInputs();

      setupTerrain();
      setupPlayer();

   }

   private void setupPlayer() {
      m_player = new Sphere("playerSphere", 15, 15, 0.6f);
      MaterialState ms = display.getRenderer().createMaterialState();
      ms.setDiffuse(ColorRGBA.red);
      m_player.setRenderState(ms);
      m_player.setLocalTranslation(0.2f, 0, m_playersZpos);
      rootNode.attachChild(m_player);
      m_renderPass.add(m_player);
   }

   private void setupCamera() {

      // just set up a static camera facing +Z
      cam.setLocation(new Vector3f(0, 3f, -5));
      cam.setDirection(Vector3f.UNIT_Z);
   }

   private void setupInputs() {
      // remove all handlers so that the cam stays static
      input.removeAllFromAttachedHandlers();

      // add some commands
      KeyBindingManager.getKeyBindingManager().add("moveLeft",
            KeyInput.KEY_LEFT);
      KeyBindingManager.getKeyBindingManager().add("moveRight",
            KeyInput.KEY_RIGHT);
      KeyBindingManager.getKeyBindingManager().add("recalculateHeights",
            KeyInput.KEY_SPACE);
   }

   private void setupTerrain() {

      // create a heightmap with 24x24 values
      m_heights = new float[576];
      for (int i = 0; i < 576; i++) {
         m_heights[i] = 0f;
      }

      // create a terrainBlock based on the heighmap with the size 24x24 and a
      // stepsize of 1
      m_terrainBlock = new TerrainBlock("terrainBlock", 24,
            Vector3f.UNIT_XYZ, m_heights, Vector3f.ZERO);
      // translate it so that it's centered and in front of the camera
      m_terrainBlock.setLocalTranslation(
            -m_terrainBlock.getSize() / 2f * 0.3f, 0, m_terrainBlock
                  .getSize() * 0.4f * 0.3f);

      // attach the terrainblock to the rootnode and to the renderpass, which
      // is added to the passmanager afterwards
      rootNode.attachChild(m_terrainBlock);
      m_renderPass.add(m_terrainBlock);
      pManager.add(m_renderPass);

      // *OPTIONAL*
      // create another TerrainBlock for getting a wireframe overlay
      // was too lazy to set up textures or whatever.
      m_terrainBlockWireframe = new TerrainBlock("terrainBlock", 24,
            Vector3f.UNIT_XYZ, m_heights, Vector3f.ZERO);
      m_terrainBlockWireframe.setLocalTranslation(
            -m_terrainBlock.getSize() / 2f * 0.3f, 0, m_terrainBlock
                  .getSize() * 0.4f * 0.3f);

      // create a WireframePass and add it to the 2nd terrainblock
      WireframeState wireframeState = display.getRenderer()
            .createWireframeState();
      wireframeState.setLineWidth(0.3f);
      wireframeState.setEnabled(true);
      m_terrainBlockWireframe.setRenderState(wireframeState);
      // add the 2nd terrainblock to the other RenderPass, and add the pass to
      // the passmanager
      m_wireFramePass.add(m_terrainBlockWireframe);
      pManager.add(m_wireFramePass);

      m_terrainBlockWireframe.updateRenderState();

   }

   @Override
   protected void simpleUpdate() {
      super.simpleUpdate();

      // System.out.println("height at playerPos: "
      // + m_terrainBlock.getHeight(m_player.getLocalTranslation().x,
      // m_playersZpos));

      if (KeyBindingManager.getKeyBindingManager().isValidCommand("moveLeft",
            true)) {

         // check that the player stays on the terrainblock
         if (!Float.isNaN(m_terrainBlock.getHeight(m_player
               .getLocalTranslation().x - 0.03f, m_playersZpos))) {

            // move the player in X and update it's Y pos
            m_player.getLocalTranslation().setX(
                  m_player.getLocalTranslation().x - 0.03f);
            m_player.getLocalTranslation().setY(
                  m_terrainBlock
                        .getHeight(m_player.getLocalTranslation().x,
                              m_playersZpos));
            m_player.updateGeometricState(tpf, true);

            // update the cam
            cam.getLocation().setX(cam.getLocation().x - 0.03f);
            cam.update();
         }
      } else if (KeyBindingManager.getKeyBindingManager().isValidCommand(
            "moveRight", true)) {

         // check that the player stays on the terrainblock
         if (!Float.isNaN(m_terrainBlock.getHeight(m_player
               .getLocalTranslation().x + 0.03f, m_playersZpos))) {

            // move the player in X and update it's Y pos
            m_player.getLocalTranslation().setX(
                  m_player.getLocalTranslation().x + 0.03f);
            m_player.getLocalTranslation().setY(
                  m_terrainBlock
                        .getHeight(m_player.getLocalTranslation().x,
                              m_playersZpos));
            m_player.updateGeometricState(tpf, true);

            // update the cam
            cam.getLocation().setX(cam.getLocation().x + 0.03f);
            cam.update();
         }
      }

      if (KeyBindingManager.getKeyBindingManager().isValidCommand(
            "recalculateHeights", true)) {

         // slope is used to raise the terrain the more it gets into depth
         float slope = 1;

         // recalculate the coulum of the terrainblock the player is
         // currently on..
         // therefore iterate over m_heights in steps of 24 (m_terrainBlock
         // .getSize())
         for (int i = (int) m_player.getLocalTranslation().x; i < 24; i += m_terrainBlock
               .getSize()) {

            // on this column, recalculate the rows randomly + the
            // increasing slope value
            // and update both terrainblocks

            // slope is what you might alter by music for example by
            // amplitude or frequency.
            // you may use the coulums instead of the rows by
            // m_heights[i*24+j]
            for (int j = 0; j < 24; j++) {
               m_heights[i + j * 24] = FastMath.nextRandomFloat() * 0.6f
                     + slope;
               m_terrainBlock.updateFromHeightMap();
               m_terrainBlockWireframe.updateFromHeightMap();
               slope *= 1.053;
            }
            //reset slope for a new coulum
            slope = 1;
         }

         m_player.getLocalTranslation().setY(
               m_terrainBlock.getHeight(m_player.getLocalTranslation().x,
                     m_playersZpos));
      }
      
      //update the passmanager
      pManager.updatePasses(tpf);
   }

   public static void main(String[] args) {
      TestHeightmapAltering app = new TestHeightmapAltering();
      app.setConfigShowMode(AbstractGame.ConfigShowMode.ShowIfNoConfig);
      app.start();
   }

};



*use the LEFT and RIGHT button for movement, and hold SPACE to alter the terrain.*



.:emp...imm0|82:. said:

well, i guess what you are looking for is altering the terrain in realtime, according to the music.
So i doesn't matter how you create the terrain at the beginning, it might be even flat.


I agree with this totally and have discovered this practically during some earlier testing. Though I must say I feel that generating the terrain in real time and pre-generation are 2 different concepts.


What is interessting is WHAT you want to change, and based on which logic?!
(which areas are influenced by which frequencies / beats per minute / amplitude or whatever - your choice)


My original thoughts are. Amplitude decides the height of the terrain, BPM decides how quickly the screen is scrolling and frequencies decides the enemy generation rates.


HOW you can alter the terrain during the runtime you can see e.g. in Monkeyworld3D.
Basically its just altering float values in a Float[].


And I think you have given a great demonstration to this!


The more problematic step is to get the frequencies / amplitude out of the musicfile.. i guess there is a usefull api out there.


This is where I am mostly stuck on as well. I am currently trying to use an AudioInputStream to turn wav files into ByteArrays then somehow trying to turn these into float[] and then somehow do FFT on these to obtain the amplitudes etc. I have been searching for a good api but haven't been able to locate any straightforward ones.


anyway - hope this helps.


This is wonderful and it has given a lot of inspiration already by just trying it out. Thank you so much! XD

Why bother with terrain pages in a 2d game? It's a side-scrolling game, right?

Trussell said:

Why bother with terrain pages in a 2d game? It's a side-scrolling game, right?


If I don't use Terrain as some form of "base-at-the-bottom", I honestly don't know what I could use. And yes, it is a side-scrolling game.

I think you'd be more suited to make your own class for this, but I could be wrong.

Well, being back after a RockFestivalWeekend and some sickness afterwards, i'm glad the test helped you along.



And concerning the TerrainPage and 2D-Game question, i'd say it's not that bad.

Because as i understand it, the scene definately should be 2.5 D, which means you definitely would need some "depth" information.



So i guess the approach is ok.

Especially if you need a quick solution because you have to figure out the music-parsing-stuff more urgendly.