Getting frequency values in audiodata

Hi there!



I ask myself if anyone knows a way to get specific information about the peaks of each of the (or parts of the) frequencies in a current playback.



I hope you can understand me…  :roll:



What I want to achieve in my game is the following:

At the end there shall be a terrain which functions like a big, threedimensional visualizer of the music that is played. So, let's say the deep frequencies are in the north of the terrain, the high ones in the south. Every time a bassdrum is kicked f.e. there errupts a big mountain in the north. :slight_smile:



So what I need to do is to grab the data I need. But I have no clue how to do that… :?

I don't really know how to get that info, but I know it's possible to use Fourier transform to separate complex signals into multiple harmonic sines and cosines that make up that signal. Then it's possible to teach a neural network f.e. which note is played.



But as I see you need just frequencies. I would start off by making a program that makes a graph of audio signal. Then I would figure out how to get the frequencies from that data. I think getting the bass drum would be the hardest part and would require maths and neural networks.



Anyways if you make some progress, please let me know also :slight_smile:



One more thing. I once asked from this forum about something similar and blue_week replied:



i used http://code.compartmental.net/tools/minim/ to make a guitar tuner once…

it's designed for processing, but can just as easily be used with jME.

Hey many thanks!



I didn't thought of an FFT, but that's the solution, of course!



Thanks for the link, also. That Minim is a library I could use in many different ways (not just for JME but for several java-projects). The nicest part is that it comes along with an FFT and a manual which includes getting the values of frequency bands! :smiley:



http://code.compartmental.net/tools/minim/manual-fft/



Thanks, man!

Okay, I’ll use this to showcase some code and an early screenshot, okay? Still poor graphics, but:











The code was pretty simple. Just generate a TerrainBlock (I used a modified version, because my slow PC couldn’t handle the VBOInfo with more than 10fps) and go over the heightMap.



In the update-Method call something like this:


private void processMusic() {
      boolean beatHit = false;
      if (beat.isKick() || beat.isSnare())
         beatHit = true;
      if (beatHit) {
         beatCounter = 0;
         sData = mPlayer.getFreqBands();
         workingHeightMap = sData.getBands();
         for (int i = 0; i < 484;/*sData.getSpecSize();*/ i++) {
            workingHeightMap[i] = FastMath.clamp(workingHeightMap[i], 0, 2.55f);
            workingHeightMap[i] = workingHeightMap[i] / 12;
         }
            
      }
      
      float beatTime = 100;
      if (heightMap != null && workingHeightMap != null && beatCounter < beatTime) {
         for (int i = 0; i < heightMap.length; i++) {
            

            float[] theBigger;
            float[] theSmaller;

            if (heightMap[i] >= workingHeightMap[i]) {
               theBigger = heightMap;
               theSmaller = workingHeightMap;
            } else {
               theBigger = workingHeightMap;
               theSmaller = heightMap;
            }
            
            float distance = theBigger[i] - theSmaller[i];
            float perc = 100 * (beatCounter / beatTime);            
            float newVal = distance * (perc / 100) - theSmaller[i];

            if (heightMap[i] < workingHeightMap[i])
               heightMap[i] += newVal;
            else
               heightMap[i] -= newVal;

         }
         
                        //go over the heightMap to make a square be zero where the chessfield is
         for (int x = 8; x < 14; x++)
            for (int y = 12; y < 19; y++)
            heightMap[x + (y * 22)] = 0;
         
         
         tb.setHeightMap(heightMap);
         tb.update();
      } else if (beatCounter >= beatTime)
         beatCounter = 0;
      
      beatCounter++;
   }



The tb.update method is the same as the updateFromHeightMap method in the original TerrainBlock class. As you see, did I link the whole thing to Minim's BeatDetector, which allows pretty cool results.

sData is of type StreamData which simply is a Dataholder class for storaging the fft-Data, the number of bands which are used (which is the totalSize of the heightMap) and the mp3's MetaData.

The getFrequency method looks like this:

public StreamData getFreqBands() {
      if (player.isPlaying()) {
         fft.forward(player.mix);
         float[] bands = new float[fft.specSize()];
            
         for(int i = 0; i < fft.specSize(); i++)
         {
            float freq = fft.getBand(i);
            bands[i] = freq;
         }         
         
         return new StreamData(fft.specSize(), bands, player.getMetaData());         
      }


wow, cool