A Guide To Audio Visualization With JavaScript And GSAP (Part 1) — Smashing Magazine

Capturing Audio From The Browser

Before we can visualize audio, we need to get it into the browser. The starting point for this project was a working audio recorder built with XState, but the core functionality boils down to two browser APIs: navigator.mediaDevices and the MediaRecorder API.

navigator.mediaDevices provides access to connected media hardware. To see what audio inputs are available, we call enumerateDevices(), filter for devices with kind: 'audioinput', and store those options. If the user selects a non-default device, we keep that selection and use its deviceId when requesting the media stream.

getDevices: async () => {
  const devices = await navigator.mediaDevices.enumerateDevices();
  return devices.filter(({ kind }) => kind === "audioinput");
},

Once we know which device to use, we request a MediaStream. Passing audio: true to getUserMedia uses the default audio input. To target a specific device, include its deviceId in the constraints.

// deviceId is stored in state if we chose something other than default
// We got that list of devices from "enumerateDevices"
const audio = deviceId ? { deviceId: { exact: deviceId } } : true;
const stream = await navigator.mediaDevices.getUserMedia({ audio })
const recorder = new MediaRecorder(stream)

A MediaStream is the necessary input for constructing a MediaRecorder. The resulting instance exposes straightforward controls: start(), stop(), pause(), and resume(). The recorder's state property reports whether it is inactive, recording, or paused, which is useful for reflecting recording status in the interface.

Recording data is handled asynchronously. We initialize an empty Array for audio chunks, then attach an ondataavailable handler. This event fires each time the MediaStream yields data — typically when recording stops, although it can fire more frequently when used with a timeslice.

const chunks = []

recorder.ondataavailable = event => {
  chunks.push(event.data)
}

When recording ends, the collected chunks are assembled into a Blob. That blob is the playable audio file, which can be fed directly into an audio element for playback.

new Blob(chunks, { type: 'audio/mp3' })

The stripped-down demo below removes all framework and state-management code, showing that these few steps are everything needed to capture audio from a user's default microphone.

const TOGGLE = document.querySelector('#toggle')
const AUDIO = document.querySelector('audio')

let recorder
const RECORD = () => {
  const toggleRecording = async () => {
    if (!recorder) {
      // Reset the audio tag
      AUDIO.removeAttribute('src')
      const CHUNKS = []
      const MEDIA_STREAM = await window.navigator.mediaDevices.getUserMedia({
        audio: true
      })
      recorder = new MediaRecorder(MEDIA_STREAM)
      recorder.ondataavailable = event => {
        // Update the UI
        TOGGLE.innerText = 'Start Recording'
        recorder = null
        // Create the blob and show an audio element
        CHUNKS.push(event.data)
        const AUDIO_BLOB = new Blob(CHUNKS, {type: "audio/mp3"})
        AUDIO.setAttribute('src', window.URL.createObjectURL(AUDIO_BLOB))
      }
      TOGGLE.innerText = 'Stop Recording'
      recorder.start()
    } else {
      recorder.stop()
    }
  }
  toggleRecording()
}

TOGGLE.addEventListener('click', RECORD)

Why Side-Scrolling Bars Are The Right Call

The original requirement called for an audio visualization similar to those on Zencastr or Google Recorder: bars that scroll horizontally as audio is being recorded. From a technical standpoint, this style has a real advantage over other visualization forms — scrolling bars let you work with a continuously growing buffer of amplitude data without needing to repaint the entire canvas or recalculate the whole waveform on each frame. The visualization only has to advance the data one step at a time, which keeps the render loop lightweight even during long recording sessions.

In part two of this guide, we'll get into how to capture frequency or waveform data from the media stream and render that into the side-scrolling bars, along with the techniques for layering animation with GSAP.

From Raw Audio Data to Visual Feedback

Once the MediaRecorder is running, we have access to its live MediaStream. To turn that stream into something we can draw, we run it through the AudioContext API. The setup requires an AnalyserNode, which exposes the time and frequency data, and a MediaStreamAudioSourceNode, created via createMediaStreamSource, to feed the stream into the analyzer.

const STREAM = recorder.stream
const CONTEXT = new AudioContext() // Close it later
const ANALYSER = CONTEXT.createAnalyser() // Disconnect the analyser
const SOURCE = CONTEXT.createMediaStreamSource(STREAM) // disconnect the source

SOURCE.connect(ANALYSER)

The boilerplate connects the source node to the analyzer. From there, we poll the analyzer for data. Using window.requestAnimationFrame keeps the data fresh at roughly the display's refresh rate. The key method is getByteFrequencyData, which copies the frequency data into a Uint8Array whose length matches the analyzer's frequencyBinCount — that count is half the fftSize. The fftSize (default 2048, and must be a power of 2) determines the number of samples taken, leaving us around 1024 values for the visualization.

Note: The starting point demo uses getByteTimeDomainData for a waveform view, which returns time-domain data. For an equalizer-style volume visualization, getByteFrequencyData is the correct choice, returning decibel values per frequency.

const ANALYSE = stream => {
  // Create an AudioContext
  const CONTEXT = new AudioContext()
  // Create the Analyser
  const ANALYSER = CONTEXT.createAnalyser()
  // Connect a media stream source to connect to the analyser
  const SOURCE = CONTEXT.createMediaStreamSource(stream)
  // Create a Uint8Array based on the frequencyBinCount(fftSize / 2)
  const DATA_ARR = new Uint8Array(ANALYSER.frequencyBinCount)
  // Connect the analyser
  SOURCE.connect(ANALYSER)
  // REPORT is a function run on each animation frame until recording === false
  const REPORT = () => {
    // Copy the frequency data into DATA_ARR
    ANALYSER.getByteFrequencyData(DATA_ARR)
    // If we are still recording, run REPORT again in the next available frame
    if (recorder) requestAnimationFrame(REPORT)
    else {
      // Else, close the context and tear it down.
      CONTEXT.close()
    }
  }
  // Initiate reporting
  REPORT()
}

That code runs the analysis loop but does nothing visible yet. You can insert a console.info or debugger in the report callback to inspect the data flow.

See the Pen [3. Sampling Input Data](https://codepen.io/smashingmag/pen/PoOXoWp) by jh3y.

One issue: even after stopping the recorder, the browser tab keeps showing the recording indicator. The MediaRecorder stops, but the underlying MediaStream tracks are still live. We need to explicitly stop all tracks in the ondataavailable handler.

// Tear down after recording.
recorder.stream.getTracks().forEach(t => t.stop())
recorder = null

The first visualization is a simple volume reading. To get a normalized value, we divide the largest value in the data array by 255, since getByteFrequencyData returns values between 0 and 255.

const REPORT = () => {
  ANALYSER.getByteFrequencyData(DATA_ARR)
  const VOLUME = Math.floor((Math.max(...DATA_ARR) / 255) * 100)
  LABEL.innerText = `${VOLUME}%`
  if (recorder) requestAnimationFrame(REPORT)
  else {
    CONTEXT.close()
    LABEL.innerText = '0%'
  }
}

Congratulations — that is a working audio visualization built on a few lines of code on top of the boilerplate.

See the Pen [4. Processing Data](https://codepen.io/smashingmag/pen/LYOMYyY) by jh3y.

Animating Values with GSAP

To move beyond a plain readout, we bring in GSAP. Its value is not just in animating DOM elements — it excels at tweening numbers and offers utilities like ticker, a wrapper around requestAnimationFrame that runs in sync with the GSAP engine. If you're already using GSAP, swap your requestAnimationFrame calls for ticker.

In this next step, the label’s scale responds to volume, and a CSS custom property --hue shifts the color. On every tick, gsap.to animates both of those properties on the label element with a configured duration.

let recorder
let report
let audioContext

const CONFIG = {
  DURATION: 0.1,
}

const ANALYSE = stream => {
  audioContext = new AudioContext()
  const ANALYSER = audioContext.createAnalyser()
  const SOURCE = audioContext.createMediaStreamSource(stream)
  const DATA_ARR = new Uint8Array(ANALYSER.frequencyBinCount)
  SOURCE.connect(ANALYSER)
  report = () => {
    ANALYSER.getByteFrequencyData(DATA_ARR)
    const VOLUME = Math.floor((Math.max(...DATA_ARR) / 255) * 100)
    LABEL.innerText = `${VOLUME}%`
    gsap.to(LABEL, {
      scale: 1 + ((VOLUME * 2) / 100),
      '--hue': 100 - VOLUME,
      duration: CONFIG.DURATION,
    })
  }
  gsap.ticker.add(report)
}

gsap.to(LABEL, {
  scale: 1 + ((VOLUME * 2) / 100),
  '--hue': 100 - VOLUME,
  duration: CONFIG.DURATION,
})

The teardown logic moves into the recorder's else branch. When stopping the recorder, we animate the label back to its default state. In GASP's onComplete, we remove the report function from ticker and close the AudioContext.

gsap.ticker.add(REPORT) // Adds the reporting function for each frame
gsap.ticker.remove(REPORT) // Stops running REPORT on each frame
gsap.ticker.fps(24) // Would update our frames to run at 24fps (Cinematic)

const RECORD = () => {
  const toggleRecording = async () => {
    if (!recorder) {
      // Set up recording code...
    } else {
      recorder.stop()
      LABEL.innerText = '0%'
      gsap.to(LABEL, {
        duration: CONFIG.DURATION,
        scale: 1,
        hue: 100,
        onComplete: () => {
          gsap.ticker.remove(report)
          audioContext.close() 
        }
      })
    }
  }
  toggleRecording()
}

See the Pen [5. Getting “fancy” with GSAP](https://codepen.io/smashingmag/pen/yLPGLbq) by jh3y.

Rendering to Canvas

The EQ bars need HTML Canvas. If you are new to Canvas, the basics are straightforward: grab a rendering context and define a size. A canvas has two dimensions: its physical CSS size and its drawing buffer size. To draw, coordinates start at the top-left corner [0, 0].

<canvas></canvas>

See the Pen [6. Adjusting Physical and Canvas Sizing for Canvas](https://codepen.io/smashingmag/pen/QWOzWgm) by jh3y.

Canvas does not clear between frames. For moving elements, call clearRect at the start of each draw; skipping it can produce smearing effects that sometimes look intentional.

// Grab our canvas
const CANVAS = document.querySelector('canvas')
// Set the canvas size
CANVAS.width = 200
CANVAS.height = 200
// Grab the canvas context
const CONTEXT = CANVAS.getContext('2d')
// Clear the entire canvas with a rectangle of size "CANVAS.width" by "CANVAS.height"
// starting at (0, 0)
CONTEXT.clearRect(0, 0, CANVAS.width, CANVAS.height)
// Set fill color to "red"
CONTEXT.fillStyle = 'red'
// Fill rectangle at (80, 80) with width and height of 40
CONTEXT.fillRect(80, 80, 40, 40) 

Combine Canvas with GSAP by animating a plain object's values, then rendering the object in a separate function that runs each frame. Here, a square object holds size, hue, and scale.

const CANVAS = document.querySelector('canvas')
const CONTEXT = CANVAS.getContext('2d')
// Match canvas size to physical size
CANVAS.width = CANVAS.height = CANVAS.offsetHeight

const SQUARE = {
  hue: 100,
  scale: 1,
  size: 40,
}

Call the draw function once at startup so the canvas isn't empty:

drawSquare()

The report callback renders the square every frame. While recording, it maps the volume (between 0 and 1) to a target hue and scale using GSAP's mapRange utility.

report = () => {
  if (recorder) {
    ANALYSER.getByteFrequencyData(DATA_ARR)
    const VOLUME = Math.max(...DATA_ARR) / 255
    gsap.to(SQUARE, {
      duration: CONFIG.duration,
      hue: gsap.utils.mapRange(0, 1, 100, 0)(VOLUME),
      scale: gsap.utils.mapRange(0, 1, 1, 5)(VOLUME)
    })      
  }
  // render square
  drawSquare()
}

There are different ways to process the volume in the audio data. These examples use the largest value for simplicity. Alternatively, compute an average using reduce:

const VOLUME = Math.floor(((DATA_ARR.reduce((acc, a) => acc + a, 0) / DATA_ARR.length) / 255) * 100)

On recording finish, GSAP tweens the square's values back to the originals, and the teardown in onComplete removes the report function and closes the audio context. Because the drawing function simply reads the current object values, GSAP can alter them from anywhere in the code and the next render picks up the changes.

gsap.to(SQUARE, {
  duration: CONFIG.duration,
  scale: 1,
  hue: 100,
  onComplete: () => {
    audioContext.close() 
    gsap.ticker.remove(report)
  }
})

The first canvas visualization is done. Pushing the idea further, we can generate a random square for every frequency sample. A smaller fftSize keeps the count manageable, and each square gets randomized properties on each recording session.

See the Pen [9. Randomly generated audio visualization 🚀](https://codepen.io/smashingmag/pen/podqoOQ) by jh3y.

To change the visualization to circles or different colors, fork the demo and experiment.

Canvas Challenge
Recreate the random visualization using circles instead of squares, or with different color palettes. Fork the demos and play with the code.

EQ Bars: Timing Is Everything

The project's core requirement is EQ bars that travel from right to left. Each bar has an x position, centered on the y axis, with its height representing size. The starting position is the far right edge of the canvas.

Unlike the prior visualizations, this one adds a new bar each frame inside the ticker callback, and creates an animation for that bar's values. Pausing and resuming requires a timeline we can reference and control en masse, rather than standalone tweens.

// Array to hold our bars
const BARS = []
// Create a new bar
const NEW_BAR = {
  x: CANVAS.width,
  size: VOLUME, // Volume for that frame
}

Here is the drawing boilerplate and the reference variables:

// Keep reference to GSAP timeline
let timeline = gsap.timeline()
// Generate Array for BARS
const BARS = []
// Define a Bar width on the canvas
const BAR_WIDTH = 4
// We can declare a fill style outside of the loop.
// Let’s start with red!
DRAWING_CONTEXT.fillStyle = 'red'
// Update our drawing function to draw a bar at the correct "x" accounting for width
// Render bar vertically centered
const drawBar = ({ x, size }) => {
  const POINT_X = x - BAR_WIDTH / 2
  const POINT_Y = CANVAS.height / 2 - size / 2
  DRAWING_CONTEXT.fillRect(POINT_X, POINT_Y, BAR_WIDTH, size)  
}
// drawBars updated to iterate through new variables
const drawBars = () => {
  DRAWING_CONTEXT.clearRect(0, 0, CANVAS.width, CANVAS.height)
  for (const BAR of BARS) {
    drawBar(BAR)
  }
}

On stop, we can clear the timeline for reuse, depending on the desired behavior:

timeline.clear()

The updated reporting function looks like this:

REPORT = () => {
  if (recorder) {
    ANALYSER.getByteFrequencyData(DATA_ARR)
    const VOLUME = Math.floor((Math.max(...DATA_ARR) / 255) * 100)
    
    // At this point create a bar and have it added to the timeline
    const BAR = {
      x: CANVAS.width + BAR_WIDTH / 2,
      size: gsap.utils.mapRange(0, 100, 5, CANVAS.height * 0.8)(VOLUME)
    }
    // Add to bars Array       
    BARS.push(BAR)
    // Add the bar animation to the timeline
    timeline
      .to(BAR, {
        x: `-=${CANVAS.width + BAR_WIDTH}`,
        ease: 'none'
        duration: CONFIG.duration,
      })
  }
  if (recorder || visualizing) {
    drawBars()
  }
}

That runs, but the sequence is off — each bar animation waits for the previous one to finish because timeline animations default to sequential timing. The fix is explicit spacing based on the bar's entry index. The bar's speed must be relative to the canvas dimensions; otherwise, resizing distorts the animation.

Note: A canvas with responsive sizing can distort visuals on resize. Updating on resize is possible but complex, beyond this article's scope.

const BAR_WIDTH = 4
const PIXELS_PER_SECOND = 100
const VIZ_CONFIG = {
  bar: {
    width: 4,
    min_height: 0.04,
    max_height: 0.8
  },
  pixelsPerSecond: PIXELS_PER_SECOND,
  barDelay: (1 / PIXELS_PER_SECOND) * BAR_WIDTH,
}

We can insert each animation into the timeline at a calculated timestamp using the second parameter of the timeline's add method, derived from the bar index.

timeline
  .to(BAR,
    {
      x: `-=${CANVAS.width + VIZ_CONFIG.bar.width}`,
      ease: 'none',
      // Duration will be the same for all bars
      duration: CANVAS.width / VIZ_CONFIG.pixelsPerSecond,
    },
    // Time to insert the animation. Based on the new BARS length.
    BARS.length * VIZ_CONFIG.barDelay
  )

Even with corrected sequencing, the animation lags behind the audio input because we haven't accounted for the actual frame rate. Passing the calculated pixels per second to gsap.ticker.fps yields the desired timing. For a concrete example, with an FPS of 50, bar width of 4, and no gap, bars move at 200 pixels per second; animating across the canvas width takes that rate into account.

gsap.ticker.fps(DESIRED_FPS)

Picking an FPS below the user's screen refresh rate is safer than exceeding it:

Note: Choose an FPS you expect your users' hardware to handle. Some screens run at 30 frames per second; 24 FPS is the film standard.

See the Pen [13. Dialling the timing and gap](https://codepen.io/smashingmag/pen/Vwrqwqm) by jh3y.

The bars can also carry a gradient fill. Applying a linearGradient across the full canvas means a bar that is taller (from louder input) shows more color variation.

const fillStyle = DRAWING_CONTEXT.createLinearGradient(
  CANVAS.width / 2,
  0,
  CANVAS.width / 2,
  CANVAS.height
)
// Color stop is two colors
fillStyle.addColorStop(0.2, 'hsl(10, 80%, 50%)')
fillStyle.addColorStop(0.8, 'hsl(10, 80%, 50%)')
fillStyle.addColorStop(0.5, 'hsl(120, 80%, 50%)')

DRAWING_CONTEXT.fillStyle = fillStyle

Play with the width, gap, and speed parameters to match a real-time feel. Grouping bars and averaging their intensities is another avenue. At this point, you have the core toolset for interactive audio visualization, ready to extend with extra features.