Using WebXR With Babylon.js — Smashing Magazine

WebXR and Babylon.js: Building Immersive Experiences for the Browser

Mixed reality (XR) — the umbrella term covering both virtual reality (VR) and augmented reality (AR) — is moving beyond the gaming niche. For web developers, the appeal is obvious: the ability to build three-dimensional experiences without learning a new language or abandoning the browser as a delivery platform. The hardware barriers that once limited XR adoption have eased, and the software stack has matured to the point where a reasonable grasp of vector geometry and matrix math is enough to get started.

WebXR is the standard that makes this possible. It is the collection of specifications that support rendering three-dimensional scenes in VR and AR environments. VR presents a fully immersive, device-generated world, while AR overlays graphical elements onto the real world through a camera or transparent display. Supporting devices range from high-end headsets with motion tracking — Vive, Oculus, Hololens — to AR-enabled smartphones that composite graphics onto their camera feeds.

The WebXR Device API and Browser Support

The WebXR Device API is the primary interface for developers to interact with immersive hardware. It provides capabilities for discovering compatible devices, rendering scenes at the correct frame rate, mirroring output to a 2D display, and tracking input controls. Still a working draft, the specification merges the older WebVR API, which was VR-only, with the experimental WebXR Augmented Reality Module. WebVR is now superseded, and frameworks typically offer migration paths to WebXR.

Browser adoption is growing but uneven. Full support exists in Chrome 79, Edge 79, Chrome for Android 79, and Samsung Internet 11.2. For Firefox, Internet Explorer, Opera, Safari, and various mobile browsers, a community-maintained WebXR Polyfill implements the API in JavaScript. Firefox users can also enable experimental support by visiting about:config and setting dom.vr.webxr.enabled to true. For development and debugging on a desktop, the WebXR API Emulator for Chrome and Firefox provide additional tooling.

The WebXR Device API relies on WebGL for rendering, lighting, and texturing. Developers familiar with WebGL will find much of the underlying work familiar. For those building on top of it, several JavaScript frameworks abstract the complexity, including Three.js and Babylon.js. A-Frame, a markup-driven approach, is built on Three.js. This article focuses on Babylon.js, which has gained traction for its extensive API surface and stability.

Field of View and Degrees of Freedom

Immersive experiences are defined by the viewer's perspective. Every headset and smartphone camera has a field of view (FOV) — the extent of the visible environment at any moment. A single human eye sees about 135º; two overlapping eyes cover 220º. Most headsets range between 90º and 150º, according to MDN.

Movement within that field is measured in degrees of freedom (DoF). Three rotational degrees (3DoF) form the baseline for most headsets:

• Pitch — looking up and down, pivoting on the horizontal x-axis.
• Yaw — looking left and right, pivoting on the vertical y-axis.
• Roll — tilting side to side, pivoting on the z-axis extending into the viewport.

For a more natural experience, users expect to move through space, not just rotate in place. That requires six degrees of freedom (6DoF), adding translational movement along the x-, y-, and z-axes: forward and reverse, left and right, up and down. Because translational tracking often demands external sensors, only higher-end headsets support all six.

WebXR Session Modes

WebXR unifies VR and AR under a single API. Each application starts by initiating a session, which represents an active immersive experience. For VR, two session modes exist: inline, which renders the scene into a browser document, and immersive-vr, which requires a headset. AR, limited by rendering targets such as smartphone cameras or transparent glasses, offers only immersive-ar.

Given that many developers lack a headset and the AR module remains in active development, the practical focus here is an immersive-vr session that can also render into a browser canvas.

Building a WebXR Scene With Babylon.js

Babylon.js is a free, open-source WebGL-based rendering engine with built-in WebXR support. It provides a more comprehensive feature set than alternatives like Three.js, which favors extensibility through interchangeable modules. For WebXR work, Babylon.js includes a Default Experience Helper that handles session setup and teardown, input controls, and even provides a basic HTML button for entering immersive mode.

The starter HTML file for this walkthrough loads Babylon.js from a CDN and defines a canvas element in the <body> that will serve as the XR display surface.

<!-- babylon-webxr/index.html -->
<!DOCTYPE html>
<html>
  <head>
    <title>Babylon WebXR Demo</title>
    
    <!-- Embed latest version of Babylon.js. -->
    <script src="https://cdn.babylonjs.com/babylon.js"></script>

    <!-- Embed Babylon loader scripts for .gltf and other filetypes. -->
    <script src="https://cdn.babylonjs.com/loaders/babylonjs.loaders.min.js"></script>

    <!-- Embed pep.js for consistent cross-browser pointer events. -->
    <script src="https://code.jquery.com/pep/0.4.3/pep.js"></script>

    <style>
      html, body {
        overflow: hidden;
        width: 100%;
        height: 100%;
        margin: 0;
        padding: 0;
        box-sizing: border-box;
      }
      #render-canvas {
        width: 100%;
        height: 100%;
        touch-action: none;
      }
    </style>
  </head>

  <body>
    <canvas id="render-canvas"></canvas>

Inside a <script> block before the closing </body> tag, the engine is instantiated by passing the canvas element to the Babylon engine constructor.

<!-- Our Babylon.js implementation. -->
<script>
  // Identify canvas element to script.
  const canvas = document.getElementById('render-canvas');

  // Initialize Babylon.js variables.
  let engine,
      scene,
      sceneToRender;
  const createDefaultEngine = function () {
    return new BABYLON.Engine(canvas, true, {
      preserveDrawingBuffer: true,
      stencil: true
    });
  };

Viewing this file in a WebXR-enabled browser (Chrome, or Firefox with the WebXR flag enabled) shows the default Babylon.js playground scene. You can drag with the mouse to reorient the view.

Understanding WebXR Geometry

Before placing any objects, it helps to recall some WebGL/WebXR geometry fundamentals. In three-dimensional space, positions are expressed as vectors with x-, y-, and z-components. One WebGL unit equals one meter, so the vector (0, 1, 2) means zero units on the x-axis, one meter up the y-axis, and two meters along the z-axis.

WebGL and WebXR distinguish between world space and local space based on the reference space in play. World space is represented by a 2-meter cube centered on the origin (0, 0, 0); when you put on a headset, your initial location is that origin, with the –y-axis in front, the –x-axis to the left, and the –z-axis below. A typical WebXR scene contains many distinct reference spaces, since every object and input controller has its own local frame of reference that relates back to world space.

Consider a sphere positioned at (1, 3, 5) in world space. Its local space places it at (0, 0, 0), its native origin. When you reposition, rotate, or scale the sphere, those transformations affect its relationship to local space, and the engine converts them into world space matrices using the origin offset — the difference between the native and effective origins. MDN's matrix math introduction covers the underlying arithmetic.

Positioning the Camera

With that context, the scene is ready for a camera. A new Babylon scene and camera are instantiated, with the camera placed at vector (0, 5, –10) — five meters above and ten meters behind the world origin — and oriented to look back toward (0, 0, 0), giving a slight downward angle.

// Create scene and create XR experience.
const createScene = async function () {

  // Create a basic Babylon Scene object.
  let scene = new BABYLON.Scene(engine);

  // Create and position a free camera.
  let camera = new BABYLON.FreeCamera('camera-1', new BABYLON.Vector3(0, 5, -10), scene);

  // Point the camera at scene origin.
  camera.setTarget(BABYLON.Vector3.Zero());

  // Attach camera to canvas.
  camera.attachControl(canvas, true);

Local transformations applied to the camera operate on its effective origin. For lower-level camera access, Babylon.js offers the WebXRCamera prototype.

Adding Light

A scene without light renders nothing visible. WebXR lighting, as described by MDN, has three possible components:

• Ambient light is omnipresent and non-directional; its effect is identical everywhere in a scene.
• Diffuse light is emitted or reflected evenly from a surface, with intensity governed by the angle of incidence.
• Specular light produces bright highlights on reflective surfaces like jewelry or eyes.

Babylon.js provides a HemisphericLight prototype for ambient light. Because a hemispheric light has no single position, it only needs a direction vector — here (0, 1, 0), pointing upward toward the sky.

// Create a light and aim it vertically to the sky (0, 1, 0).
let light = new BABYLON.HemisphericLight('light-1', new BABYLON.Vector3(0, 1, 0), scene);

Babylon.js also offers three other light source types. Point lights emit from a single position in all directions (like a bulb) and require just a position vector. Directional lights emit along a single direction (like sunlight) and also need only one vector. Spot lights are defined by a position, a direction, a conical beam angle, and a decay exponent.

The repository branches lighting-1, lighting-2, and lighting-3 demonstrate point, directional, and spot light replacements respectively.

// Create a point light.
let light = new BABYLON.PointLight('light-1', new BABYLON.Vector3(0.5, 5, 0.5), scene);

// Create a directional light.
let light = new BABYLON.DirectionalLight('light-1', new BABYLON.Vector3(-1, 0, 0), scene);

The spot light example places the source high and to the rear at (0, 15, –15) to mimic theater lighting, points it downward and forward with direction (0, –1, 1), limits its beam to π/4 radians (45 degrees), and sets a decay rate of 3.

// Create a spot light.
let light = new BABYLON.SpotLight('light-1', new BABYLON.Vector3(0, 15, -15), new BABYLON.Vector3(0, -1, 1), Math.PI / 4, 3, scene);

The screenshots below compare the visual results of the four light types.

Adjusting Light Parameters

Light properties such as intensity (default 1) and color are configurable. Lights also respond to setEnabled(false) and setEnabled(true). Halving the intensity to 0.25 produces a dimmer scene (lighting-4 branch).

// Set light intensity to a lower value (default is 1).
light.intensity = 0.5;

Diffuse and specular colors are set separately. lighting-5 in the repository uses blue diffuse light and red specular light, yielding a shiny red highlight over a broader blue wash.

// Set light intensity to a lower value (default is 1).
light.intensity = 0.25;

// Set diffuse light to blue and specular light to red.
light.diffuse = new BABYLON.Color3(0, 0, 1);
light.specular = new BABYLON.Color3(1, 0, 0);

Babylon.js documentation covers the full range of lighting options, including lightmaps and projection textures.

Creating Shapes

With camera and lighting in place, physical geometry can populate the scene. Babylon.js's mesh builder supports set shapes — common forms like boxes, spheres, cylinders, cones, polygons, and planes, plus toruses and polyhedra — and parametric shapes generated from input parameters.

A basic sphere with diameter 2 and 32 horizontal segments is created as follows:

// Add one of Babylon's built-in sphere shapes.
let sphere = BABYLON.MeshBuilder.CreateSphere('sphere-1', {
  diameter: 2,
  segments: 32
}, scene);

// Position the sphere up by half of its height.
sphere.position.y = 1;

Distinct x-, y-, and z-axis diameters turn the sphere into an ellipsoid (shapes-1 branch). The depthY and depthZ parameters override the default 2-unit diameter per axis.

// Add one of Babylon's built-in sphere shapes.
let sphere = BABYLON.MeshBuilder.CreateSphere('sphere-1', {
  diameter: 2,
  diameterY: 3,
  diameterZ: 4,
  segments: 32
}, scene);

Applying differentiated diameters to a cylinder creates a cone when one diameter is zero and a truncated cone when both differ (shapes-2 branch). The tessellation parameter determines the number of radial sides rendered.

// Add one of Babylon's built-in cylinder shapes.
let cylinder = BABYLON.MeshBuilder.CreateCylinder('cylinder-1', {
  diameterTop: 2,
  diameterBottom: 5,
  tessellation: 32
}, scene);

// Position the cylinder up by half of its height.
cylinder.position.y = 1;

Babylon.js also supports more advanced geometry: parametric shapes like lines, ribbons, tubes, extruded forms, and lathes; polyhedra with polygonal faces; tiled planes and boxes carrying patterns; and keyframe-driven and built-in animations for materials and objects.

Rendering the Scene

The default Babylon.js environment supplies a ground plane and a skybox — a simulated sky.

// Create a default environment for the scene.
scene.createDefaultEnvironment();

The Default Experience Helper checks for WebXR compatibility, and the helper function returns a constructed scene when support is available.

// Initialize XR experience with default experience helper.
const xrHelper = await scene.createDefaultXRExperienceAsync();
if (!xrHelper.baseExperience) {
  // XR support is unavailable.
  console.log('WebXR support is unavailable');
} else {
  // XR support is available; proceed.
  return scene;
  }

};

A default canvas is created by instantiating a new engine and attaching it to the HTML canvas element.

// Create engine.
engine = createDefaultEngine();
if (!engine) {
  throw 'Engine should not be null';
}

Finally, createScene() is invoked to begin rendering via the engine. In pure WebXR implementations, the XRSession method requestAnimationFrame() supplies each frame; Babylon.js uses the engine's runRenderLoop() method for the same purpose.

// Create scene.
scene = createScene();
scene.then(function (returnedScene) {
  sceneToRender = returnedScene;
});

// Run render loop to render future frames.
engine.runRenderLoop(function () {
  if (sceneToRender) {
    sceneToRender.render();
  }
});

Since the XR application fills the whole browser viewport, a resize event listener keeps scene dimensions in sync when the user resizes the window.

// Handle browser resize.
      window.addEventListener('resize', function () {
        engine.resize();
      });

    </script>
  </body>
</html>

Running the code from any repository branch in a WebXR-compliant browser displays the complete scene. Adding an animation is the natural next experiment to observe the render loop in action.

Approaching User Input

Building an interactive world is more involved than rendering one. WebXR distinguishes targeting (identifying a single point in space via eye-tracking, tapping, or cursor movement) from actions (selections like button presses, or squeezes like trigger pulls). Input can arrive through touchscreens, motion controllers, grip pads, voice commands, and more; WebXR does not mandate preferred input types beyond sensible defaults.

Managing the full breadth of input sources — especially within Babylon.js — is substantial enough to warrant a dedicated article covering eye movements, joystick motion, gamepad input, haptic gloves, and keyboard plus mouse interactions.

Debugging With The Babylon.js Inspector

Babylon.js ships with an inspector built in React that lets you debug your scene directly in the browser. Since the library has no official command-line interface, you add the inspector by loading an extra script alongside the existing Babylon.js files in your <head>:

<!-- Embed Babylon inspector for debugging. -->
<script src="https://cdn.babylonjs.com/inspector/babylon.inspector.bundle.js"></script>

With the script in place, enable debug mode just before your scene is finalized. Add scene.debugLayer.show() right before the return statement in your scene-creation code:

// Initialize XR experience with default experience helper.
const xrHelper = await scene.createDefaultXRExperienceAsync();
if (!xrHelper.baseExperience) {
  // XR support is unavailable.
  console.log('WebXR support is unavailable');
} else {
  // XR support is available; proceed.
  scene.debugLayer.show();
  return scene;
}

On the next page load, you will see a "Scene Explorer" panel for navigating rendered objects and an "Inspector" panel for viewing and modifying the properties of every entity Babylon.js knows about. The debug view is shown in the screenshot below; the tutorial code in the debugging-1 branch reflects this state.

The official documentation covers both loading and using the inspector as well as a series of video guides on inspection and debugging.

Packaging Babylon.js With Other JavaScript

So far the tutorial has used an inline script inside the HTML with the canvas, but for real projects you will likely want an external script or integration with a framework like React or Ionic. Babylon.js publishes all packages to NPM, so you can pull it in with NPM or Yarn as a regular dependency:

# Add ES6 version of Babylon.js as dependency using NPM.
$ npm install @babylonjs/core

# Add ES6 version of Babylon.js as dependency using Yarn.
$ yarn add @babylonjs/core

# Add non-ES6 version of Babylon.js as dependency using NPM.
$ npm install babylonjs

Official docs describe React integration (including the react-babylonjs renderer) and Ionic. For React Native, Julien Noble has published an experimental guide using its web renderer.

For performance, consider server-side rendering of Babylon.js apps. The library provides NullEngine, a headless engine that replaces the standard Engine when WebGL is unavailable, such as in Node.js or server environments. Be aware that you must provide replacements for browser APIs like XMLHttpRequest in frameworks such as Express.

On the client, keeping the bundle small improves parse time. Beyond downloading a minified Babylon.js from its CDN, you can combine Babylon.js with React or other scripts using a bundler like Webpack. Webpack also lets you consume Babylon.js modularly with ES6 and TypeScript, producing a single bundle that covers your full JavaScript footprint.

WebXR Accessibility And Forward Outlook

WebXR is still maturing, but adoption will grow as more people look for fully immersive virtual and augmented experiences. Browser support is firming up, and developer tooling is improving; the complete code from this tutorial is available on GitHub for reference.

Those immersive experiences are not without costs. Virtual reality works by convincing the eyes and brain to accept objects that are not physically present, which can trigger virtual reality sickness — marked by disorientation, discomfort, and nausea. Real-world objects hidden by a headset can be hazardous, and many immersive experiences remain unusable for people with visual, cognitive, or vestibular impairments.

Mixed reality also remains out of reach for many users who lack a headset or a WebXR-enabled browser, and for developers navigating shifting specifications. Still, as digital marketing turns toward immersive media, expect this corner of the web to take shape quickly.

Related Reading

WebXR

• Fundamentals of WebXR
• WebXR application life cycle
• Starting up and shutting down a WebXR session
• Movement, orientation, and motion: A WebXR example

Babylon.js WebXR

• Introduction to WebXR
• WebXR Experience Helpers
• WebXR Session Managers
• WebXR Camera
• WebXR Features Manager
• WebXR demos and examples
• WebXR input and controller support
• WebXR selected features
• WebXR augmented reality

Graphics Background

• Learn WebGL
• WebGL fundamentals
• Learn OpenGL