Home/Engineering/Elegant APIs with JSON Schema

Engineering

Elegant APIs with JSON Schema

BL

Brandur LeachBrandur

·

JSON Schema: A Declarative Foundation for API Design

API description formats have proliferated in recent years—Swagger, Blueprint, and RAML among them. JSON Schema takes a different route: rather than inventing a new syntax, it builds directly on JSON itself to create a declarative language for validating the format and structure of JSON objects. Combined with JSON Hyper-schema, it becomes a tool for describing not just data shapes but entire HTTP APIs, including links, requests, and responses.

At its core, JSON Schema lets you specify primitives that define exactly what a valid JSON object looks like, with a nesting mechanism that scales from a single value to documents of arbitrary complexity. The concept echoes the XML era, when documents commonly referenced XSDs for validation.

The simplest schema describes a single value:

{
  "type": "string"
}

The string "foo" passes validation here, while 123 or false would not. More sophisticated constraints can be layered in, such as validating a string against a regex pattern:

{
  "pattern": "^[a-z][a-z0-9-]{2,30}$",
  "type": "string"
}

Composing Complex Objects

Validating single values is only the starting point. The real power comes from composing nested validation using the properties keyword, which maps object keys to schemas that validate their values:

{
  "properties": {
    "name": {
      "pattern": "^[a-z][a-z0-9-]{2,30}$",
      "type": "string"
    }
  },
  "required": ["name"],
  "type": "object"
}

The required keyword marks name as mandatory, so {"name":"foo"} is valid but {} is rejected. A key elegance emerges here: every object in a schema is itself a schema conforming to the same specification. Subschemas can be hoisted into a definitions section and referenced elsewhere, promoting reuse without implying that those keys are actual properties on the validated object:

{
  "definitions": {
    "name": {
      "pattern": "^[a-z][a-z0-9-]{2,30}$",
      "type": "string"
    }
  },
  "properties": {
    "name": {
      "$ref": "#/definitions/name"
    }
  },
  "required": ["name"],
  "type": "object"
}

The $ref keyword is a JSON Reference—it tells parsers to fetch a schema from elsewhere in the document (or a different one), rather than treating the value as an inline schema. The # designates the document root, and slashes descend through keys until the target value is reached.

Nesting to Arbitrary Depth

Schemas compose to any level. Consider building a root schema for an API that defines both an app and a domain resource:

{
  "definitions": {
    "name": {
      "format": "hostname",
      "type": "string"
    }
  },
  "properties": {
    "name": {
      "$ref": "#/definitions/name"
    }
  },
  "required": ["name"],
  "type": "object"
}

Domain mirrors app's structure with its own name, but constrained to the hostname format—a built-in string validation in JSON Schema. Wiring both into a single root schema adjusts the references to reflect the deeper nesting:

{
  "definitions": {
    "app": {
      "definitions": {
        "domains": {
          "items": {
            "$ref": "#/definitions/domain"
          },
          "type": "array"
        },
        "name": {
          "pattern": "^[a-z][a-z0-9-]{2,30}$",
          "type": "string"
        }
      },
      "properties": {
        "domains": {
          "$ref": "#/definitions/app/definitions/domains"
        },
        "name": {
          "$ref": "#/definitions/app/definitions/name"
        }
      },
      "required": ["name"],
      "type": "object"
    },
    "domain": {
      "definitions": {
        "name": {
          "format": "hostname",
          "type": "string"
        }
      },
      "properties": {
        "name": {
          "$ref": "#/definitions/domain/definitions/name"
        }
      },
      "required": ["name"],
      "type": "object"
    }
  },
  "properties": {
    "app": {
      "$ref": "#/definitions/app"
    },
    "domain": {
      "$ref": "#/definitions/domain"
    }
  },
  "type": "object"
}

Alongside the new domain resource, app gains a property:

"domains": {
  "items": {
    "$ref": "#/definitions/domain",
  },
  "type": "array"
}

The items keyword applies specifically to arrays, declaring that every element must validate against the referenced schema. In this case, domains must be an array of objects matching the domain schema, so this array is valid:

[
  { "name": "example.com" },
  { "name": "heroku.com" }
]

This demonstrates two things: schemas nest to any depth, and subschemas can reference each other to build modular validation rules. The root schema may describe an object containing both an app and a domain simultaneously, which seems non-sensical—but that matters little once we move from validation to API construction.

From Validation to Hyper-Schema

JSON Hyper-schema extends the base specification to host a collection of links, moving from pure validation into API definition. Here are two links on the app schema—one for creating an app via POST /apps, another for listing via GET /apps:

{
  "definitions": ...,
  "links": [
    {
      "description": "Create a new app.",
      "href": "/apps",
      "method": "POST",
      "rel": "create",
      "title": "Create"
    },
    {
      "description": "List apps.",
      "href": "/apps",
      "method": "GET",
      "rel": "instances",
      "title": "List"
    }
  ],
  "properties": ...,
  "required": ["name"],
  "type": "object"
}

Each link declares an HTTP endpoint with a method verb and a href URI, tagged with metadata useful for documentation and code generation.

Describing Requests

Knowing the endpoints isn't enough—we need to specify what parameters to send. Hyper-schema lets links declare request schemas, reusing references already defined:

{
  "description": "Create a new app.",
  "href": "/apps",
  "method": "POST",
  "rel": "create",
  "schema": {
    "$ref": "#/definitions/app"
  },
  "title": "Create"
}

In non-trivial cases, the request payload should be a subset of what a fully valid object allows. Because the request definition is itself a schema, it can deconstruct and reference particular properties:

{
  "description": "Create a new app.",
  "href": "/apps",
  "method": "POST",
  "rel": "create",
  "schema": {
    "properties": {
      "name": {
        "$ref": "#/definitions/app/definitions/name"
       }
     },
     "required": ["name"],
     "type": "object"
  },
  "title": "Create"
}

A request hitting such an endpoint might look like:

curl -X POST http://example.com/apps \
  -H "Content-Type: application/json" \
  -d '{"name":"my-app"}'

The name requirement could be dropped by removing "required": ["name"], allowing an empty object {} as a valid request if the API should auto-generate a name. The modularity is worth noting: name is defined once on the app object, referenced to describe a valid app, and referenced again to describe a request.

Declarative request definitions simplify input sanitization and automatic error generation. In Ruby, the Committee middleware collection provides schema-related tooling for this. The API modeled here expects JSON input rather than form-encoded data, though hyper-schema's encType allows any format. Symmetric JSON requests and responses make for a clean API model.

Describing Responses

The targetSchema keyword specifies the response shape. For the create-app endpoint, the response is the app itself:

{
  "description": "Create a new app.",
  "href": "/apps",
  "method": "POST",
  "rel": "create",
  "targetSchema": {
    "$ref": "#/definitions/app"
  },
  "title": "Create"
}

For the list endpoint, the response is an array of apps:

{
  "description": "List apps.",
  "href": "/apps",
  "method": "GET",
  "rel": "instances",
  "targetSchema": {
    "items": {
      "$ref": "#/definitions/app"
    },
    "type": "array"
  },
  "title": "List"
}

Both reuse the existing object definitions. Knowing expected response shapes makes acceptance-level regression testing straightforward—again, Committee offers Ruby test helpers for rack-test.

Meta-Schemas: Enforcing Conventions

Schema and hyper-schema each provide meta-schemas, since a schema is just JSON and can therefore be validated like any other document. The hyper-schema meta-schema uses the $schema keyword to point back to its own id. This allows a tool like the json_schema gem to validate your hyper-schema's format:

validate-schema --detect my-schema.json

Convention is hard to enforce across a team with differing ideas about API design. A declarative solution is to write a meta-schema that layers additional constraints on top of the base specifications. Where hyper-schema only demands href and rel on links, we can require more:

{
  "$schema": "http://example.com/my-hyper-schema",
  "definitions": {
    "resource": {
      "properties": {
        "links": {
          "items": {
            "$ref": "#/definitions/link"
          },
          "type": "array"
        }
      }
    },
    "link": {
      "required": [ "href", "method", "rel", "targetSchema" ],
      "type": "object"
    }
  },
  "id": "http://example.com/my-hyper-schema#",
  "title": "My JSON Hyper-Schema Variant",
  "properties": {
    "definitions": {
      "additionalProperties": {
        "$ref": "#/definitions/resource"
      }
    }
  }
}

The specification declares that everything under definitions in the hyper-schema is an API resource, and those resources may contain links. Each link must carry href, method, rel, and targetSchema. Running validate-schema from json_schema confirms the schema is valid:

validate-schema -d -s meta.json schema.json
schema.json is valid.

Omitting targetSchema from a link produces an error instead:

validate-schema -d -s meta.json schema.json
schema.json#/definitions/app/links/0: failed schema #/definitions/resource/properties/links/items: Missing required keys "targetSchema" in object; keys are "description, href, method, rel, schema, title".

Conventions can go further—for example, mandating lowercase property names with underscores only:

"resource": {
  "properties": {
    ...,
    "properties": {
      "additionalProperties": false,
      "patternProperties": {
        "^[a-z][a-z_]+[a-z]$": {}
      }
    }
  }
},

Here patternProperties matches schemas against property names rather than values, and additionalProperties: false rejects keys not in properties or patternProperties. The property names in our schema all pass:

validate-schema -d -s meta.json schema.json
schema.json is valid.

Layering Meta-Schemas

The hyper-schema meta-schema itself uses an allOf attribute to assert that data validates against both its own constraints and the JSON Schema meta-schema. The same pattern applies when building a custom variant:

{
  "$schema": "http://example.com/my-hyper-schema#",
  "allOf": [
      {
          "$ref": "http://json-schema.org/draft-04/hyper-schema#"
      }
  ],
  ...
}

This compositional approach means conventions can be layered without weakening the underlying standards—each meta-schema inherits all upstream constraints and adds its own.

Schema endpoint

Heroku follows the convention of exposing the API schema itself at GET /schema. A useful trick is including a link to this endpoint within the schema, with a reference to the meta-schema as its validation target. This way, the schema verifies itself against the meta-schema using the same machinery used for any other response check in the acceptance test suite.

{
  "href": "/schema",
  "method": "GET",
  "rel": "self",
  "targetSchema": {
    "$ref": "http://example.com/my-hyper-schema#",
  }
}

The complete code for this simple hyper-schema and its meta-schema is published on GitHub.

Beyond JSON

If hyper-schema itself is not the right fit, Hyperschema.org hosts schemas for other media types, including established hypermedia formats such as Collection+JSON, HAL, and UBER.

What the schema covers

In practice, JSON Schema here defines several layers for the API:

Individual resources like app and domain.
A single-document “super schema” aggregating all resources.
Hyper-schema links spelling out actions on those resources.
Schemas to validate incoming payloads on each link.
Schemas describing the response JSON for each link.
A meta-schema that enforces the API’s own structural conventions.

What you get for free

The implementation still requires the application logic, but pairing this schema with the surrounding HTTP toolchain offers ready-made features:

Automatic API documentation via Prmd.
A Ruby client generated with Heroics.
A Go client generated with Schematic, the same pattern used in Heroku’s hk CLI.
A working stub server with Committee for proving out endpoint behavior.
Request validation middleware from Committee that checks incoming data against the schema before reaching the backend.
Committee’s test helpers allow verifying that your stack’s responses conform to the schema.