Building an API in Node is a natural next step for front-end developers, but the shift from browser JavaScript to server-side JavaScript involves several conceptual adjustments—especially around asynchronous execution, error handling, and the module system.

Node has been built around a single-threaded event loop with asynchronous callbacks since its inception; the Node team blogged in 2011 that it "promotes an asynchronous coding style from the ground up," and Ryan Dahl, in his 2009 talk announcing Node, described the performance benefits of doubling down on asynchronous JavaScript. This asynchronous-first design helped Node outpace earlier server-side JavaScript efforts like Netscape's application servers and Narwhal. For developers used to fetch, setTimeout, or event handling on the client, the difference is one of degree: in Node, asynchronous code is nearly constant.

Building a sample quiz API with SQLite

The practical case here is a quiz app in which users build quizzes from multiple-choice questions. The data lives in an SQLite database, with a table for quizzes (integer ID, text title), a table for questions (integer ID, body text, integer reference to the owning quiz), and a table for answers (integer ID, body text, correctness flag, integer reference to the owning question). Because SQLite has no boolean type, correctness is stored as an integer where 0 is false and 1 is true.

Setup starts by initializing npm and installing the sqlite3 package:

npm init -y
npm install sqlite3

To use modern ES6 module syntax, "type": "module" is added to the top-level object in package.json. A migration script, migrate.js, then creates the tables and seeds one quiz, one question, and four answers into quiz.db:

// migrate.js

import sqlite3 from "sqlite3"; 

let db = new sqlite3.Database("quiz.db");
    db.serialize(function () {
      // Setting up our tables:
      db.run("CREATE TABLE quiz (quizid INTEGER PRIMARY KEY, title TEXT)");
      db.run("CREATE TABLE question (questionid INTEGER PRIMARY KEY, body TEXT, questionquiz INTEGER, FOREIGN KEY(questionquiz) REFERENCES quiz(quizid))");
      db.run("CREATE TABLE answer (answerid INTEGER PRIMARY KEY, body TEXT, iscorrect INTEGER, answerquestion INTEGER, FOREIGN KEY(answerquestion) REFERENCES question(questionid))");
      // Create a quiz with an id of 0 and a title "my quiz" 
      db.run("INSERT INTO quiz VALUES(0,\"my quiz\")");
      // Create a question with an id of 0, a question body
      // and a link to the quiz using the id 0
      db.run("INSERT INTO question VALUES(0,\"What is the capital of France?\", 0)");
      // Create four answers with unique ids, answer bodies, an integer for whether
      // they're correct or not, and a link to the first question using the id 0
      db.run("INSERT INTO answer VALUES(0,\"Madrid\",0, 0)");
      db.run("INSERT INTO answer VALUES(1,\"Paris\",1, 0)");
      db.run("INSERT INTO answer VALUES(2,\"London\",0, 0)");
      db.run("INSERT INTO answer VALUES(3,\"Amsterdam\",0, 0)");
  });
db.close();

The script runs from the command line:

node migrate.js

The resulting database file can be inspected with a tool such as DB Browser for SQLite. In index.js, a sqlite3.Database is opened against the file path, and its get method takes an SQL string and a callback:

// index.js
import sqlite3 from "sqlite3";

let db = new sqlite3.Database("quiz.db");

db.get(`SELECT * FROM quiz WHERE quizid  = 0`, (err, row) => {
  if (err) {
    console.error(err.message);
  }
  console.log(row);
  db.close();
});

Running this prints { quizid: 0, title: 'my quiz' }.

Why naive callback usage fails

Passing a quiz ID into a function and using only the return value seems straightforward, and it does not work. The first instinct is often to declare let result; in the outer function and reassign it inside the callback:

// index.js
// Be warned! This code contains BUGS
import sqlite3 from "sqlite3";

function getQuiz(id) {
  let db = new sqlite3.Database("quiz.db");
  let result;
  db.get(`SELECT * FROM quiz WHERE quizid  = ?`, [id], (err, row) => {
    if (err) {
      return console.error(err.message);
    }
    db.close();
    result = row;
  });
  return result;
}
console.log(getQuiz(0));

The console prints undefined, for four reasons: result is unassigned; db.get() hands a file-system read off to the background while execution continues; the outer function returns before the callback fires; and when SQLite finishes reading and finally runs the callback, the return value has already been fixed.

Callbacks, nesting, and promises

The pre-2015 remedy is to pass the callback in alongside the quiz ID so it receives the row:

// index.js
import sqlite3 from "sqlite3";
function getQuiz(id, callback) {
  let db = new sqlite3.Database("quiz.db");
  db.get(`SELECT * FROM quiz WHERE quizid  = ?`, [id], (err, row) => {
    if (err) {
       console.error(err.message);
    }
    else {
       callback(row);
    }
    db.close();
  });
}
getQuiz(0,(quiz)=>{
  console.log(quiz);
});

Queries chained in sequence are where this style breaks down. To look up which quiz an answer belongs to, getQuiz is generalized into a get function that also accepts the table and column. Since SQL parameters cannot be used for a table name, the query uses a template string; production code would need to scrub that string to prevent SQL injection.

function get(params, callback) {
  // In production these strings should be scrubbed to prevent SQL injection
  const { table, column, value } = params;
  let db = new sqlite3.Database("quiz.db");
  db.get(`SELECT * FROM ${table} WHERE ${column} = ${value}`, (err, row) => {
    callback(err, row);
    db.close();
  });
}

Error handling adds another layer: following Node's convention, the error object is the first callback argument, and each query must be checked before continuing. Looking up the quiz for answer id 2 reads as follows:

// index.js
import sqlite3 from "sqlite3";

function get(params, callback) {
  // In production these strings should be scrubbed to prevent SQL injection
  const { table, column, value } = params;
  let db = new sqlite3.Database("quiz.db");
  db.get(`SELECT * FROM ${table} WHERE ${column} = ${value}`, (err, row) => {
    callback(err, row);
    db.close();
  });
}

get({ table: "answer", column: "answerid", value: 2 }, (err, answer) => {
  if (err) {
    console.log(err);
  } else {
    get(
      { table: "question", column: "questionid", value: answer.answerquestion },
      (err, question) => {
        if (err) {
          console.log(err);
        } else {
          get(
            { table: "quiz", column: "quizid", value: question.questionquiz },
            (err, quiz) => {
              if (err) {
                console.log(err);
              } else {
                // This is the quiz our answer belongs to
                console.log(quiz);
              }
            }
          );
        }
      }
    );
  }
});

Each step in the chain adds two levels of nesting—one to check for errors, one for the next callback. Named functions reduce the depth but add verbosity and require inventing names for every intermediate callback.

Promises, which arrived in Node in 2015, flatten the structure. Wrapping the query in a Promise and passing callbacks to then changes the lookup code substantially:

// index.js
import sqlite3 from "sqlite3";
function get(params) {
  // In production these strings should be scrubbed to prevent SQL injection
  const { table, column, value } = params;
  let db = new sqlite3.Database("quiz.db");

  return new Promise(function (resolve, reject) {
    db.get(`SELECT * FROM ${table} WHERE ${column} = ${value}`, (err, row) => {
      if (err) {
        return reject(err);
      }
      db.close();
      resolve(row);
    });
  });
}
get({ table: "answer", column: "answerid", value: 2 })
  .then((answer) => {
    return get({
      table: "question",
      column: "questionid",
      value: answer.answerquestion,
    });
  })
  .then((question) => {
    return get({
      table: "quiz",
      column: "quizid",
      value: question.questionquiz,
    });
  })
  .then((quiz) => {
    console.log(quiz);
  })
  .catch((error) => {
    console.log(error);
  }
);

Errors no longer need individual handling per call; a single catch covers the whole chain.

Async/await and top-level await

Node 7.6.0, which shipped with V8 5.5, added ES2017 async/await. Because get() already returns a promise, only the user code needs to change:

async function printQuizFromAnswer() {
  const answer = await get({ table: "answer", column: "answerid", value: 2 });
  const question = await get({
    table: "question",
    column: "questionid",
    value: answer.answerquestion,
  });
  const quiz = await get({
    table: "quiz",
    column: "quizid",
    value: question.questionquiz,
  });
  console.log(quiz);
}

printQuizFromAnswer();

Node's more recent top-level await makes this even more concise by dropping the printQuizFromAnswer() wrapper. The resulting code runs the tasks sequentially while other asynchronous work—file reads, HTTP responses—continues in the background.

Pushing work into SQL

An alternative to chaining fetches is to retrieve everything in a single SQL JOIN:

// index.js
import sqlite3 from "sqlite3";

function quizFromAnswer(answerid, callback) {
  let db = new sqlite3.Database("quiz.db");
  db.get(
    `SELECT *,a.body AS answerbody, ques.body AS questionbody FROM answer a 
    INNER JOIN question ques ON a.answerquestion=ques.questionid 
    INNER JOIN quiz quiz ON ques.questionquiz = quiz.quizid 
    WHERE a.answerid = ?;`,
    [answerid],
    (err, row) => {
      if (err) {
        console.log(err);
      }
      callback(err, row);
      db.close();
    }
  );
}
quizFromAnswer(2, (e, r) => {
  console.log(r);
});

The result is one object containing the answer, question, and quiz, with the two body columns renamed to answerbody and questionbody. Moving logic into the database layer can simplify the JavaScript and possibly improve performance, at the cost of learning SQL's feature set on top of everything else.

New APIs and Node conventions

  • Database connections, filesystem reads (as in the error-first readFile usage below), HTTP server setup, OS checks via the OS module, and encryption via the Crypto module have no browser counterpart.
  • Browser-side HTTP requests have no Node equivalent in fetch or XMLHttpRequest; the https module is the standard route, though a pull request in the node.js repository indicates fetch is on the way. Deno has taken aim at closing these gaps.
  • package.json will be familiar to anyone who has used front-end build tools, but the main property, which points at a library's entry point, may not be.
  • Error-first callbacks, as in the example below, are a widespread convention rather than a language feature.
import fs from 'fs';

fs.readFile('myfile.txt', 'utf8' , (err, data) => {
  if (err) {
    console.error(err)
    return
  }
  console.log(data)
})

Modules and operational hazards

Adding "type":"module" to package.json enables native modules, but the history is split. Because JavaScript had no module specification in 2009, Node's creators built Common.js, and when a module spec arrived in 2015 Node's system diverged from the standard. The Node team has since made native modules usable. Many tutorials and npm packages still rely on the older system, and some libraries use native modules in incompatible ways.

Two production concerns round out the differences. First, an unhandled fatal exception on a Node server terminates it and leaves every user unserved—unlike client-side bugs, which affect one user who can refresh. Second, a back end widens the attack surface well beyond cross-site scripting and cross-site request forgery, adding risks such as brute force attacks and SQL injection; anyone storing other people's data with Node carries a corresponding responsibility.