Understanding Plugin Development In Gatsby — Smashing Magazine

Gatsby ships with a fixed feature set, and plugins exist to extend it with anything the core does not handle — generating a manifest.json for a progressive web app, embedding tweets, logging page views, and similar tasks.

Plugin Types And Plugin Forms

Two distinctions matter here, and they are independent of each other: where a plugin lives, and what job it performs.

By location, plugins are either local or external. A local plugin sits inside a Gatsby project under the /plugins directory. External plugins are fetched through npm or Yarn; alternatively they can live on the same machine and be wired into a project with the yarn link or npm link command.

By use case, plugins fall into three forms:

• Source plugins — supply data sources to a Gatsby site, e.g. gatsby-source-filesystem, gatsby-source-contentful, gatsby-source-wordpress.
• Transformer plugins — take data produced by source plugins and convert it into a more usable form, e.g. gatsby-transformer-remark, gatsby-transformer-json, gatsby-transformer-sharp.
• Generic plugins — everything beyond sourcing and transforming, e.g. gatsby-plugin-mdx and gatsby-plugin-sharp.

The API Files That Make Up A Plugin

Gatsby's documentation calls the files a plugin can define its API files, and they belong in the root of the plugin directory, whether that directory is local or under node_modules:

• gatsby-node.js — hooks into Gatsby's build processes.
• gatsby-config.js — handles configuration and setup.
• gatsby-browser.js — runs code in the browser during one of Gatsby's processes.
• gatsby-ssr.js — customizes and extends the server-side rendering (SSR) process.

None of these are mandatory. The comment plugin built alongside this article implements only two of them — gatsby-node.js and gatsby-config.js.

Sourcing Comments Into Gatsby's Data Layer

Our example plugin is a comment system that can be installed on any Gatsby blog. Comments are read from and written to a hosted comment server at gatsbyjs-comment-server.herokuapp.com, which exposes GET /comments for loading and POST /comments for saving. A POST body carries four fields: content: [string] (the comment text), author: [string] (the commenter's name), website (the originating site), and slug (the page the comment belongs to).

Plugin Scaffolding

Start from the plugin boilerplate, the same approach used when creating a Gatsby blog. From any convenient directory, run:

gatsby new gatsby-source-comment-server https://github.com/Gatsbyjs/gatsby-starter-plugin

Change into the plugin directory, open it in an editor, and add axios for the network calls to the comment server:

npm install axios --save
// or
yarn add axios

Defining A Comment Node Type

Comments need a node type to extend before they can be fetched, so gatsby-node.js should declare one:

exports.sourceNodes = async ({ actions }) => {
  const { createTypes } = actions;
  const typeDefs = `
    type CommentServer implements Node {
      _id: String
      author: String
      string: String
      content: String
      website: String
      slug: String
      createdAt: Date
      updatedAt: Date
    }
  `;
  createTypes(typeDefs);
};

Here actions is pulled from Gatsby's APIs and the createTypes action is extracted from it. A CommentServer type extending Node.js is defined and passed to createTypes.

Fetching And Linking Comments

With the type in place, axios can pull comments and store them as CommentServer nodes — the process Gatsby calls node sourcing. Sourcing requires implementing the sourceNodes API in gatsby-node.js: make the network request, shape the response to match a GraphQL type, then create a node in Gatsby's GraphQL layer with the createNode action.

const axios = require("axios");

exports.sourceNodes = async (
  { actions, createNodeId, createContentDigest },
  pluginOptions
) => {
  const { createTypes } = actions;
  const typeDefs = `
    type CommentServer implements Node {
      _id: String
      author: String
      string: String
      website: String
      content: String
      slug: String
      createdAt: Date
      updatedAt: Date
    }
  `;
  createTypes(typeDefs);

  const { createNode } = actions;
  const { limit, website } = pluginOptions;
  const _website = website || "";

  const result = await axios({
    url: `https://Gatsbyjs-comment-server.herokuapp.com/comments?limit=${_limit}&website=${_website}`,
  });

  const comments = result.data;

  function convertCommentToNode(comment, { createContentDigest, createNode }) {
    const nodeContent = JSON.stringify(comment);

    const nodeMeta = {
      id: createNodeId(`comments-${comment._id}`),
      parent: null,
      children: [],
      internal: {
        type: `CommentServer`,
        mediaType: `text/html`,
        content: nodeContent,
        contentDigest: createContentDigest(comment),
      },
    };

    const node = Object.assign({}, comment, nodeMeta);
    createNode(node);
  }

  for (let i = 0; i < comments.data.length; i++) {
    const comment = comments.data[i];
    convertCommentToNode(comment, { createNode, createContentDigest });
  }
};

The code imports axios, sets defaults for cases where plugin options are absent, and requests the comments endpoint. A helper converts comments into Gatsby nodes via the action helpers, and the fetched comments are iterated so each one passes through convertCommentToNode.

Comments also have to resolve against their posts. The createResolvers API handles this, again in gatsby-node.js:

exports.createResolvers = ({ createResolvers }) => {
  const resolvers = {
    MarkdownRemark: {
      comments: {
        type: ["CommentServer"],
        resolve(source, args, context, info) {
          return context.nodeModel.runQuery({
            query: {
              filter: {
                slug: { eq: source.fields.slug },
              },
            },
            type: "CommentServer",
            firstOnly: false,
          });
        },
      },
    },
  };
  createResolvers(resolvers);
};

This extends MarkdownRemark with a comments field that resolves to the CommentServer type, matched on the slug stored with the comment and the slug of the post.

The combined sourcing and transforming logic for the plugin's gatsby-node.js looks like this:

const axios = require("axios");

exports.sourceNodes = async (
  { actions, createNodeId, createContentDigest },
  pluginOptions
) => {
  const { createTypes } = actions;
  const typeDefs = `
    type CommentServer implements Node {
      _id: String
      author: String
      string: String
      website: String
      content: String
      slug: String
      createdAt: Date
      updatedAt: Date
    }
  `;
  createTypes(typeDefs);

  const { createNode } = actions;
  const { limit, website } = pluginOptions;
  const _limit = parseInt(limit || 10000); // FETCH ALL COMMENTS
  const _website = website || "";

  const result = await axios({
    url: `https://Gatsbyjs-comment-server.herokuapp.com/comments?limit=${_limit}&website=${_website}`,
  });

  const comments = result.data;

  function convertCommentToNode(comment, { createContentDigest, createNode }) {
    const nodeContent = JSON.stringify(comment);

    const nodeMeta = {
      id: createNodeId(`comments-${comment._id}`),
      parent: null,
      children: [],
      internal: {
        type: `CommentServer`,
        mediaType: `text/html`,
        content: nodeContent,
        contentDigest: createContentDigest(comment),
      },
    };

    const node = Object.assign({}, comment, nodeMeta);
    createNode(node);
  }

  for (let i = 0; i < comments.data.length; i++) {
    const comment = comments.data[i];
    convertCommentToNode(comment, { createNode, createContentDigest });
  }
};

exports.createResolvers = ({ createResolvers }) => {
  const resolvers = {
    MarkdownRemark: {
      comments: {
        type: ["CommentServer"],
        resolve(source, args, context, info) {
          return context.nodeModel.runQuery({
            query: {
              filter: {
                website: { eq: source.fields.slug },
              },
            },
            type: "CommentServer",
            firstOnly: false,
          });
        },
      },
    },
  };
  createResolvers(resolvers);
};

Persisting Comments As JSON

Storing each slug's comments in its own JSON file allows them to be fetched over HTTP on demand instead of through a GraphQL query. Implement the createPageStatefully API in the plugin's gatsby-node.js and use the fs module to verify a path exists before writing into it:

import fs from "fs"
import {resolve: pathResolve} from "path"
exports.createPagesStatefully = async ({ graphql }) => {
  const comments = await graphql(
    `
      {
        allCommentServer(limit: 1000) {
          edges {
            node {
              name
              slug
              _id
              createdAt
              content
            }
          }
        }
      }
    `
  )

  if (comments.errors) {
    throw comments.errors
  }

  const markdownPosts = await graphql(
    `
      {
        allMarkdownRemark(
          sort: { fields: [frontmatter___date], order: DESC }
          limit: 1000
        ) {
          edges {
            node {
              fields {
                slug
              }
            }
          }
        }
      }
    `
  )

  const posts = markdownPosts.data.allMarkdownRemark.edges
  const _comments = comments.data.allCommentServer.edges

  const commentsPublicPath = pathResolve(process.cwd(), "public/comments")

  var exists = fs.existsSync(commentsPublicPath) //create destination directory if it doesn't exist

  if (!exists) {
    fs.mkdirSync(commentsPublicPath)
  }

  posts.forEach((post, index) => {
    const path = post.node.fields.slug
    const commentsForPost = _comments
      .filter(comment => {
        return comment.node.slug === path
      })
      .map(comment => comment.node)

    const strippedPath = path
      .split("/")
      .filter(s => s)
      .join("/")
    const _commentPath = pathResolve(
      process.cwd(),
      "public/comments",
      `${strippedPath}.json`
    )
    fs.writeFileSync(_commentPath, JSON.stringify(commentsForPost))
  })
}

The implementation requires fs and resolves the path module's function. A GraphQL helper pulls the comments stored earlier, avoiding extra HTTP requests, and removes the Markdown files created by the helper. It checks that the comment path is present under the public path, creating it if not. Then it loops over all nodes of the Markdown type, pulls the comments for the current post, and writes them into public/comments under a filename taken from the post's slug.

Because the root .gitignore of a Gatsby site excludes the public path from commits, writing there is safe. On every rebuild, Gatsby invokes this API so comments are re-fetched and saved locally as JSON.

Client-Side Rendering With gatsby-browser.js

Rendering in the browser is handled by the gatsby-browser.js API file. The plugin needs an insertion point in the page, so every page that uses it is expected to contain an HTML element whose ID is commentContainer.

Route Update As The Entry Point

Fetching files and inserting components is best done when a page has just been visited, which the onRouteUpdate API provides; it passes apiHelpers and pluginOpions to the callback.

exports.onRouteUpdate = async (apiHelpers, pluginOptions) => {
  const { location, prevLocation } = apiHelpers
}

A small helper keeps the code tidy by creating an element, assigning its className and adding content:

// Creates element, set class. innerhtml then returns it.
 function createEl (name, className, html = null) {
  const el = document.createElement(name)
  el.className = className
  el.innerHTML = html
  return el
}

The comments section header is appended at the insertion point from within onRouteUpdate, after confirming the element exists and building it with the createEl helper:

// ...

exports.onRouteUpdate = async ({ location, prevLocation }, pluginOptions) => {
  const commentContainer = document.getElementById("commentContainer")
  if (commentContainer && location.path !== "/") {
    const header = createEl("h2")
    header.innerHTML = "Comments"
    commentContainer.appendChild(header)
  }
}

Rendering The Comment List

The list itself is a ul appended to the insertion point via createEl, with its className set to comment-list:

exports.onRouteUpdate = async ({ location, prevLocation }, pluginOptions) => {
  const commentContainer = document.getElementById("commentContainer")
  if (commentContainer && location.path !== "/") {
    const header = createEl("h2")
    header.innerHTML = "Comments"
    commentContainer.appendChild(header)
    const commentListUl = createEl("ul")
    commentListUl.className = "comment-list"
    commentContainer.appendChild(commentListUl)
}

Comments saved in the public directory then render into that ul as li elements. A helper loads the comments for a page by path name:

// Other helpers
const getCommentsForPage = async slug => {
  const path = slug
    .split("/")
    .filter(s => s)
    .join("/")
  const data = await fetch(`/comments/${path}.json`)
  return data.json()
}
// ... implements routeupdate below

The getCommentsForPage helper accepts paths, loads the comments from public/comments using fetch, parses them as JSON and returns them to the caller. It is invoked in the onRouteUpdate callback:

// ... helpers
exports.onRouteUpdate = async ({ location, prevLocation }, pluginOptions) => {
  const commentContainer = document.getElementById("commentContainer")
  if (commentContainer && location.path !== "/") {
    //... inserts header
    const commentListUl = createEl("ul")
    commentListUl.className = "comment-list"
    commentContainer.appendChild(commentListUl)
   const comments = await getCommentsForPage(location.pathname)
}

A further helper builds the individual list items:

// .... other helpers

const getCommentListItem = comment => {
  const li = createEl("li")
  li.className = "comment-list-item"

  const nameCont = createEl("div")
  const name = createEl("strong", "comment-author", comment.name)
  const date = createEl(
    "span",
    "comment-date",
    new Date(comment.createdAt).toLocaleDateString()
  )
  // date.className="date"
  nameCont.append(name)
  nameCont.append(date)

  const commentCont = createEl("div", "comment-cont", comment.content)

  li.append(nameCont)
  li.append(commentCont)
  return li
}

// ... onRouteUpdateImplementation

Each li gets a className of comment-list-item, plus a div for the author and time and another for the comment's text with a className of comment-cont. Rendering iterates the comments returned by getComments, calls getCommentListItem for each, and appends the result to the <ul class="comment-list"></ul> element:

// ... helpers
exports.onRouteUpdate = async ({ location, prevLocation }, pluginOptions) => {
  const commentContainer = document.getElementById("commentContainer")
  if (commentContainer && location.path !== "/") {
    //... inserts header
    const commentListUl = createEl("ul")
    commentListUl.className = "comment-list"
    commentContainer.appendChild(commentListUl)
   const comments = await getCommentsForPage(location.pathname)
    if (comments && comments.length) {
      comments.map(comment => {
        const html = getCommentListItem(comment)
        commentListUl.append(html)
        return comment
      })
    }
}

Posting A Comment

Posting means a POST to the API's /comments endpoint, which requires a form. A form helper returns the HTML form element:

// ... other helpers
const createCommentForm = () => {
  const form = createEl("form")
  form.className = "comment-form"
  const nameInput = createEl("input", "name-input", null)
  nameInput.type = "text"
  nameInput.placeholder = "Your Name"
  form.appendChild(nameInput)
  const commentInput = createEl("textarea", "comment-input", null)
  commentInput.placeholder = "Comment"
  form.appendChild(commentInput)
  const feedback = createEl("span", "feedback")
  form.appendChild(feedback)
  const button = createEl("button", "comment-btn", "Submit")
  button.type = "submit"
  form.appendChild(button)
  return form
}

That helper produces an input with a className of name-input, a textarea with a className of comment-input, a span with a className of feedback, and a button with a className of comment-btn. The form is then appended at the insertion point using createCommentForm:

// ... helpers
exports.onRouteUpdate = async ({ location, prevLocation }, pluginOptions) => {
  const commentContainer = document.getElementById("commentContainer")
  if (commentContainer && location.path !== "/") {
    // insert header
    // insert comment list
    commentContainer.appendChild(createCommentForm())
  }
}

Users need to be told what is happening — that input is required, or that the API returned an error. The <span class="feedback" /> element serves that purpose, and a helper updates it by setting the element and inserting a class matching the feedback type: error, info or success.

// ... other helpers
// Sets the class and text of the form feedback
const updateFeedback = (str = "", className) => {
  const feedback = document.querySelector(".feedback")
  feedback.className = `feedback ${className ? className : ""}`.trim()
  feedback.innerHTML = str
  return feedback
}
// onRouteUpdate callback

The helper grabs the element with the querySelector API, updates its className attribute to set the class, and writes its contents via innerHTML before returning it.

Submission is detected by listening for the form's onSubmit event. Empty data should not reach the server, so a feedback message is set and the submit button disabled as needed:

exports.onRouteUpdate = async ({ location, prevLocation }, pluginOptions) => {
  // Appends header
  // Appends comment list
  // Appends comment form
  document
    .querySelector("body .comment-form")
    .addEventListener("submit", async function (event) {
      event.preventDefault()
      updateFeedback()
      const name = document.querySelector(".name-input").value
      const comment = document.querySelector(".comment-input").value
      if (!name) {
        return updateFeedback("Name is required")
      }
      if (!comment) {
        return updateFeedback("Comment is required")
      }
      updateFeedback("Saving comment", "info")
      const btn = document.querySelector(".comment-btn")
      btn.disabled = true
      const data = {
        name,
        content: comment,
        slug: location.pathname,
        website: pluginOptions.website,
      }

      fetch(
        "https://cors-anywhere.herokuapp.com/gatsbyjs-comment-server.herokuapp.com/comments",
        {
          body: JSON.stringify(data),
          method: "POST",
          headers: {
            Accept: "application/json",
            "Content-Type": "application/json",
          },
        }
      ).then(async function (result) {
        const json = await result.json()
        btn.disabled = false

        if (!result.ok) {
          updateFeedback(json.error.msg, "error")
        } else {
          document.querySelector(".name-input").value = ""
          document.querySelector(".comment-input").value = ""
          updateFeedback("Comment has been saved!", "success")
        }
      }).catch(async err => {
        const errorText = await err.text()
        updateFeedback(errorText, "error")
      })
    })
}

document.querySelector retrieves the form, and its submit event is observed. The feedback is cleared to an empty string regardless of what it previously held. If the name or comment field is empty, an error message is set instead. The code then issues the POST to the comments server's /comments endpoint and listens for the response, using the feedback element both to report creation errors and to confirm a successful submission.

Styling And Wrapping Up

Styles live in a new style.css file at the root of the plugin folder:

#commentContainer {
}

.comment-form {
  display: grid;
}

Import it at the top of gatsby-browser.js:

import "./style.css"

The rule makes the form's components fill 100% of the width of their container. With that, every component of the comment plugin is in place and ready to be installed and tested.

Wiring The Plugin Into A Live Blog

To test the comment plugin, create a Gatsby website in the directory immediately above the plugin folder:

// PARENT
// ├── PLUGIN
// ├── Gatsby Website

gatsby new private-blog https://github.com/gatsbyjs/gatsby-starter-blog

Local Installation And Configuration

A link has to be registered before the blog can resolve the plugin, so move into the blog directory first:

cd /path/to/blog
npm link ../path/to/plugin/folder

Inside the blog's gatsby-config.js, add an object with a resolve key whose value is the plugin folder name — here gatsby-comment-server-plugin:

module.exports = {
  // ...
  plugins: [
    // ...
    "gatsby-plugin-dom-injector",
    {
      resolve: "gatsby-comment-server-plugin",
      options: {website: "https://url-of-website.com"},
    },
  ],
}

The website option is passed so the source of comments can be distinguished when they are fetched and saved.

Mounting The Comments

Add the insertion point <section class="comments" id="commentContainer"> to src/templates/blog-post.js. Any suitable position works; placing it after the final hr and before the footer is one option.

Run gatsby develop to bring the site up at https://localhost:8000. Visiting a post such as https://localhost:8000/new-beginnings shows the comment rendered at the insertion point. The comment form gives feedback while it is used to post a comment.

Because the content is static, newly posted comments appear only after the server is restarted.

Taking Stock

The plugin leans on several Gatsby APIs plus its own API files to supply comments, which shows how much functionality a plugin can add to a Gatsby site. Even though it pulls from a live server, comments are persisted as JSON files. Loading comments on demand from the API server is possible, but it would contradict the premise of a static blog with no dynamic content requirement.

The finished plugin is published as an npm module, and the complete code is on GitHub.