Cloud Cannon moves from polling to webhooks

Cloud Cannon builds and hosts static websites from files customers store in Dropbox. When file changes happen on a customer’s Dropbox, those changes need to reach Cloud Cannon’s servers quickly so the live site stays in sync. Originally, the only option was to poll Dropbox’s delta API as fast as possible. The introduction of Dropbox webhooks changed that, allowing instant notification of user file changes.

Cloud Cannon has been using webhooks since the beta release, and the shift has eliminated the delay users previously experienced. Before webhooks, a user would drag a file into Dropbox and wait a couple of seconds for the change to appear on Cloud Cannon. Now, the update is instant.

Setting up a webhook endpoint

Webhook setup starts with configuring a URL endpoint that Dropbox will POST to on every file update from your users. From there, the endpoint performs delta calls on the affected users to retrieve the changes. There are two practical issues to solve before webhooks are useful in production: testing locally and validating requests.

Local development without a public URL

Dropbox calls a public URL, which makes testing webhooks in a local development environment awkward. A tool like ngrok exposes a local web server publicly and works for a single developer, but doesn’t scale to multiple developers working at once. Cloud Cannon built a node.js app for local development that polls the delta API and simulates a webhook call instead.

Verifying requests come from Dropbox

The webhook endpoint itself needs protection from unauthorized callers. To confirm a request genuinely comes from Dropbox, you should verify the request signature sent in the X-Dropbox-Signature header. The signature is generated from the request body using your app secret as the key, which requires a hashing step. Below are working validation examples.

Ruby:

def valid_dropbox_request?(message)
    digest = OpenSSL::Digest::SHA256.new
    signature = OpenSSL::HMAC.hexdigest(digest, APP_SECRET, message)
    request.headers['X-Dropbox-Signature'] == signature
end

Node.js:

var crypto = require('crypto');
 
function isValidRequest(message, request) {
    var signature = request.headers['x-dropbox-signature'],
        hash = crypto.createHmac('SHA256', APP_SECRET).update(message).digest('hex');
 
    return signature == hash
};

Managing load under webhook traffic

Polling made server load predictable. If traffic peaked, Cloud Cannon could lower the polling rate to regain control. Webhooks remove that lever: if 10,000 users happened to update files at once, the endpoint could receive a sustained burst of incoming requests. Dropbox also cuts the connection if the endpoint does not respond within 10 seconds. With these constraints, the answer was to handle webhook calls efficiently so the endpoint never gets backed up.

The approach that works for Cloud Cannon is to perform the delta calls for each user inside the webhook handler, then push the resulting file operations for the user’s site onto a queue to be processed later. The endpoint responds to Dropbox quickly, and asynchronous processing smooths out the actual workload.