The Fundamentals of Webhooks

A webhook is a route created on one application with the expectation that data will be sent to that route periodically from another application.

For a concrete example, let us talk about two applications. The first application is a music streaming service we will call Jukebox; the second application is a credit card provider that we'll call Cashier.

The Scenario

Let us say that in order to play music without advertisements, members in Jukebox can elect to pay a monthly fee charged to their credit card. Now Jukebox knows that it has absolutely no business in handling the sensitive nature of credit card data and opts to use a credit card provider, Cashier, in order to manage the monthly payments.

There are many ways that Jukebox can be configured to handle this sort of setup, but a basic setup would involve the following rules:

1. Jukebox manually keeps track of the last payment date for each user
2. When the user logs in, their last payment date is checked; if it is more than a month ago, Jukebox sends a request to Cashier to check if that user has paid this month
3. If the user has paid, then the user's account on Jukebox is updated to be valid and ad-free
4. If the user has not paid, then every time the user logs in Jukebox has to repeat steps 2 and 3 until the user is paid up

Another alternatives is for Jukebox to implement a scheduler and routinely check for user updates.

We now have an issue where Jukebox relies on knowledge from an external provider, where the data may change at some point or may not change.

A webhook is a term for an inversion of this process, where applications that consume data such as Jukebox create routes on their own web server that will parse data that is submitted from the application that produces data, such as Cashier. Jukebox would have routes setup to receive data from Cashier at any point in time, as soon as there is a change in a user's payment status.

The usage of a webhook will allow Jukebox to have continually up-to-date information, while also dramatically reducing the number of HTTP requests that need to be made to Cashier.

Configuring Cashier to Send Webhooks

When dealing with Webhooks, most of the complexity is on the side of the application that supplies the data. Cashier will allow API users to register a simple URL to send HTTP requests to when credit card charges fail or succeed.

To add webhook capabilities to Cashier, we will first start with creating a list of events that should notify consumer applications when they occur. For now, we will focus on 2 simple events:

• A recurring charge fails
• A recurring charge succeeds

In Cashier's API call to register a credit card for recurring monthly charges, Cashier would generate a token of some sort (such as a UUID) and return that to applications such as Jukebox. It would store that token and credit card information together, as well as a reference to which client owns that token. For example, when Jukebox registers a user to be billed $5.00 each month with a credit card number of 4242-4242-4242-4242, Cashier would generate a token such as 9ff4a588-df1c-49a7-85c3-11a11e7218b1 and store a record in the database that includes the card number, the token, and the time of next billing. In the API call to register this user, Cashier would provide Jukebox with the token of 9ff4a588-df1c-49a7-85c3-11a11e7218b1.

Jukebox would store that token on its user account for the user that submitted the credit card charge for later use. When it comes time for a user's account to be billed again, Jukebox would send a charge to the user's credit card. If the charge succeeds, it would fire off an event that the charge suceeded; if it fails, it would fire off an event that the charge failed.

There are many ways to implement an event system, but we will say that Cashier implements a basic message queue system. When Cashier fires off an event, it is placed in a message queue and consumed by any number of consumer processes.

Each time one of those two events are fired, Cashier will query the API account associated with the credit card charge. In our case, it would lookup Cashier's account information for Jukebox. If Jukebox has a webhook registered, it would send an HTTP request with the status of the payment (success or failure) and the charge token to the URL provided to Cashier that references a URL on Jukebox's servers.

Configuring Jukebox to Accept Webhooks

It is incredibly easy to support webhooks on Jukebox's end. Jukebox would create a route on its own servers that accepts an HTTP Post call. This post call would contain only two pieces of data:

{
  "token": "9ff4a588-df1c-49a7-85c3-11a11e7218b1",
  "status": "succeeded"
}

Each time Jukebox receives an HTTP request on this route, it would update the user's account to be paid (or, not paid if the status was failed), and the user would not be given ads.

Benefits

The benefits of integrating webhooks comes from a dramatic reduction in the number of useless requests from Jukebox to Cashier, and instead allows Jukebox to maintain a simpler setup for handling user states. Jukebox can then assume that until it receives a POST call stating that a user's card was failed to be charged, that the account is paid up and valid.

Cashier attains many benefits in scalability from this structure, because it can now control how often it handles posting requests, and how. Rather than relying on a web servers to handle many API calls into Cashier, it can now use any number of serverless strategies or worker strategies to send out the requests in a performant, scalable manner.

Demonstration

I've created a quick demonstration, where Cashier is simply a constantly running process and using static data, of these interactions on Github.