What are Webhooks

Webhooks are a messaging communication pattern for distributed systems. They allow one source application to send a message or payload to another application via an application layer protocol like HTTP, when specific event occurs. For example, a payment system might use a webhook to notify a retailer's system when a customer's payment has been processed.

The Application Programming Interfaces (APIs) provide the rules and protocols for how two applications can communicate with each other. And webhooks are one way that an API can be used to send real-time information between applications.

As webhooks are a communication pattern they can be implemented by the source system in any language and variety of different technologies. The same is true for the receiving system.

Characteristics of Webhooks

Real-time communication: Webhooks allow for near real-time communication between applications, as they trigger an event or action as soon as it occurs.
Asynchronous: Webhooks are asynchronous in nature, meaning that they do not require a constant connection between the sending and receiving applications.
Event-driven: Webhooks are triggered by specific events, such as a new data being available or a message being received.
Customizable: Webhooks can be configured to trigger different actions based on the type of event or data received.
Easy to implement: Webhooks are relatively simple to implement, as they typically involve sending an HTTP request to a specific URL.
Flexibility: Webhooks can be used to send and receive data in various formats, such as JSON, XML and etc.
Scalability & Interoperability: Webhooks can handle large numbers of events and can be integrated with various services or platforms, because of their flexibility and simplicity.
Security: Webhooks require authentication and secure transport protocols like HTTPS to protect the data being sent.

Use-cases

Let us explore a more practical example of what a Webhooks system looks like.

Payments provider

Let's explore the following scenario and how a retail website and a payment provider communicate critical information when processing an order via webhooks.

A customer initiates a payment on a retail website: The customer selects a product and proceeds to checkout, entering their payment information.
The payment provider is notified of the payment: The retail website sends a request to the payment provider's API to initiate the payment. The payment provider receives the request and verifies that all necessary information has been provided.
The payment provider processes the payment: The payment provider uses its secure processing systems to validate the payment and process it. If the payment is approved, the payment provider updates its internal systems to reflect the transaction.
The payment provider triggers a webhook: Once the payment has been processed, the payment provider triggers a webhook to notify the retail website that the payment has been processed. The webhook contains information about the transaction, such as its status and any relevant details.
The retail website receives the webhook: The retail website's callback URL receives the webhook and processes the information contained within it.
The retail website updates the order status: Based on the information received in the webhook, the retail website updates the order status in its own internal systems to reflect that the payment has been processed.
The customer is redirected to a confirmation page: The customer is redirected to a confirmation page on the retail website that confirms that their payment was successful. The confirmation page displays the updated order status and any relevant information about the transaction.

In this example, the webhook system is used to notify the retail website of the status of the customer's payment in real-time, without the need for the retail website to constantly poll the payment provider for updates. This allows for a more seamless and efficient experience for the customer.

How Webhooks work

There are two main components to a webhook system: the source application and the receiving application.

Source application

The source application is the application that triggers the webhook. It is responsible for sending the webhook payload to the receiving application's callback URL.

One of the more obvious requirements for the source application is that it needs to have network connectivity to the public internet. This is because the webhook payload will be sent to the receiving application's callback URL, which is often a publicly accessible URL. The protocol used to send the webhook payload is typically HTTP or HTTPS.

For a great developer experience when implementing webhooks, the source application should also provide a way for developers to select the events that they want to receive webhooks for. This is normally done by providing a list of events that the source application can trigger webhooks for, and allowing developers to select which events they want to receive webhooks for when they are providing their callback URL.

Who are the source applications? Most commonly the source applications are SaaS platforms like GitHub, Stripe, Twilio, Shopify, Slack, Discord, customer service platforms and marketing automation tools just to name a few.

Receiving application

For any webhook system to work, the receiving application needs to have a publicly accessible URL that can receive the webhook payload. This is often referred to as the callback URL.

There are a few things that should be considered when implementing an application that consumes webhooks:

Webhooks would be received as an HTTP/HTTPS request that means the receiving application needs to have a web server that can handle HTTP/HTTPS requests. This is often implemented using a web framework like Express, Flask, Django, Rails and etc.
The application should be able to handle the payload that is sent by the source application. This is often done by parsing the payload and storing it in a database for auditing or later processing. Of course implementers can always choose to process the webhook as part of the request-response cycle without persisting the webhook payload. The payload can be in any format, such as JSON, XML and etc. The webhook provider will usually provide the format of the payload in their documentation.
Webhooks are often a one-way communication channel, which means that no response will be processed by the source application. However, it is necessary to provide a response with a 200 status code to indicate that the webhook has been received successfully. Otherwise the source application will retry sending the webhook payload. The webhook provider will usually specify the response code that should be returned in their documentation.
Webhooks should be idempotent. This means that the receiving application should be able to handle the same webhook multiple times without any issues. This is often done by storing the webhook payload or event id in a database and checking if the webhook has already been processed before processing it again.
Webhooks should be processed in a timely manner. This is often done by using a message queue like RabbitMQ, Redis, Kafka and etc. to process the webhook payload asynchronously.
Webhooks should be authenticated and verified to ensure that they are coming from a trusted source. This is often done by using a secret key that is shared between the source application and the receiving application. The receiving application will then verify that the payload is coming from a trusted source by using the secret key to generate a signature and comparing it to the signature that is sent by the source application.
Webhooks a usually sent asynchronously, which means that the receiving application should be able to handle multiple requests at the same time. This is often done by using a web server that can handle multiple requests at the same time, such as Nginx, Apache and etc.

Conclusion

Webhooks are a simple yet powerful way for applications to communicate in real-time. And in a world where everything is moving towards a more distributed and asynchronous architecture, webhooks are becoming more and more popular. As the internet becomes ever more connected, as a developer, it is important to understand how webhooks work and how to implement them in your own applications.