Skip to main content

When to use Webhooks and alternatives

If you offer an API but you do not have webhooks your users will be forced to implement a polling framework to consume and construct events. This is going to be a significant effort for the user as there is so much work to do from ensuring they do not get rate limited across their whole service to ensuring that API responses can be used to construct an event possibly forcing them to execute multiple queries in order to get the full picture. If it sounds like it can be annoying it's because it is. Think about developer experience (DX) otherwise as soon as something slightly better comes along your users might churn.

Not yet familiar with what webhooks are? I recommend reading the What are Webhooks article first.

In order to help you decide if you should use webhooks we will go through all of the most common alternatives and their pros and cons.

Webhooks vs Polling​

What is Polling​

Polling is a synchronous conversation pattern where the requestor queries an API provider asking for new changes. If no changes have occurred the API would return an empty response.

It should be noted that in practice unless the API has been designed with polling in mind i.e. the user can request what has happened in the system since a given timestamp or since last successful request. Especially in the case where there are no API endpoints to fetch events, in those cases the requester will be forced to read the service objects of the system and try to figure out if anything has changed since the last time they did the request.

What is Long Polling​

Long Polling is an implementation extension of the polling pattern where the requester queries an API provider asking for new changes. If no changes have occurred the API would keep the connection open and as soon as updates are ready it will send them through the open connection and close it. This is done to ensure the requester can process updates near real-time.

In practice to use Long Polling the provider must support the feature otherwise it will just be standard polling. Also, the connections are not kept-alive for long period of time usually no longer then 30 seconds, after which the connection is closed and a new one is opened.

Advantages​

  • The requester is always in control of how many events they process. Regardless of the volume of changes produced at the provider.

  • The requester machine does not need to be exposed to the public internet to request updates from the provider. By extension this also means that you do not need a domain or to manage SSL certificates.

Limitations​

  • In the case of long polling the requester's underlying machine will need to be always-on for as long as the requester wants to consume changes. This means that it may not be the most suitable for serverless architectures from a cost perspective. Of course where there is a will there is a way.

When to use Polling​

Polling is a very simple pattern to implement and it is very easy to get started with it. It is also very easy to implement rate limiting and other security features. However, it is not very efficient and it is not very developer friendly. If you are building an API that is going to be consumed by a small number of users and you do not expect a lot of traffic then polling might be a good choice. However, if you are building an API that is going to be consumed by a large number of users and you expect a lot of traffic then polling is not a good choice.

Webhooks vs WebSockets​

What is WebSocket​

WebSocket (WS) is an application layer communication protocol for real-time, bi-directional, full-duplex communication over a single, long-lived connection between a client and a server. This is unlike the HTTP protocol, which is request-response based and requires a separate connection for each request. Where as WebSockets allow for a single connection to be maintained between the client and the server, enabling continuous communication.

In a WS connection, either the client or the server can initiate communication and send messages at any time. This makes WS a highly flexible and efficient solution for real-time application, as it eliminates the need for a periodic polling and reduces the amount of network traffic.

Advantages​

  • Increased efficiency in communication as the need for periodic polling is eliminated, reducing the amount of network traffic, and also reduce the resource intensive operation of opening a connection between the server and client for HTTP request on every message sent.

  • Wide support across technology stacks. As the protocol is implemented in the application layer, most languages have a ready to use implementation.

Limitations​

  • WebSockets are much more complex than traditional HTTP communication, as they require more sophisticated handling of errors, timeouts, and other edge cases. Which is why there are many SaaS WebSocket providers which deal with the complexity of providing a production grade WebSocket infrastructure like Pusher Channels, Ably and etc.

  • WebSockets are resource intensive as they require persistent connection between the client and server. This can put a strain on server resources, especially in high-traffic environments.

When to use WebSockets​

WebSockets are an ideal solution for real-time applications that also require bi-directional communication between the clients and server for example:

  • Online multiplayer games: WebSockets provide real-time communication between players, allowing for fast and responsive gameplay.

  • Chat applications: WebSockets allow for real-time messaging between users, making them ideal for chat applications.

  • Real-time data streaming: Delivering real-time updates to clients. For example, stock tickers, dashboard updates, sports scores, and weather updates.

  • Collaborative applications: WebSockets allow multiple users to work together in real-time, making them ideal for collaborative applications such as online document editing, whiteboard applications, and online code editors.

  • Internet of Things (IoT): WebSockets can be used to transfer real-time data between IoT devices and a central server, making them ideal for IoT applications.

To give a more general ideal of when WebSockets should be used, it's when low latency, real-time communication is required, and when the overhead of traditional HTTP communication is too high.

Webhooks vs Pub/Sub​

Publish/Subscribe (Pub/Sub) is a messaging pattern where publishers send messages to channels, and subscribers receive messages from channels they have subscribed to. It is a communication model that decouples the sender (publisher) of a message from the receiver (subscriber) of the message.

In a Pub/Sub system, publishers do not send messages directly to subscribers. Instead, they send messages to a channel, and subscribers subscribe to one or more channels to receive messages. When a publisher sends a message to a channel, the message is broadcast to all subscribers of that channel. This allows multiple subscribers to receive the same message, and it also enables subscribers to receive messages even if they are not actively connected to the channel.

It's important to note that this is a pattern and as such, a lot of it's concepts are borrowed and used in other technologies we've listen like in WebSockets.

Advantages​

  • Communication between the sender and the message recipient is decoupled, which simplifies the architecture of an application and reduces the coupling between components.

  • A Pub/Sub system can scale horizontally by adding more channels and subscribers.

  • Pub/Sub allows for durable channels which means that messages in a channel can be persisted, ensuring that messages are not lost even if subscribers are temporarily disconnected.

Limitations​

  • Pub/Sub systems have higher latency compared to other communication patterns, such as WebSockets, as messages must pass through the message broker that is in charge of managing the channels and subscribers before a message is delivered to subscribers.

  • With the benefit of scalability and decoupling of senders and recipients, a Pub/Sub system can be complex to set up and maintain, especially when dealing with large number of subscribers or channels.

  • While some technologies that implement the Pub/Sub pattern come with order guarantees, it comes with tradeoffs in throughput and latency. Therefore, the lack of order guarantee in traditional set ups can be considered a limitation as some application require messages to be processed in a specific order.

When to use Pub/Sub​

  • Real-time data streaming: Pub/Sub can be used to deliver real-time data streams, such as financial market data or social media updates, to multiple subscribers.

  • Chat applications: Pub/Sub can be used to implement chat applications, where messages are broadcast to all subscribers of a chat room.

  • Online gaming: Pub/Sub can be used to deliver real-time game events, such as player movements or score updates, to multiple players.

  • Event-driven architecture: Pub/Sub can be used in an event-driven architecture to decouple the sender of an event from the receiver of the event, making it easier to add or remove components without affecting the overall system.

  • Microservices: Pub/Sub can be used to implement communication between microservices, where a message produced by one microservice is broadcast to multiple subscribers.

In general, Pub/Sub is a suitable communication pattern for applications that require real-time delivery of messages, scalability, and message persistence.

When to use Webhooks​

We've explored the different communication patterns and their advantages and limitations. And now we can use what we've learned to determine when to use Webhooks. We've prepared a checklist of questions to help guide you in deciding if Webhooks are the right solution for your system.

  • Real-time updates: Is real-time delivery of updates or notifications important to your service?

  • Notification system: Does your service require real-time updates or notifications to be sent based on specific events?

  • One-way Communication: Does your service only need to send data from your service to your customers and not receive any data back?

  • Flexibility: Do you want to allow customers to self-configure?

  • Automation: Do you want to allow your customers to automate tasks based on specific events that happen in your system?

  • Data integration: Do you want to allow your customers to integrate data from your system with other systems?

  • Frequency of updates: Do you expect to send less then 60 updates per minute?

If you answered yes to most of the questions above, then Webhooks are a good fit for your application. If you answered no to some of the questions, then it would be better to spend some more time understanding your problem and evaluating if any of the alternatives like Polling, WebSockets or Pub/Sub would be better suited for your use-case. We offer free consultation to help you determine the best solution for your needs reach out to support@simplyq.io.