Building a Webhooks System with Event Driven Architecture
source link: https://codeopinion.com/building-a-webhooks-system-with-event-driven-architecture/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Building a Webhooks System with Event Driven Architecture
Sponsorship available! If you'd like to sponsor CodeOpinion.com and have your product or service advertised exclusively on every post, contact me.
Do you need to integrate with external systems or services? Leveraging an event driven architecture enables you to build a webhooks system that can be decoupled from your main application code. Enabling you to call external systems that have subscribed via webhooks in complete isolation from your application code.
YouTube
Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything that is in this post.
In-Process
The simplest approach to building a webhooks system is to make calls in-process to the external HTTP APIs that want to be notified when we actually make some type of state change within our system.
For example, we have an e-commerce application where a user places an order. When this occurs, the application must persist the order data to our database.
Once the order is saved to our database, we then make an HTTP call to the 3rd part external HTTP API.
There are a few issues with this simplistic approach. The first is that because it’s in-process, the latency added to calling the external HTTP is added to the overall request the client made to place the order. Meaning this is blocking the client from getting their result of trying to place their order.
If the external HTTP API accepts our HTTP request but takes a long time to handle it, this could have a very negative impact on our own client. Do we have a timeout? What happens if the external HTTP API is unavailable and we can’t connect? Do we retry? All of this is adding latency to the overall call from our client.
Ideally, we separate placing an order with the integration (webhooks) to the external HTTP APIs.
Publish-Subscribe
We can leverage event driven architecture to decouple the concerns of saving the order and doing our webhooks integrations.
After the application (producer) has saved the order to the database, it then publishes an OrderPlaced event to a topic on our message broker. At this point, the Application can return back to the client.
Our webhook system can be a separate logical and/or physical component that is a consumer of that OrderPlaced Event by subscribing to the Topic on the message broker.
When it consumes the OrderPlaced event, it can then make the HTTP call to the external HTTP APIs.
Now if there is latency with one of the external HTTP APIs that doesn’t affect placing the order from our client. That’s already done and now completely separated since we’ve moved to asynchronous messaging.
Example
An example of this is done in the eShopOnContainers sample application. It does exactly have I have it outlined above.
It has a consumer that handles the OrderStatusChangedToPaidIntegrationEvent. When this event is consumed, it gets all the webhook subscriptions and then sends all the HTTP requests out.
namespace Webhooks.API.IntegrationEvents;
public class OrderStatusChangedToPaidIntegrationEventHandler : IIntegrationEventHandler<OrderStatusChangedToPaidIntegrationEvent> { private readonly IWebhooksRetriever _retriever; private readonly IWebhooksSender _sender; private readonly ILogger _logger; public OrderStatusChangedToPaidIntegrationEventHandler(IWebhooksRetriever retriever, IWebhooksSender sender, ILogger<OrderStatusChangedToShippedIntegrationEventHandler> logger) { _retriever = retriever; _sender = sender; _logger = logger; }
public async Task Handle(OrderStatusChangedToPaidIntegrationEvent @event) { var subscriptions = await _retriever.GetSubscriptionsOfType(WebhookType.OrderPaid); _logger.LogInformation("Received OrderStatusChangedToShippedIntegrationEvent and got {SubscriptionsCount} subscriptions to process", subscriptions.Count()); var whook = new WebhookData(WebhookType.OrderPaid, @event); await _sender.SendAll(subscriptions, whook); } }
Below is the implementation of the IWebhooksSender which makes the HTTP calls to the external HTTP APIs and sends them the webhook data.
namespace Webhooks.API.Services;
public class WebhooksSender : IWebhooksSender { private readonly IHttpClientFactory _httpClientFactory; private readonly ILogger _logger; public WebhooksSender(IHttpClientFactory httpClientFactory, ILogger<WebhooksSender> logger) { _httpClientFactory = httpClientFactory; _logger = logger; }
public async Task SendAll(IEnumerable<WebhookSubscription> receivers, WebhookData data) { var client = _httpClientFactory.CreateClient(); var json = JsonSerializer.Serialize(data); var tasks = receivers.Select(r => OnSendData(r, json, client)); await Task.WhenAll(tasks.ToArray()); }
private Task OnSendData(WebhookSubscription subs, string jsonData, HttpClient client) { var request = new HttpRequestMessage() { RequestUri = new Uri(subs.DestUrl, UriKind.Absolute), Method = HttpMethod.Post, Content = new StringContent(jsonData, Encoding.UTF8, "application/json") };
if (!string.IsNullOrWhiteSpace(subs.Token)) { request.Headers.Add("X-eshop-whtoken", subs.Token); } _logger.LogDebug("Sending hook to {DestUrl} of type {Type}", subs.Type.ToString(), subs.Type.ToString()); return client.SendAsync(request); }
}
More isolation
There’s one issue with the eShopOnContainers example above.
There likely would be multiple webhooks subscriptions that we would need to handle. The issue with this now is that if we do all HTTP requests in the same process, that means that each HTTP request will add latency to the overall processing of the event. What happens if one of the HTTP APIs fails to connect? What happens if one takes a really long time complete? We’re in a similar situation as before where we want to isolate work into a very specific task.
To accomplish this, when we receive the OrderPlaced event, we can look at all the webhook subscriptions we have, create a command, and send it to a queue on our message broker.
As an example, if there were three different webhook subscriptions, we would create three commands, SendOrderPlacedWebhookCommand.
Each command would then also be processed by our webhook system asynchronously where it would pull each message off the queue and then send the HTTP request to the appropriate external HTTP API.
After it finished processing the first command, it would then pick up the second command and perform the HTTP call.
What this now does is separate each individual webhook into its own unit of execution.
Now if one HTTP call fails or takes a long time, it is totally independent of any other.
Webhooks
Using an event driven architecture and messaging can facilitate building a webhooks system that can be very robust, fault-tolerant, resilient, and decoupled from your primary application code.
Event Driven Architecture
Follow @CodeOpinion on Twitter
Leave this field empty if you're human:
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK