Introduction

A webhook is a method of responding to a change in a system using a custom callback in your web applications. Webhook requests are triggered from SaaS applications like Shopify, Stripe, etc., to an API endpoint in your application, and if these requests increase rapidly, you will need a way to asynchronously process requests to ensure that your server is not overloaded.

In this post, we will be taking a look at Hookdeck and Amazon EventBridge, two services that offer systems for asynchronous processing, to find out which is the best option for processing webhook requests.

Hookdeck and how it works

Hookdeck is an Infrastructure as a Service system for processing webhooks. Hookdeck provides a message queue that asynchronously processes webhooks by ingesting webhook requests from your SaaS applications, and distributing them to your callback endpoint based on the load your API can handle. With Hookdeck, you also get a logging system for your requests, request filtering, retries, and a dashboard to search and troubleshoot your webhooks.

Amazon EventBridge and how it works

Amazon EventBridge is an Infrastructure as a Service system for event-driven architectures. It uses an Event bus to ingest events from event sources, and routes them to the appropriate event consumers. The event sources and event consumers include AWS services, SaaS applications and HTTP endpoints. Apart from the default event bus in every instance, EventBridge also allows you to create a custom event bus for peculiar needs of your system. EventBridge also offers filtering, monitoring and auditing of events.

My experience working with webhooks on EventBridge vs Hookdeck

To get a first-hand experience with these two, I decided to set up webhooks from my Shopify store on both AWS EventBridge and Hookdeck.

EventBridge uses events, so you cannot make HTTP webhook requests to it. Thus, I will be using the architecture below to get EventBridge to work with webhooks (check out this article for more details about this architecture).

I will be sharing my experience using the following criteria:

• Setup requirements
• Number of steps to complete the process
• UI/UX
• Time to complete the setup

Setup requirements

AWS EventBridge

In order to implement the above architecture and route my Shopify webhooks through EventBridge to my backend API, the following components are required:

• An AWS Lambda function to produce and publish events into EventBridge's default event bus
• An AWS API Gateway to enable HTTP requests targeting my (event producer) Lambda function
• An AWS Lambda function to consume events from EventBridge's default event bus and then call my backend API
• Knowledge of one of the supported languages for writing Lambda functions (I used Nodejs, but PHP, Java, Ruby, C# and Go are also supported)
• An HTTPS endpoint to my backend API

Hookdeck

To achieve the same webhook routing using Hookdeck requires the following component(s):

• An HTTPS endpoint to my backend API

Number of steps to complete the process

AWS EventBridge

Brace yourself because this is a huge one. Honestly, I have cut this list down to remove all the extra steps I took to troubleshoot the entire setup, yet it still feels like a lot of steps. Here are the main steps I took to get webhooks working on EventBridge:

• Set up IAM Role for services in this architecture
  • Create the IAM role
  • Add policies/permissions for
    • Publishing events to EventBus
    • Creating log groups on Cloudwatch
    • Publishing logs to Cloudwatch
  • Add the IAM role to your AWS user
• Create (event producer) Lamdba function
  • Create Lambda function using the Author from scratch option
  • Write the Lambda function code to:
    • Receive the webhook request body
    • Build an event object defining the DetailType , EventBus and Source parameters
    • Add the webhook request body as the Detail parameter for the event object
    • Use the AWS SDK to publish the event object to EventBridge
  • Deploy the function
• Create an API Gateway
  • Create a new REST API Gateway
  • Add a new resource
  • Add a POST method to the resource
  • Link the POST method to the Lambda function created above
• Create (event consumer) Lambda function
  • Create the same function as the event producer and add code to do the following
    • Extract the event from the EventBridge default bus to access the webhook request body
    • Make a POST request to your backend API with the webhook request body
  • Deploy the function
• Setup EventBridge
  • Create a new Rule for the EventBridge default bus
  • Add an event matching pattern that corresponds to your DetailType and Source from the event producer Lambda function. This will ensure that only events that match the DetailType and Source will be pushed into the event bus
  • Add your event consumer Lambda function as the target for the Rule created
• Test your setup

Phew! Now that's some setup.

Hookdeck

Now on to Hookdeck. Here is a list of steps I took to successfully get my Shopify webhooks to be processed through Hookdeck's message queues:

• Create a new connection
  • Name your connection (for me this was Shopify Store Hooks)
  • Enter destination label (for me this was My production API)
  • Enter destination URL (your backend https endpoint)
  • Deploy Connection (click the Create Connection)
• Replace the endpoint in Shopify with the one generated by Hookdeck after the connection has been created.

Done! That's all there is to it.

UI/UX

The next criteria I used to judge my experience was user interface and user experience, which often depend on each other. So let's look at what to expect from these two services as far as the number of screens and forms there are to go through when setting up webhooks. Please note that these are minimum estimates based on my demo, and you might have a different number depending on how you accomplish the task.

AWS EventBridge

• Creating a Lambda function: 2 screens, 1 form, 3 compulsory fields (double this for both Lambda functions)
• Creating the API gateway: 5 screens, 4 forms, 6+ compulsory fields (depends on options)
• Setting up EventBridge: 2 screens, 1 form, 5 compulsory fields

In total, we have 11 screens, 7 forms, and 17+ compulsory fields. I intentionally left out the number of screens it takes to set up IAM just to have a fair comparison. You might also end up moving through more screens and forms while troubleshooting.

Hookdeck

• Creating a connection: 1 screen, 1 form, 3 compulsory fields
• Replacing your endpoint in Shopify: 1 screen, 1 form, 1 compulsory field

In total, we have 2 screens, 2 forms, and 4 compulsory fields. This is obviously a less laborious user experience.

Time to complete the setup

This is one metric that will not be exactly the same for everybody, but I will try and breakdown my experience to give some insight into how much time you should expect to spend setting up webhooks on each platform.

AWS EventBridge

As a first-time user, I had to take time to get some knowledge on how to set it up, even though I perfectly understand what it does. I am not including the time spent on Youtube videos and documentation pages, however I believe this is also an important consideration for anyone using EventBridge for the first time.

I began setting up the architecture at 7pm and finished configuring all the moving parts by 10pm (3 hours later). Looking at the steps above, you might think that doesn't correlate but there was quite a learning curve to making sure everything is correctly set up, which adds to the overall time you spend on it.

Then I moved to the next step, which was testing the entire setup to ensure that everything worked fine. There, I ended up spending another 2 hours troubleshooting the whole process. If you're familiar with AWS, you might guess that IAM policies/permissions are the reason for such an exhausting process.

I was humbled by IAM and was super exhausted after figuring it out eventually. This might not be your issue while setting up, but be conscious that you might have to create some room for more time troubleshooting your setup

Hookdeck

Filling one single form that mainly required my https endpoint was quite a straightforward task on Hookdeck. It took me around 10 minutes to go through the whole process, from Hookdeck to updating my Shopify store, and an extra 5 minutes testing the setup from Shopify to my backend API.

Conclusion

So there you have it, my experience working with webhooks on both AWS EventBridge and Hookdeck. I will leave the final judgement to you however obvious it may seem, but my take away is that if you want the following...

• An all-in-one tool that is quick to set up and does not require advanced systems design knowledge or ties to a certain cloud provider
• To save time setting up webhooks
• A simple onboarding process for other members in your team that want to setup webhooks for asynchronous processing
• A CLI to receive webhooks locally for the purpose of debugging and troubleshooting
• A collaborative platform where your webhook processing and information can be accessible and manageable by members of your team
• An all-around pleasing user experience working with webhooks

…then Hookdeck is the way to go. Get started today for free.