Anatomy of a Webhook HTTP Request

An HTTP message is a common means by which two systems, usually a server and a client, exchange data. We typically refer to each HTTP message as an HTTP request or an HTTP response.

Webhook HTTP requests are a specific subset of HTTP requests which transfer data between systems based on events in those systems. Webhooks are used with many event-driven integrations.

When working with our customers, we find that some are new to webhooks and want to learn more about what comprises a webhook HTTP request. If you're in that category, this post should help.

What are the sections of a webhook HTTP request?

A webhook HTTP request generally consists of the following:

• Start line
• Header(s)
• Body (payload)

Start line. Each request has a single start line. It comes at the beginning of the request and includes the method, URL, and version. Here's an example of a start line:

POST /webhook/E474BA38/58E1/4544 HTTP/2

Header(s). Each request can have zero or more headers. Headers usually describe something about the request (such as the type of data or the HTTP client), but you can create custom headers for almost any purpose. Here are example headers:

Host: example.com
user-agent: curl/7.79.1
accept: */*
myapp-hmac-sha1: f237e4a4062590a674b0adc1e84614196aae79f4
myapp-api-key: 90B649F2-70F2-4180-95BC-951F5D832F0D
content-type: application/json
content-length: 188

Body (payload). Each request (except in the case of GET and DELETE) has a single body that could be JSON, XML, or some binary file, though a multipart request may encode multiple types of data into one request. Here's an example of a body:

{
  "orderId": "abc-123",
  "state": "update",
  "updates": [
    { "action": "remove", "item": "widgets", "quantity": 5 },
    { "action": "add", "item": "gadgets", "quantity": 20 }
  ]
}

When we put all the pieces together, a webhook HTTP request (and corresponding response) might look like this:

curl https://example.com/ \
 --verbose \
 --request POST \
 --header 'myapp-hmac-sha1: f237e4a4062590a674b0adc1e84614196aae79f4' \
 --header 'myapp-api-key: 90B649F2-70F2-4180-95BC-951F5D832F0D' \
 --header 'content-type: application/json' \
 --data '{
  "orderId": "abc-123",
  "state": "update",
  "updates": [
    { "action": "remove", "item": "widgets", "quantity": 5 },
    { "action": "add", "item": "gadgets", "quantity": 20 }
  ]
}'

> POST / HTTP/2
> Host: example.com
> user-agent: curl/7.79.1
> accept: */*
> myapp-hmac-sha1: f237e4a4062590a674b0adc1e84614196aae79f4
> myapp-api-key: 90B649F2-70F2-4180-95BC-951F5D832F0D
> content-type: application/json
> content-length: 188
>
* We are completely uploaded and fine
< HTTP/2 200
< accept-ranges: bytes
< cache-control: max-age=604800
< content-type: text/html; charset=UTF-8
< date: Thu, 02 Jun 2022 20:26:44 GMT
< etag: "3147526947"
< expires: Thu, 09 Jun 2022 20:26:44 GMT
< last-modified: Thu, 17 Oct 2019 07:18:26 GMT
< server: EOS (vny/044E)
< content-length: 1256

Examining the start line

Each start line consists of the following:

• Method
• URL
• Version

Method

Request methods (verbs) define the action performed by an HTTP request. At present, HTTP supports eight methods:

• DELETE
• GET
• HEAD
• OPTIONS
• PATCH
• POST
• PUT
• TRACE

However, webhooks only use a subset of these. POST is used most of the time, even when data is being updated or deleted instead of created. Occasionally, we might see a webhook use GET to verify that a webhook endpoint exists. Less commonly, PUT and PATCH are used to modify/replace data. And probably least common of all, some webhooks use DELETE.

URL

The most common URL for a webhook is something like https://example.com/my-webhook.

Some apps append an absolute path to the URL to let you know the record type that's being requested: for example, https://example.com/my-webhook/order-confirmation when an order is confirmed.

The URL with a query string is a widespread pattern for web page requests (such as search engines) where some value is appended to the end of the standard URL https://example.com/my-webhook?param1=Param-Value1&param2=Param-Value2. While infrequent, some apps use query strings to send metadata via the URL instead of in custom headers.

Version

The version is just that, the version of the HTTP protocol used for the request. It will generally be HTTP/1.1 or HTTP/2. While webhook HTTP requests include the version, it usually has no impact and is there to ensure that the HTTP request is valid.

Examining the headers

Headers for a webhook HTTP request may either be default (standard) headers or custom headers.

Default headers

Many headers are default and are automatically generated by the source system. Here are a few default headers commonly used with webhooks:

• Content-Type: Describes the data sent in the body (example: application/json)
• User-Agent: Describes the HTTP client used for the request (example: Mozilla/5.0)
• Content-Length: Defines the size of the request in bytes.
• Accept or Accept-Encoding: Defines the type of response expected.

Custom headers

Custom headers for webhook HTTP requests can vary quite a bit but are used frequently to sign the body, send some other type of authentication (such as an API key), or send other data (such as a Customer-ID) that, for whatever reason, didn't make it into the body of the request. A custom header may also be used for an HMAC signature to secure the webhook endpoint.

Here's an example of a couple custom headers for a webhook HTTP request:

myapp-hmac-sha1: f237e4a4062590a674b0adc1e84614196aae79f4
myapp-api-key: 90B649F2-70F2-4180-95BC-951F5D832F0D

Examining the body

The body of a webhook HTTP request contains the data sent via POST (in most cases) or sometimes PUT or PATCH.

This data is often in JSON format but can also be XML, CSV, a PDF, or any other format you'd like to use. If you need to send several types of data at once, you can set up the body as a multipart body. Doing this allows for files such as PDF or MP3 to be transmitted via HTTP requests alongside JSON, etc. Please note that a multipart body requires a corresponding multipart Content-Type header.

Here's an example of a simple body:

{"renderId":51266,"s3Bucket":"test-customer-renders","status":"complete"}

Here's an example of a multipart body (with the multipart header):

curl 'https://example.io/webhook/' \
  --request POST \
  --header "Content-Type: multipart/form-data" \
  --form person='{"firstname":"Sam","lastname":"McElhaney"};type=application/json' \
  --form photo=@sam.jpeg \
  --form resume=@resume.pdf

> POST /webhook HTTP/2
> Host: example.com
> user-agent: curl/7.79.1
> accept: */*
> content-length: 73686
> content-type: multipart/form-data; boundary=------------------------0c985f7380ec6342

--------------------------0c985f7380ec6342
Content-Disposition: form-data; name="person"
Content-Type: application/json

{"firstname":"Sam","lastname":"McElhaney"}
--------------------------0c985f7380ec6342
Content-Disposition: form-data; name="photo"; filename="sam.jpeg"
Content-Type: image/jpeg

SOME BINARY DATA...
--------------------------0c985f7380ec6342
Content-Disposition: form-data; name="resume"; filename="resume.pdf"
Content-Type: application/pdf

%PDF-1.3
MORE BINARY DATA
%%EOF
--------------------------0c985f7380ec6342--

Conclusion

With an increasing number of companies (from Salesforce to Shopify) implementing webhooks, this technology is fast becoming the standard for SaaS integrations. Webhooks are straightforward to understand and implement, and they are highly flexible – allowing you to make them as simple or complex as the data requires.