Using a Netlify Edge Function to cut down on header bloat by removing HTML-only headers from static assets

I host my site on Netlify, which is a great service. It makes it really easy to add custom headers via either a _headers file or the netlify.toml config file.

A long time ago, I added some security-related headers to my site:

[[headers]]
  for = "/*"
  [headers.values]
    x-frame-options = "DENY"
    x-xss-protection = "1; mode=block"

I used the /* pattern to add these headers to every request coming from Netlify, to make sure I'd covered all my bases. Nobody puts Baby in the corner my website in an iframe!

Fast-forward to tonight, and I'm running my website through the really nice webhint browser extension and I see an area I could improve on: 'Unneeded HTTP Headers':

Unneeded HTTP Headers
no-html-only-headers warns against responding with HTTP headers that are not needed for non-HTML (or non-XML) resources

Interesting! It turns out that a lot of the security-related headers only need to be sent at the document level (HTML responses), not on responses containing images, fonts, scripts and so on. Sending these headers for non-HTML responses does nothing other than waste bytes and make our responses that little bit larger.

So we just remove them right?

Ah, not that fast… Once you've set a header in Netlify with a general path pattern like /* there doesn't seem to be an easy way to undo those headers later on.

Normally in this situation we'd use a more specific rule than /*, but that's hard to do with HTML responses.

There's a big Netlify support thread about this where one piece of advice is to use a rule like /*.html, but that's not much use as the majority of us like our visitors to access our pages using URLs like https://wolstenhol.me/ rather than https://wolstenhol.me/index.html.

I was getting ready to give up until I remembered that that support thread pre-dated Netlify Edge Functions…

Edge Functions to the rescue!

Edge Functions are small scripts that run on 'the edge' – in this case Netlify's CDN infrastructure. They allow us to do the sort of things that you'd need to write Nginx or Apache config to do in the past, and they allow us to do that for static sites on free hosting like Netlify. They let us programmatically intercept network requests coming from our website and going to a user, and they let us modify them, which is exactly what we need to do here.

Let's use a Netlify Edge Function to remove those HTML-only security headers if our function catches a non-HTML response being sent to one of our visitors.

First of all, in our netlify.toml file, let's configure Netlify to let it know about the function we are about to create:

[[edge_functions]]
  path = "/*"
  function = "strip-non-html-headers"

Then, add the JavaScript function below to a file in your repo at this location netlify/edge-functions/strip-non-html-headers.js.

Read through the comments in the function, and let me know if you have any questions in the comments:

export default async (request, context) => {
  // Get the response.
  const response = await context.next();
  const contentType = response.headers.get('content-type');

  // If we can't work out the content-type, or it's HTML
  // then we had better leave the security headers in place.
  // In this case we can return the untransformed response.
  if (!contentType || contentType.startsWith('text/html')) {
    return response;
  }

  // This is a list of headers that only need to be sent on
  // HTML/document responses. It's a waste of bytes to send
  // them on fonts, images, etc as they will have no effect.
  const htmlOnlyHeaders = [
    'content-security-policy',
    'x-content-security-policy',
    'x-ua-compatible',
    'x-webkit-csp',
    'x-xss-protection',
    'x-frame-options',
    // https://webhint.io/docs/user-guide/hints/hint-no-html-only-headers
  ];

  // Loop over the headers of our response…
  response.headers.forEach((value, key, object) => {
    // Some of the security headers make sense to apply to
    // JavaScript files, so we do a bit of logic here:
    if (contentType.startsWith('text/javascript') && (key === 'content-security-policy' || key === 'x-content-security-policy')) {
      // In case of a JavaScript file, Content-Security-Policy and X-Content-Security-Policy
      // can be ignored since CSP is also relevant to workers.
      context.log(`Ignoring as a JS file`);
      return;
    }

    // Otherwise, we delete any headers from the object that
    // contains them within the response.
    if (htmlOnlyHeaders.includes(key)) {
      object.delete(key);
    }
  });

  return response;
};

It gets a little bit complicated there with the revelation that two of the headers actually are useful for JavaScript files, but hopefully the rest of the function makes sense.

You can use a tool like curl (curl -I https://example.com/your-file-name.jpg) or the Network tab in your browser's developer tools to validate that a header like x-xss-protection is now missing from something like a font file or an image.

This is a great example of the power and flexibility that edge functions/edge workers give developers. Rather than wait for Netlify to add a 'override previously set headers' option to their configuration systems we can instead use some standard web APIs (the Response interface that we are using in the Edge Function is part of the Fetch family of APIs) to programmatically alter the response of the CDN ourselves.