After an unspecified "werewolf incident" we have become the new maintainer of the hogwarts.edu
web app.
Our first day on the job begins with Professor Dumbledore approaching us, explaining that his official hogwarts.edu
account has recently begun sending mysterious messages such as "Potter sux, Malfoy rulez" to all the students.
As Dumbledore has an administrator account, this security hole could lead to much worse problems than pranks. He's asked us to fix the vulnerability before someone exploits it to cause more serious damage.
1. Authentication
The first thing we do is look at the server-side code that handles posting messages. It's very simple. Here's what it does:
- Listen for a HTTP request to
"hogwarts.edu/dumbledore/send-message?to=all_students&msg=blahblah"
- Send
"blahblah"
(or whatever themsg
parameter was set to) from@dumbledore
to all students.
There's no attempt to check whether the request actually came from the owner of the @dumbledore
account, meaning any attacker can send a HTTP request to hogwarts.edu/dumbledore/send-message
and it will be treated as legitimate. Possibly our werewolf predecessor thought this would be fine.
To prevent this from happening in the future, we introduce an authentication system.
First we add a Secret Authentication Key to each user's account, which we randomly generate when the user logs in and delete when they log out.
We've heard that cookies have security problems, so we don't go down that road. Instead, when the user logs in, we record this key in localStorage
and have some JavaScript code include it as a header called "secret-authentication-key"
in our (legitimate) HTTP requests.
Next we add a step to our server-side logic to verify the key. Our new process:
- Listen for a HTTP request to
"hogwarts.edu/dumbledore/send-message?to=all_students&msg=blahblah"
- Check for a header called
"secret-authentication-key"
and make sure it matches the Secret Authentication Key we stored in the database for the@dumbledore
account. If it doesn't match, reject the request. - Send
"blahblah"
(or whatever came after themsg
parameter) from@dumbledore
to all the students.
Now when we try to send phony messages as Dumbledore, the server rejects them for lacking the proper authentication key. When Dumbledore himself logs in and tries to send them himself, it works. Huzzah!
2. Cookies
The day after we roll out this new authentication scheme, Professor Snape apparates with a complaint. When he visits hogwarts.edu/snape/messages
to view his private messages, there's now a brief loading spinner before his messages show up. Snape demands that we put it back to the old way, where the messages loaded immediately.
Why did we add the loading spinner? Well, we realized hogwarts.edu/snape/messages
was also unsecured, so naturally we secured it with our new "secret-authentication-key"
header.
The trouble is, when Snape visits hogwarts.edu/snape/messages
the browser doesn't know how to send that custom header in that initial HTTP request to the server. Instead, the server sends back some HTML containing a loading spinner and some JavaScript. The JavaScript reads the key out of localStorage
and makes a second request (this time setting the "secret-authentication-key"
header), which is finally allowed to fetch Snape's messages from the server.
While that second request is processing, all Snape sees is that rage-inducing spinner.
We fix this usability problem by replacing our custom "secret-authentication-key"
header with the Cookie
header. Now when Snape logs in, we no longer use localStorage
- or for that matter any JavaScript at all - to store the key. Instead, our server puts a "Set-Cookie: key_info_goes_here"
header in the response; the browser knows that when it sees a Set-Cookie
header on a HTTP response, it should persist the key on Snape's machine, in the form of a cookie.
Now whenever Snape's browser makes a HTTP request to hogwarts.edu
, it will automatically send the contents of that cookie in a Cookie
header. This is true even for the original HTTP GET
request it makes when Snape visits hogwarts.edu/snape/messages
- meaning that now our server can authenticate him right there on that first request, and serve the messages in the first response without needing a second HTTP roundtrip.
Here's our new process:
- Listen for a HTTP request to
"hogwarts.edu/snape/send-message?to=all_students&msg=blahblah"
- Check for a header called
"Cookie"
and make sure it matches the Secret Authentication Key we stored in the database for the@snape
account. If it doesn't match, reject the request. - Send
"blahblah"
(or whatever came after themsg
parameter) from@snape
to all the students.
Performance problem solved!
3. Cookie GET Vulnerabilities
Wasn't there some reason we hadn't used cookies in the first place? Oh, right. Security concerns.
Sure enough, the day after we roll out our cookie-based solution, Professor McGonagall turns up with a strange story. Just after she visited Draco Malfoy's blog, her official hogwarts.edu
account sent another of those "Potter sux, Malfoy rulez" messages to all students. How could this have happened?
Although cookies solved our performance problem, they also opened us up to a new angle of attack: Cross-Site Request Forgeries, or CSRF attacks for short. (Commonly pronounced "C-Surf.")
Viewing the HTML source code of Draco's blog, we notice this:
<img src="http://hogwarts.edu/mcgonagall/send-message?to=all_students&msg=Potter_sux">
As soon as Professor McGonagall visited his blog, her browser did what it always does when it encounters an <img>
: send a HTTP GET
request to the URL specified in its src
. Because the browser is sending this request to hogwarts.edu
, it automatically includes Professor McGonagall's stored authentication cookie in the Cookie
header. Our server checks to see if the cookie matches - which of course it does - and dutifully posts the malicious message.
Argh!
Avoiding this form of CSRF attack is one reason it's important that all our GET
requests do not result in our server taking any important actions. They should be pretty much read-only, give or take perhaps some logging.
We can fix this by adding a new second step to our list:
- Listen for a HTTP request to
"hogwarts.edu/mcgonagall/send-message?to=all_students&msg=blahblah"
- If it is not a
POST
request, reject it. - Check for a header called
"Cookie"
and make sure it matches the Secret Authentication Key we stored in the database for the@mcgonagall
account. If it doesn't match, reject the request. - Send
"blahblah"
(or whatever came after themsg
parameter) from@mcgonagall
to all the students.
Great! Now the <img>
CSRF attack no longer works, because <img>
only ever results in GET
requests to load the src
. Professor McGonagall should be able to visit Draco's blog again with no problem.
4. Cookie POST Vulnerabilities
Unfortunately, a few days later, Draco has found a workaround. He replaced the <img>
tag with a form instead:
<form method="POST" action="http://hogwarts.edu/mcgonagall/send-message?to=all_students&msg=Potter_sux">
He also put some JavaScript on the page which silently submits this <form>
as soon as the page loads. As soon as Professor McGonagall visits the page, her browser submits this form - resulting in a HTTP POST
which automatically includes the cookie as usual - and our server once again posts the message.
Double argh!
In an effort to make things a bit more difficult, we change the msg
and to
fields from URL query parameters to requiring that this information be sent via JSON in the body of the request. This fixes the problem for another day or two, but Draco quickly gets wise and puts the JSON in a <input type="hidden">
inside the form. We're back to square one.
We consider changing the endpoint from POST
to PUT
, since <form>
only supports GET
and POST
, but semantically this clearly makes more sense as a POST
. We try upgrading to HTTPS (doesn't fix it) and using something called "secure cookies" (still doesn't fix it), and eventually stumble upon OWASP's list of other approaches that do not solve this problem before finally finding something that does work.
5. Enforcing Same Origin
OWASP has some clear recommendations on how to defend against CSRF attacks. The most reliable form of defense is verifying that the request was sent by code running on a hogwarts.edu
page.
When browsers send HTTP requests, those requests include at least one (and possibly both, depending on whether it was an HTTPS request and how old the browser is) of these two headers: Referer
and Origin
.
If the HTTP Request was created when the user was on a hogwarts.edu
page, then Referer
and Origin
will begin with https://hogwarts.edu
. If it was created when the user was viewing a non-hogwarts.edu
page such as Draco's blog, then the browser will dutifully set the Referer
and Origin
headers to the domain of his blog rather than hogwarts.edu
.
If we require that Referer
and Origin
be set to hogwarts.edu
, we can reject all HTTP requests that originated from Draco's blog (or any other third-party site) as malicious.
Let's add this check to our algorithm:
- Listen for a HTTP request to
"hogwarts.edu/mcgonagall/send-message"
- If it is not a
POST
request, reject it. - If the
Origin
and/orReferer
headers are present, verify that they matchhogwarts.edu
. If neither is present, per OWASP's recommendation, assume the request is malicious and reject it. - Check for a header called
"Cookie"
and make sure it matches the Secret Authentication Key we stored in the database for the@mcgonagall
account. If it doesn't match, reject the request. - Send a message from
@mcgonagall
based on the JSON in the request body.
Great! Now if a request comes from outside the browser, it won't have the necessary Cookie
header, and if it comes from inside the browser by way of Draco Malfoy's malicious blog, it won't pass the Referer
/ Origin
Same Origin header check.
Importantly, we should not perform this Same Origin check on all requests.
If we did it on all GET
requests, for example, then no one could link to hogwarts.edu
pages from different websites, as they would be rejected for having a different Referer
! We only want to do this Same Origin check for endpoints that no one should ever be able to access from outside a hogwarts.edu
page.
This is why it's so important that GET
requests be essentially "read-only" - anytime we have to skip this Same Origin check, Draco can use the <img>
trick from earlier to cause the endpoint's logic to run. If all that logic does is return information, then the result will be nothing more than a broken-looking <img>
on Draco's blog. On the other hand, if the result is that messages get sent from the current user's account, that means an attacker can potentially use CSRF to send messages from the current user's account!
6. Second Line of Defense
Although OWASP does not list any known ways an attacker could circumvent this Same Origin Check defense (other than a successful Cross-Site Scripting attack, which must be defended against separately, as such an attack can circumvent any number of CSRF countermeasures), they still recommend "a second check as an additional precaution to really make sure."
One good reason to have a second check is that browsers can have bugs. Occasionally these bugs result in new vulnerabilities which attackers exploit, and it's always possible that someone might someday uncover a vulnerability in a popular browser allowing them to spoof the Origin
and Referer
headers.
Having a second line of defense means that if our first line of defense becomes suddenly compromised, we already have a backup defense in place while browser vendors work on patching the vulnerability.
The easiest to implement of OWASP's recommended supplemental defense measures is Custom Request Headers. Here's how it works.
When the browser sends HTTP requests via XMLHttpRequest
(aka XHR aka AJAX Request) they are forced to obey the Same-origin Policy. In contrast, HTTP requests sent via <form>
, <img>
, and other elements have no such restriction. This means that even though Draco can put a <form>
on his blog which submits a HTTP request to hogwarts.edu
, he can't have his blog use an XHR to send requests to hogwarts.edu
. (That is, unless we've explicitly configured hogwarts.edu
to enable Cross-Origin Resource Sharing, which of course we haven't.)
Great! Now we know that if we can be sure that our request came from a XHR rather than something like a <form>
or <img>
, it must have originated from hogwarts.edu
(assuming a valid Cookie
header, of course) regardless of what the Origin
or Referer
headers say.
By default, there's no way to tell that a request came from an XHR or not. A POST
from a vanilla XHR is indistinguishable from a POST
from a <form>
. However, XHR supports a feature that <form>
doesn't: configuring custom headers.
By having our our XHR set a "Content-Type: application/json"
header (which is a semantically sensible header for us to send regardless, since we are sending JSON now), we will have created a HTTP request that a <form>
could not have created. If our server then checks for a "Content-Type: application/json"
header, that will be enough to ensure the request came from an XHR. If it came from an XHR, then it must have respected the Same-origin Policy, and therefore must have come from a hogwarts.edu
page!
This method is a better Second Line of Defense than a First Line of Defense, because it can be circumvented via Flash. So we definitely shouldn't skip the Origin
/ Referer
Same Origin check! We should use this only as an added layer of defense against a theoretical future vulnerability in Origin
/ Referer
.
Final Process
Here's our final server-side process:
- Listen for a HTTP request to
"hogwarts.edu/mcgonagall/send-message
" - If it is not a
POST
request, reject it. - If the
Origin
and/orReferer
headers are present, verify that they matchhogwarts.edu
. If neither is present, assume the request is malicious and reject it. - Check for a header called
"Content-Type"
and make sure it is set toapplication/json
. - Check for a header called
"Cookie"
and make sure it matches the Secret Authentication Key we stored in the database for the@mcgonagall
account. If it doesn't match, reject the request. - Send a message from
@mcgonagall
based on the JSON in the request body.
This covers our current use case, but there are other things to keep in mind for potential future needs.
- If someday we want to use an actual
<form>
ourselves (instead of an XHR), and we still want a second line of defense on top of the Same Origin check, we can use a synchronizer token. - If we still want to use an XHR but don't want to set a custom header (like
Content-Type
), or use a synchronizer token, we can use a double submit cookie or an encrypted token instead. - If we want to support CORS, well...then we need to totally rethink our authentication approach!
Summary
hogwarts.edu
is now in much better shape. Here's what we've done:
- Introduced an authentication system to prevent attackers from impersonating users.
- Used cookies to do this in a way that does not require two HTTP roundtrips (with a loading spinner in between) to view pages with private information, like a page listing a user's private messages.
- Defended against
<img src="some-endpoint-here">
GET
CSRF attacks by requiring that endpoints which make changes to things use HTTP verbs other thanGET
. (In this case, we usedPOST
.) - Defended against
<form>
POST
CSRF attacks by checking that theOrigin
and/orReferer
headers matchhogwarts.edu
(and rejecting the request if neither header is present). - Added a second line of defense against future potential
Origin
and/orReferer
vulnerabilities by requiring that theContent-Type
header be set toapplication/json
.
With all of these put together, we now have some solid defenses against the dark art of CSRF attacks!
If you found this useful, check out the book I'm writing for Manning Publications. I've put a ton of time and love into writing it!
Top comments (14)
It's now 2019 and yet IMO this is the best article on CSRF out there 😁
I have this in my favorites and I check it every now and then.
I have been to hogwarts.edu/. Apparently the maintainer has already been severely hacked since the hackers suceeded in taking control of the domain name and unregistering it. So I'm not sure if I should listen to the maintainer because his own security seems lacking.
as far as my knowledge goes: there was a bug to add custom headers without a pre-flight in flash in 2013/2014.
bugs.chromium.org/p/chromium/issue...
bugs.chromium.org/p/chromium/issue...
it could happen again with any other plugin. Therefore implementing tokens is not only second-line, but should be first-line of defense :)
It looks like the Origin/Referer check would have prevented these though, yeah? (I think these are the Flash hacks that OWASP warned about.)
If you are able to set referer/origin, the check would be useless. The only thing why the check works: you can't set certain headers without a preflight because of CORS restriction.
BUT: you are right. This particular bug does not affect the referer/origin check, because some headers are blacklistet in flash.
Just wanted to display, that it happened before and planing a fail in a security system, because of another software is never a good idea.
So implementing one solution (token), instead of a solution which could break, and one solid one, is more cost effective.
Tokens are implemented in nearly every framework. Using them are most of the time the easier option.
But i like your writing, and that you supply all the information :)
Wow! How vivid this is! An I the only one who attempt to go to
hogwarts.edu
? This post is so fun to read!Very nice topic. And I love your pictures...very cool ;)
My old websites have totally been a subject of these attacks o_O
This was great, thanks for writing. Reminds of the Hogwarts IT guy tumblr.
Very well written Richard! Even though I'm not familiar with HP terminology :).
Great story-telling on this technical (and often overlooked) problem! Thanks.
A great post on csrf. 👏👏